LogViewPlus Support

Parser detection based on content instead of logfilename

https://www.logviewplus.com/forum/Topic565.aspx

By MatthiasB - 10 Jun 2020

At the moment a parser is only automatically been detected by the filename. It would be nice if this can be extended to detect the parser based on the content.

There can be 2 options:
1) the logfile contains a specific keyword which can be used to identify it
2) the logfile structure fits to the parser configuration and therefore will be selected. In case that multiple parser could fit a popup with dropdown is shown to select the right parser.

Best regards,
Matthias
By LogViewPlus Support - 10 Jun 2020

Hi Matthias,

Having multiple methods of automatically detecting how a file should be parsed is tricky.  Each method will add additional complexity which will need to be "processed" by new users - even if they don't use the functionality.  For this reason, we try to keep things simple.  I am not against adding additional detection methods, but it needs to be simple and add value to a wider audience.  

Have you noticed the 'Parse With" settings when opening a file?  Does this help?
https://www.logviewplus.com/docs/open_file_settings.html

Thanks,

Toby

By MatthiasB - 10 Jun 2020

Hi Toby,

thanks for the fast response and I can understand to keep the software as simple as possible. :-)
The option "Parse With" is what I currently use but dragn'drop is always easier than clicking the "Open" button.

Maybe would it be possible to make if configurable that when you drag'n drop a file on the software and no parser is detected this "Open window" is displayed? Then the parser can be changed manually and no autodetection is used. In case a parser is detected then the parser will be used and no popup is shown.

Best regards,
Matthias
By LogViewPlus Support - 10 Jun 2020

Maybe the drag-drop operation should open a scaled down "Open" dialog whenever the user tries to open new files in this way.  I think I would prefer to do this with the help of a short-cut key by default (CTRL+DROP) for example.  However, this could be configurable in application settings to make it the default.

The same dialog could be shown when a user tries to open a files via the OS.  For example, by double-clicking on a *.log file.

Would this help?

Thanks,

Toby
By MatthiasB - 10 Jun 2020

Hi Toby,

that would be very helpful! :-)
Thanks a lot!

Best regards,
Matthias
By LogViewPlus Support - 10 Jun 2020

Great - I will take a look and see what we can do for the next release.  I will post back here when we have a BETA version available.

Thanks Matthias!

Toby
By LogViewPlus Support - 21 Oct 2020

Hi Matthias,

I just wanted to let you know that we have just released LogViewPlus 2.4.45 into BETA.  This version of LogViewPlus has a new "Quick Open" feature.  If you hold down the CTRL key when dragging and dropping files into LogViewPlus, you get 'Quick Open' menu:



The Quick Open menu has some basic settings, like parser configuration, merging, and the open action.

Hope that helps.  Thanks again for the feedback!

Toby

By Colin Foster - 26 Nov 2020

Can you add this feature to ctrl-clicking a file in the recently used list? Or better still be able to apply a different parser while the file is already open e.g. with ctrl-F5
In other words, any situation when a file is (re)opened, not just drag-and-drop
By LogViewPlus Support - 26 Nov 2020

Thanks for the feedback Colin - I think that's a great idea.  I will take a look and see what I can do.  If this is a small change, we may be able to get it into the next release.

Thanks again,

Toby
By Colin Foster - 27 Nov 2020

And while I think about it, if I override the default parser I would expect LVP to remember that and use my selected parser the next time without asking or without me having to specify it every time.
By LogViewPlus Support - 27 Nov 2020

Hi Colin,

The purpose of the Quick Open form is to override any LogViewPlus settings.  I think it would be confusing to have this dialog used for both overriding and configuring LogViewPlus because the user intent is not clear.

If you would like to open a file with the Basic Parser every time, the best approach would be to apply the appropriate application settings.

Hope that helps,

Toby
By strat68 - 23 Dec 2020

I've encountered the need for what @MatthiasB and @Colin Foster suggest, but also have another idea, though it may be too simple.

The logging format has changed in our last release.  However the file naming convention has not.  So filename.log can have two formats depending what release I am looking at.

I try to get around this with the up and down arrows that remind me of outlook rules in Settings>Parser Mappings.  But what about a user that supports multiple applications (rather than minor release variants of the same app)?

How about in the Parser Mappings section under settings, have an enabled column next to isRegex? (same kind of checkbox).  The user can take ones out of the process by unchecking it when they don't need it.  Default enabled of course, so it doesn't change the user experience for those who are looking at a steady set of parsings.

Thanks!
By LogViewPlus Support - 23 Dec 2020

First of all, let me say that the current BETA release (v2.5.6) includes support for using the CTRL key when opening from the Recent History menu as suggested by Colin.

Given the amount of feedback on this thread it seems clear to me that more thought needs to go into this area.  It is a limitation in our approach that has been there since day one, but it may be time for a rethink.

The enabled idea makes sense and I think it is a good one, but I am not really sure it provides any advantage over command ordering.  Adding redundant features can sometimes make sense, but in this case I think it may lead to confusion from users who are not familiar with this problem.

I think a more solid solution would look like Matthias's original suggestion - some kind of 'test parse' which is used to determine the appropriate parser configuration.  Thinking about it, this is really just saying the relationship between file name patterns and parsers should be one-to-many and not one-to-one.  This relationship change can be handled by the existing configuration - it is only the assumptions made in code that need to change.

This change would not be trivial, but it's an interesting idea.  I think we need something better than the current approach and I am happy to make this change, but I am not able to commit to a timeline.  It will likely be several months.  I will post back here as soon as something is available.

Thanks to all for helping me see the light!  :-)

Toby
By Colin Foster - 28 Dec 2020

Another solution, which would help with different formats over time, and different formats from different sources in different folders, would be to have a file, lets say in the form <parsername>.lvp, that would allow LVP to pick/override the default parser on a folder by folder basis.

Our company's main application generates half a dozen different CSV and TSV files, but it groups them into different folders, so a special file in each folder would automatically choose the right parser for me.

Yet another alternative. Allow a parser to match against the file folder name as opposed to the file.
By LogViewPlus Support - 28 Dec 2020

Thanks Colin - that's an interesting idea.

I particularly like that this solution would handle a 'team' scenario where a user has never seen these log files before and therefore would not have any appropriate configuration.

One concern would be navigating the directories.  For example, from an SFTP perspective navigating up a directory tree can be expensive.  Especially if you hit an exception on one of the root directories (access denied for example).  Users who are not interested in this feature would want to avoid the performance hit.  For this reason, I think the parser configuration file would need to be in the same directory as the log file.

Directory detection is useful, but it would require individual configuration.  If directory names were not fully qualified, you may have some of the same conflicts.  (For example, *logdir* is too generic and should probably not be allowed).

I suggest we stick with the 'one-to-many / auto-detection' approach for now.  If this approach works well, I think these other suggestions may be redundant.  I also like that this approach does not add any additional configuration options (this keeps things simple for most users).

Thanks again,

Toby
By LogViewPlus Support - 7 May 2021

We have now released LogViewPlus v2.5.17 as a BETA release.  This release includes a new 'heuristic' log parser detection strategy:



Heuristics work by finding all matching patterns for a given log file name. The target file will then be scanned using all matching patterns. The final pattern will be selected based on the criteria below (listed in order of importance).
1. The number of log entries parsed.
2. The number of arguments in the pattern. More detail is better.
3. The configuration position in the list.

Heuristics processing is slightly slower than First Match and it may be difficult to determine 'why' a certain parser configuration is chosen. For these reasons, the First Match algorithm is still recommended and will be the default option.

We have discussed a number of different approaches in this forum thread.  I would suggest trying the new implementation and seeing how it meets your needs.  Please do post feedback here, but if you have any issues consider raising a new support request - just to keep the conversation focused.

Hope that helps,

Toby
By keli - 29 Sep 2021

I've learned yet another useful feature by browsing the forums today BigGrin
Since I mostly just drag&drop files into LVP (or use the context menu to open them) I never noticed I could pre-define a parser. Now I hope I won't forget by the next time I need this feature, to drop files with Ctrl pressed. Thank you!

Still, I would give a vote to the ability of changing parsers on an already loaded file. For me a one-time, manual option would be just fine for the occasional mis-applied parser, or perhaps a file name pattern that hasn't been updated, whenever I forgot that Ctrl+dropping the file opens up the option for a custom parser Smile
By LogViewPlus Support - 29 Sep 2021

Thanks for the feedback Keli - glad to hear you are finding the forum helpful!  Smile

For the 2.5.37 release we revisited the log file context menu and consolidated commands into a new 'File System' menu.  This helped with usability in a number of ways including giving a space for new commands which used to be shortcut-only (like 'Explore Directory'). 

It might makes sense to do something similar with the 'Reload' command.  A 'Reload' menu with 3 commands: Reload, Reload with Encoding, and Reload with Parser Configuration.  I think that might work.  It is certainly something to think about.

Thanks again,

Toby