Parser detection based on content instead of logfilename


Author
Message
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Hi Colin,

The purpose of the Quick Open form is to override any LogViewPlus settings.  I think it would be confusing to have this dialog used for both overriding and configuring LogViewPlus because the user intent is not clear.

If you would like to open a file with the Basic Parser every time, the best approach would be to apply the appropriate application settings.

Hope that helps,

Toby
strat68
strat68
New Member (32 reputation)New Member (32 reputation)New Member (32 reputation)New Member (32 reputation)New Member (32 reputation)New Member (32 reputation)New Member (32 reputation)New Member (32 reputation)New Member (32 reputation)
Group: Forum Members
Posts: 5, Visits: 41
I've encountered the need for what @MatthiasB and @Colin Foster suggest, but also have another idea, though it may be too simple.

The logging format has changed in our last release.  However the file naming convention has not.  So filename.log can have two formats depending what release I am looking at.

I try to get around this with the up and down arrows that remind me of outlook rules in Settings>Parser Mappings.  But what about a user that supports multiple applications (rather than minor release variants of the same app)?

How about in the Parser Mappings section under settings, have an enabled column next to isRegex? (same kind of checkbox).  The user can take ones out of the process by unchecking it when they don't need it.  Default enabled of course, so it doesn't change the user experience for those who are looking at a steady set of parsings.

Thanks!
Edited 4 Years Ago by strat68
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
First of all, let me say that the current BETA release (v2.5.6) includes support for using the CTRL key when opening from the Recent History menu as suggested by Colin.

Given the amount of feedback on this thread it seems clear to me that more thought needs to go into this area.  It is a limitation in our approach that has been there since day one, but it may be time for a rethink.

The enabled idea makes sense and I think it is a good one, but I am not really sure it provides any advantage over command ordering.  Adding redundant features can sometimes make sense, but in this case I think it may lead to confusion from users who are not familiar with this problem.

I think a more solid solution would look like Matthias's original suggestion - some kind of 'test parse' which is used to determine the appropriate parser configuration.  Thinking about it, this is really just saying the relationship between file name patterns and parsers should be one-to-many and not one-to-one.  This relationship change can be handled by the existing configuration - it is only the assumptions made in code that need to change.

This change would not be trivial, but it's an interesting idea.  I think we need something better than the current approach and I am happy to make this change, but I am not able to commit to a timeline.  It will likely be several months.  I will post back here as soon as something is available.

Thanks to all for helping me see the light!  :-)

Toby

Colin Foster
Colin Foster
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 4, Visits: 26
Another solution, which would help with different formats over time, and different formats from different sources in different folders, would be to have a file, lets say in the form <parsername>.lvp, that would allow LVP to pick/override the default parser on a folder by folder basis.

Our company's main application generates half a dozen different CSV and TSV files, but it groups them into different folders, so a special file in each folder would automatically choose the right parser for me.

Yet another alternative. Allow a parser to match against the file folder name as opposed to the file.

LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Thanks Colin - that's an interesting idea.

I particularly like that this solution would handle a 'team' scenario where a user has never seen these log files before and therefore would not have any appropriate configuration.

One concern would be navigating the directories.  For example, from an SFTP perspective navigating up a directory tree can be expensive.  Especially if you hit an exception on one of the root directories (access denied for example).  Users who are not interested in this feature would want to avoid the performance hit.  For this reason, I think the parser configuration file would need to be in the same directory as the log file.

Directory detection is useful, but it would require individual configuration.  If directory names were not fully qualified, you may have some of the same conflicts.  (For example, *logdir* is too generic and should probably not be allowed).

I suggest we stick with the 'one-to-many / auto-detection' approach for now.  If this approach works well, I think these other suggestions may be redundant.  I also like that this approach does not add any additional configuration options (this keeps things simple for most users).

Thanks again,

Toby
Edited 4 Years Ago by LogViewPlus Support
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
We have now released LogViewPlus v2.5.17 as a BETA release.  This release includes a new 'heuristic' log parser detection strategy:



Heuristics work by finding all matching patterns for a given log file name. The target file will then be scanned using all matching patterns. The final pattern will be selected based on the criteria below (listed in order of importance).
1. The number of log entries parsed.
2. The number of arguments in the pattern. More detail is better.
3. The configuration position in the list.

Heuristics processing is slightly slower than First Match and it may be difficult to determine 'why' a certain parser configuration is chosen. For these reasons, the First Match algorithm is still recommended and will be the default option.

We have discussed a number of different approaches in this forum thread.  I would suggest trying the new implementation and seeing how it meets your needs.  Please do post feedback here, but if you have any issues consider raising a new support request - just to keep the conversation focused.

Hope that helps,

Toby
Zoltan Kelemen
Zoltan Kelemen
Gaining Respect (103 reputation)Gaining Respect (103 reputation)Gaining Respect (103 reputation)Gaining Respect (103 reputation)Gaining Respect (103 reputation)Gaining Respect (103 reputation)Gaining Respect (103 reputation)Gaining Respect (103 reputation)Gaining Respect (103 reputation)
Group: Forum Members
Posts: 21, Visits: 97
I've learned yet another useful feature by browsing the forums today BigGrin
Since I mostly just drag&drop files into LVP (or use the context menu to open them) I never noticed I could pre-define a parser. Now I hope I won't forget by the next time I need this feature, to drop files with Ctrl pressed. Thank you!

Still, I would give a vote to the ability of changing parsers on an already loaded file. For me a one-time, manual option would be just fine for the occasional mis-applied parser, or perhaps a file name pattern that hasn't been updated, whenever I forgot that Ctrl+dropping the file opens up the option for a custom parser Smile
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Thanks for the feedback Keli - glad to hear you are finding the forum helpful!  Smile

For the 2.5.37 release we revisited the log file context menu and consolidated commands into a new 'File System' menu.  This helped with usability in a number of ways including giving a space for new commands which used to be shortcut-only (like 'Explore Directory'). 

It might makes sense to do something similar with the 'Reload' command.  A 'Reload' menu with 3 commands: Reload, Reload with Encoding, and Reload with Parser Configuration.  I think that might work.  It is certainly something to think about.

Thanks again,

Toby
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Login

Explore
Messages
Mentions
Search