Update to 2.5.37 causes some logs to be parsed as having the wrong Level

I have the following parser mapping for Unreal log files:

[%d{yyyy.MM.dd-HH.mm.ss:fff}][%3t]%c: %p: %m%n
[%d{yyyy.MM.dd-HH.mm.ss:fff}][%3t]%c: %m%n
%c: %m%n
%m%n

It used to work very well, until recently where (one of) the update broke the Level such that some logs without a Level are parsed as having a Level. If I were to guess, it looks like LogViewPlus is searching for the Level with wildcards in between each character e.g., "e*r*r*o*r". Just a guess.

In the following screenshot, you can see that the log is clearly not 'Severe'. All the letters (underlined red by me) are present though (and I found that to be the case with all the logs that have incorrect Level.

Hi Samaursa,

It looks like this log entry is being parsed with the first match - which is the first pattern provided.  In this case, the log level is "Selected Device Profile".  This isn't much of a log level, so LogViewPlus is doing the best it can and matching Selected = S = Severe.

I think your log file might parse better if you simplify your parser configuration to:

[%d{yyyy.MM.dd-HH.mm.ss:fff}][%3t]%c: %m%n

Hope that helps,

Toby

I have another screenshot where the severity is not detected incorrectly.



But I realized that the difference was the missing Local Log Level. After adding the local log levels, the issue disappeared. However I assume it will still identify something as one of those log levels if it can find the letters using wildcard matching.



Regardless, is it possible to prevent LogViewPlus from wildcard searching for the Log Level? i.e. %p should not find "Severe" in the text "Selected Device Profile"?

Hi Samaursa,

LogViewPlus doesn't have a setting like that at the moment.  There are a few problems with implementing such a setting unless it was done on a per-parser level and I would hesitate to add additional complexity to those settings.

LogViewPlus matches S to Severe for performance reasons.  It has been configured to believe it has found the log level - the only question is, which level?  There is only one that begins with S, so further processing is unnecessary. 

The problem here is that it hasn't found the log level.  The easiest fix for this would be to change the configuration.

I could maybe add a flag to "require an exact match" but this wouldn't address the configuration issue.  A wildcard / default log level might also work, but again you would have the wong data in the log level column.

Maybe a custom post- processor would be a better solution for you?
https://www.logviewplus.com/docs/post_processors.html

Hope that helps,

Toby

If I simplify the parser the way you suggested earlier I lose the Log Level. This is because Unreal is inconsistent with some of the logs where there is no Log Level unless it's a Warning or Error (i.e. it does not have a Log Level for regular logs). Here's a screenshot where you can see that the Log Level is no longer being parsed correctly and I do not get a Log Level at all:



With the way I have set it up I get the Log Level correctly for the lines which do have the Log Level



Your suggestion of the "require an exact match" option would fix this issue I think.

Yes, the optionality of the log level field is what is causing the issue.  LogViewPlus expects the log entry metadata to be consistent across all log entries.

> "require an exact match" option would fix this issue

If we added this flag, your log entry could show up as INFO.  However, your log level would still read "Selected Device Profile" - which is incorrect.  The log entry message would read "[WindowsEditor]" - which is also incorrect.

Should it not show up as NONE? That is what I have in the logs with the pattern I posted in my original post:

You would probably want to specify a default as further configuration.  So the setting would be something like "require an exact match or default to ...".  However, the data that is written to the LogEntry->Priority field would be "Selected Device Profile" as the parser configuration instructed.

The problem is "Selected Device Profile" is not a priority even though the configuration says it is.

I understand, however in the above screenshot (pasted again below) should the field not say, "Process Physical Memory" but instead it correctly identifies it as "NONE"?

Right, but where did "Process Physical Memory" go?  It is not part of the log entry message which begins "302.48 MB...".  So the message is wrong. The log level is likely showing as 'NONE' because there are no log levels beginning with 'p', so LogViewPlus could not find a match.  The 'NONE' in this case is an indication of a problem in the configuration.

I see what you mean. The "Process Physical Memory" got consumed and is not part of the Message column.

It would be great to have the option similar to what you suggested "require an exact match"

Thanks for the feedback.  

It would be great to handle this scenario better.  LogViewPlus is close to achieving what you want without custom code, but the last 5% is still needed.  I will try and think of a way that this can be done without complicating the configuration.  

Okay, great thank you.

Looking forward to a solution for this issue with Unreal logs.

Hi Samaursa,

I just wanted to let you know that we have now released LogViewPlus v3.0.1 as a BETA release.  This release allows you to define the priority levels in the parser explicitly.  This gives the parser a chance to fail if one of the specified priorities is not detected.  This failure will allow your second patten (which does not have a priority) to be used.  Log entries without a priority will still be displayed as None. 

To specify the priority levels, you need to use the new -expectedValues parameter with a comma separated list of priorities.  Values are case sensitive.  For example:

%p{-expectedValues:info,warn,error}

Hope that helps.  Thanks again for bringing this issue to our attention.

Toby

Thanks for adding the feature! I'll try it out.