Hello,

I work for a medium-sized developer and I'm currently investigating the feasibility of us using LogViewPlus to view and debug the logs generated by the software we make. It has lots of killer features that would be very useful for us, but unfortunately at least one killer drawback.

The software being developed generates a text log, and overwrites this log each time the application is started. By default, LogViewPlus keeps the output from old logs in its view, so if you run the application five times and then try to filter the view, you see output from the previous four runs along with the current run. This confuses the output and is not desired.

I found the option in LogViewPlus settings to "Clear all log entries when the log file is rolled or recreated" which looks like it should solve this. However, when this option is enabled, LogViewPlus now shows the following prompt when the software being developed is restarted.



LogViewPlus then has to be restarted, or the logfile has to be re-added to its view, in order to view logs again.

A potential factor is that the log-generating software, when restarted, copies the old log file and renames it within the same folder. For example, if the application is set to write logs to client.log, when it starts up it will copy the previous client.log to client-backup-<timestamp>.log and then start a new client.log file. This behaviour may or may not be related.

Has anyone seen this issue before, and/or know how to fix it?

It might be that your application is writing an invalid log entry and this is confusing LogViewPlus.  For example, a header at the top of the log file?  LogViewPlus can correct for this if matching log entries are then found, but there could be some kind of race condition between the header being written, the log format being detected and then additional log entries written.  The speed of the writes / read operations may explain why you only notice the problem with the "Clear all log entries" flag enabled.

The file format *.log is also very generic.  I suspect it is shared between your two parser configurations.  Is there any way to narrow down the file name pattern?  Possibly by using a regex?

I would like to get to the bottom of this one, but a work-around might be to use a rule to clear the log.  This has the advantage of being specific to your log parser (the application setting would apply to all logs opened by LogViewPlus).

The invalid log entry theory is promising. Our application does indeed write out several hundred lines of logfile entries in a different format, prior to its first line that conforms to the parser mapping.

First, there is a line in the form

Log file open, 04/04/24 15:25:27

Then, hundreds of lines in the form

Logger: Example log message

Then, the rest of the log entries in the main format, an example of which I give here:

[2024.04.04-14.25.28:584][123]Logger: Example log message

LogViewPlus only shows the log entries that conform to the main format. If I select the first of these, however, in the pane that shows the full log message it is displaying all log entries from the start of the file, as if it's interpreting all of them as one single entry (this view gets truncated as there are lots of them).

I followed the instructions on the link you gave to clear logs automatically if "Log file open," is found, but it didn't work, unfortunately.

I also tried to add "Logger: message" as a separate parser mapping that it can fall back on but it seems that some kind of timestamp is mandatory, so the wizard didn't let me proceed. I was able to use the wizard to make a separate parser configuration with a timestamp, and then remove the timestamp field manually, but this didn't work either.

That is all expected behaviour.  LogViewPlus works hard to never hide a log entry - even if that log entry cannot be parsed.  In this case, the header is considered part of the original log entry belonging to the first parsed log entry in the file.  This way, header information is available when you search for it even though it is not visible in the grid.

> if "Log file open," is found, but it didn't work

That is a little odd.  The log line cannot be accurately parsed, so this may be causing some problems (especially if there is a delay in writing the first log entry).  Still, I would expect the text to be found and the action to work.  When you filter for 'log file open', do you see a matching log entry?  It is the matching log entry which triggers the action.

Fortunately, there is a timestamp in your header.  I would suggest using a multi-pattern configuration like:

[%d{yyyy.MM.dd-HH.mm.ss:fff}][%t]%c: %m%n
%m, %d{dd/MM/yy %H:mm:ss}%n

Your log should then parse the header out separately.  Something like:


I suspect this will resolve the issue with both of the 'clear' operations.

Hope that helps,

Toby

Firstly, this:

When you filter for 'log file open', do you see a matching log entry?

Yes I do. A filter for "Log file open," shows just the first properly-formatted line, with the detail view showing a truncated view of the several-hundred lines that precede it. However, the rule with description "For all log entries matched in filter "'Log file open,'" clear all log entries from the log file" doesn't seem to apply.

This no longer matters for me because, as you suspected, your multi-pattern configuration (almost) works. When applied, it now shows the first "Log file open" line, and then the main log entries with proper formatting, skipping the several-hundred entries that are missing a timestamp. However, I adapted your multi-line configuration to this:

[%d{yyyy.MM.dd}-%d{%H.mm.ss:fff}][%t]%c: %m%n
%c: %m%n
%m, %d{dd/MM/yy %H:mm:ss}%n

LogViewPlus now seems to full understand our logfile format - I get one "Log file open" entry with the correct time, several hundred entries with today's date but no timestamp (not that important), and then the 'proper' logs following that. Success!

Furthermore:

1. The "Clear all log entries when the log file is rolled or recreated." option now works entirely as expected.
2. The rule-based approach to clear log entries also now works, but it takes a reload of the logfile first.

Glad that helped and you have a working solution - thanks for letting me know.

I will look into the issues with the rule based approach.  I know there was a rule fix in v3.1.6, so it may be worth checking you are on the latest version.  I will try and do some testing here before the next release.  

Regarding...

%c: %m%n

I am actually surprised that this configuration works as it is a requirement that all log entries have a timestamp.  You might find that some features in LogViewPlus (such as log file merging) work incorrectly with these log entries.  We do not test for this scenario.