I'm introducing new colleagues to the marvel of LogViewPlus, and in the process, I've discovered the Save Analysis function, which is great.

However, the logfiles we're working on are full of json data, with a bunch of quotes (and then json within json, etc. it's a mess, but anyway)
I've noticed, that the log content reloaded from a saved logzip archive is not equivalent with the original log files, in particular, the json content in the message field seems to suffer, this has its quotes duplicated.

Looking at the entries, it seems, that this is probably a byproduct, that the logfiles are saved as a sort of CSV (including adding extra empty columns where needed - I have a complex parser for the original file, that does not provide values for all fields in all cases.)

Two questions:
 - can the duplicated quote issue be fixed at save/load time with the current saved analysis solution?
 - wouldn't it be easier to include logfiles as they are in the intial analysis, and include the used parsers with the archive? I guess, this would also have its implementation drawbacks, but you would keep the original logs intact.

a simple example. Original entry:

2021-11-23T10:21:10.883+0100; INFO ; 10.10.10.10; 0c62cf#a313efa2-8649-4dd0-8fb7-bd6013eede88; [ps-client-configurator_galatea/SelectPageRoleContextViewModel]; #Interaction Change role context to | {"role":"Rfc556a2d-4257-4fd5"};

From the logzip:

"2021-11-23 10:21:10,883",Interaction,,,0c62cf#a313efa2-8649-4dd0-8fb7-bd6013eede88,,INFO,,,10766,20211123_102103_lifeXCentral.log,10.10.10.10,ps-client-configurator_galatea/SelectPageRoleContextViewModel,,"Change role context to | {""role"":""Rfc556a2d-4257-4fd5""};"

As a side-effect, by throwing away the original log entries, some of my manually added filters in my analysis will fail as well (for example, if I filtered for #Interaction for the line above)

Hi Keli,

Thanks for reporting these issues.  The double quote problem seems like a bug.  We have a release planned for mid-Dec, so we should be able to turn this around quickly. 

The CSV file written by the Save Analysis command should match the columns available in your log entry grid.  It is really the output of your parser configuration.  It sounds like you are seeing something else?  I would expect the filters to continue working normally.  You can contact me directly if sharing data online is an issue.

We considered exporting application settings such as parser configurations.  However, this had a number of issues such as versioning and 'non-parsed' data.  In the end, we decided to keep it simple.  The CSV files should reproduce the same data as the local parser, but not having the original application settings means that some minor functionality may be missing.  For example, without local or global priority mappings, some of the row highlights may be different.  However, the data and the filters should be equal.

It's also worth mentioning that if you look in the log file properties of an analysis CSV file, you should find a note referencing the original source data (this feature is helpful if a bit hidden :-).

Thanks again.  I will take a look at the double quote issue and get back to you.

Toby

I actually understand why some filters don't work in my case - it has to do with how I have built my parsers and how I'm using the filters, so it is somewhat my fault  

I parse a particular field as a word that has a hashtag before it, but the hashtag itself is not included in the parsed field. (for example: #%s{Action} )   So the Action field will contain action verbs, but without the hashtag.
However, sometimes, I'm searching for #<Action verb> 
This particular parser/search combination will not survive being Saved as analysis and reloaded on a different machine.

But I'm aware this is due to how I have made my own parsers and how I'm searching my logs, so my mistake here, sort of

That makes sense. The problem here is that your filter is searching on the original log entry and the search spans multiple fields. As the look of the original log entry changes after export the search no longer works. This is an edge case I had not considered. I am not sure what could be done about it other than including the original log entry in the export, but this would double the size.

As work-around you could set the column value in your filter.

Hi Zoltan,

I have reviewed this issue.  It turns out the problem with double quotes is not a bug, but a consequence of escaping the data so it can be written to a CSV file.  While not a bug, I can see why this was causing confusion.

This plus the 'filter missing data' problem you highlighted above has made us rethink our approach.  We have decided to add the full original log entry to the *.lvp file format.  This does have the downside of almost doubling the size of these files.  However, as the 'Save Analysis' feature compresses the logs, you will not notice much of a file size difference.  Once opened in LogViewPlus the data should look very familiar.

The file size issue is a better problem to have than the 'unfamiliar / missing data' issues described above - so I think this is a solid improvement in this feature.

This change has been implemented in the latest beta release - v2.5.41.

Hope that helps.  Thanks for bringing this issue to our attention!

Toby