LogViewPlus Support » LogViewPlus Support » Configuration & Customization » Parse log with structured text and JSON

Parse log with structured text and JSON

remo0033 — Tue, 21 May 2024 16:48:27 GMT

The log file below is a LogViewPlus log entry, with pretty print JSON enabled

The first line has the DateTime and Level, and is structured text

Everything else is in JSON

I'd like to "squeeze" more fields out of my log by not only using the Pattern Parser, but also the JSON parser.

Is this possible?

The pattern parser only gives me 3 fields: Time, Level, and Message.

I'd like to extract more fields, from the JSON section, such as ProcessName, ProcessVersion, and even JSON arrays such as Variables and Arguments.

RE: Parse log with structured text and JSON

LogViewPlus Support — Tue, 21 May 2024 16:48:27 GMT

Thanks for reaching out. There are a few tricks with column reparsing.

First, only the Message column can be re-parsed. This is done by using the Parse Message command.

There is no dedicated JSON parse when parsing the log entry message. Instead, what you will need to do is determine which parts of the message you really need and which can be ignored. Here is a example message parser configuration which works on the example log entry above:
[code]> %S{IP} {"EventTime":"%S{EventTime}", %S}[/code]

Here we are extracting the IP and EventTime fields. Notice how all other data is ignored by using the %S conversion specifier. It is important to exclude data which is not common across all of the target log entries as it is often the exclusion of data which results in the creating the correct set of matching log entries.

Hope that helps. If you need further assistence, it may be worth creating a new ticket with a few sample log entries and a description of what you are trying to do.

Thanks,

Toby

RE: Parse log with structured text and JSON

bluecrow76 — Tue, 21 May 2024 16:30:37 GMT

I have the exact same situation and after working with the product for a few hours have not found a way to re-parse the column that has the JSON data in it.

The specific logs I'm needing to analyze are Windows Event Logs shipped by NXLog to a syslog-ng server, so each log entry is of the format:
2024-05-21T10:10:13-05:00 <user.info> 172.172.172.89 {"EventTime":"2024-05-21 10:10:13", ... other JSON data...}

Using the basic parser, I can get the full JSON data into a single column, but I'm unable to further process the data in that column using a parse message filter as the only options presented are automatic, pattern, regex, and data table.

Your product is exactly what I need right now, but there doesn't seem to be a way to get it to parse the logs in the way they're currently formatted. Any help would be greatly appreciated. 👍

RE: Parse log with structured text and JSON

LogViewPlus Support — Sat, 17 Aug 2019 01:04:42 GMT

Hi Remo,

Apologies for the slow reply.

The only way to achieve this at the moment would be to use a custom parser:
https://www.logviewplus.com/docs/custom_parsers.html

We do have plans to allow a log entry message to be "re-parsed" to allow extra data. However, these plans are unlikely to be actioned this year.

Also, you may be interested in a new setting in LogViewPlus which allows you to change the default syntax highlighting for a log file by changing it for a single log entry. Under the "Syntax Highlight" settings, please see:

Hope that helps,

Toby