JSON Parser

New to LogViewPlus?  Find out more about how you can use LogViewPlus to help you analyze JSON log files.

LogViewPlus has a built in JSON parser which is capable of analyzing your JSON log files.  It does this by parsing your JSON file according to a template.  A template is a sample JSON log entry that has certain fields identified with Conversion Specifiers.

LogViewPlus will not parse an entire log file as JSON.  Rather, it will parse the log file line by line while checking the input structure.  Only when the structure represents a complete JSON object will a parse be attempted.  This approach allows for monitoring JSON log files in tail mode, but may cause issues if your JSON log is a single block of text without new lines.

Because each log entry is parsed separately, our template will only need to match a single log entry.  For example, let's look at a simple JSON log enry:

{
   "firstName":"John",
   "lastName":"Doe",
   "employeeId":"12345",
   "other":"ignore me",
   "dateJoined":"2014-05-16 10:50:14,125"
} 

This is a JSON log entry with five fields: firstName, lastName, employeedId, other, and dateJoined.  What we need to do is replace the field data with a Conversion Specifier that identifies the field data type.  This might give us the following mapping.

JSON Field	Conversion Specifier	LogViewPlus Column
firstName	%S{First Name}	First Name
lastName	%S{Last Name}	Last Name
employeeId	%s{Employee Id}	Employee Id
other		We want to ignore this field.
dateJoined	%d	Date and Time

Therefore, we could parse this JSON log entry with the template:

{
   "firstName":"%S{First Name}",
   "lastName":"%S{Last Name}",
   "employeeId":"%s{Employee Id}",
   "dateJoined":"%d"
}

Notice that in the above template the "other" field has been ignored. To ignore a field we simply do not include it in our template.  If one of the elements we were interested in had been a child of a parent node, we would have needed to include the parent node in our template. The important thing is that the template has the full path to the target node.

Once we load this template into LogViewPlus it will appear as:

To do this, we just need to give LogViewPlus our parsing template as an argument for the JSON parser.  We can do this in Parser Mappings:

White space will be ignored, so we are free to format the JSON as needed.

If our log file contained multiple log entries, LogViewPlus would expect them all to have the same format.  New log entries should also be separated by a new line as discussed above.

Log files parsed with the JSON parser support automatic pretty-printing by default.

Finally, notice the similarities between the JSON Parser and the XML Parser discussed in the next section.  Both use the concept of templates, so once you have learned one you have basically learned the other.

Parsing Embedded JSON

LogViewPlus v2.5.56 and greater can parse JSON log entries are embedded within a parent object.  For example, consider the JSON log file:

{
   logid: "App Log 1",
   entries: [
      {
         "firstName":"John",
         "lastName":"Doe",
         "employeeId":"12345",
         "other":"ignore me",
         "dateJoined":"2014-05-16 10:50:14,125"
      },
      {  ...  }
   ]
}

Our conversion pattern for this log file will largely look the same as before with one crucial difference.  We must specify the outer element which will act to identify the resulting log entries.  The outer element must be an object.

In the example above, the outer element is "entries".  Using this, we can define our conversion pattern as:

{
   entries: [
      {
         "firstName":"%S{First Name}",
         "lastName":"%S{Last Name}",
         "employeeId":"%S{Employee Id}",
         "dateJoined":"%d"
      }
   ]
}

If the outer object containing our log entries was a child of another object, it would not be necessary define the additional outer object.  LogViewPlus will always traverse an object hierarchy automatically.  Log entry identifiers only need to exist at the log entry root.

Compact Log Event Format (CLEF)

LogViewPlus is a great tool for viewing CLEF log entries, for example messages created by Serilog.   CLEF stands for Compact Log Event Format and it is a method of producing log entries where the log message data is extracted and stored as separate fields withing the log entry.  This helps ensure the log entries are machine readable.

You can view CLEF log messages in LogViewPlus by adding a parser hint when processing the JSON message.  For example, consider the following CLEF log entry:

{
   "@t": "2020-04-25T04:03:29.3546056Z",
   "@mt": "Connection id '{Id}' accepted.",
   "Properties":
   {
      "Id": "0HMA0H"
   }
}

This log entry can be parsed using the JSON parser and the pattern:

{ "@t": "%d", "@mt": "%m{-parserhint:CLEF}" }

Notice that the pattern configuration contains a parser hint which is including using the special parameter -parserhint:CLEF.  This instruction tells the parser that the log entry message may be formatted to include data from within the JSON message.  The included data may be found either at the root level or (more commonly) within a 'Properties' child element.

Parsing our sample log entry using the CLEF parser hint will produce a log entry message that is easier to read.  In our example, this message would be:

Connection id '0HMA0H' accepted.

Notice how the connection ID has been extracted from the property data and inserted into the log entry message.