JSON Parsing Error


Author
Message
Brad Konia
Brad Konia
I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)
Group: Forum Members
Posts: 64, Visits: 99
Just sent you an error report generated by LVP when it failed to parse my JSON file.

I'm guessing the parsing error is happening beause one of the JSON properties contains encoded JSON, which is somehow confusing the parser. The encoded JSON is properly escaped, so it really should be handled the same as any other string value.

By the way, the reason I'm encoding JSON in a JSON object property is beause I need to log an API response that's sent as JSON. Unfortuntaely, your JSON parser only supports primitives, which kind of defeats the purpose of using JSON. Ideally, I could take the API response and store it as a proper JSON object within a property, instead of having to encode it to a string.
LogViewPlus Support
LogViewPlus Support
Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Thanks Brad.  I have had a look at the bug report and there is nothing wrong with the JSON line that failed.  This is unfortunate as it is going to make the issue harder to track down.  The issue seems to be with the prior line.

What's happening here is that LogViewPlus needs to determine when a full JSON log entry has been read (keeping in mind that JSON may span multiple lines in the log file).  This is done prior to JSON parsing.  In this case, it is receiving a line of perfectly formatted JSON which cannot be parsed correctly because apparently something in the prior line has confused the pre-scan.

We have made a lot of JSON changes in this release and plan to make a lot more in the next release.  However, in this case, I think this issue would have existed in prior versions as well.  There seem to be something in the file, prior to this log line, which is causing the invalid parse.

I realize this is not always possible, but if you are able to contact me via email and send me the log file I can probably get this turned around very quickly.  Regardless, I will take a closer look at the pre-scan code to see if I can determine where the error might be.

Also - it is only the parser definition that must work with JSON primitives.  This is because the conversion specifier is is a string.  There should be no issue with using a conversion specifier to store a JSON object.

Brad Konia
Brad Konia
I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)
Group: Forum Members
Posts: 64, Visits: 99
I figured out the problem. There was some non-JSON data on the previous line that shouldn't have been there. Nevertheless, it shouldn't crash like that. I guess the crash is coming from the JSON parser, but it should recover more elgantly like how the other parsers do, and display the dialog where you can edit the template.
Brad Konia
Brad Konia
I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)
Group: Forum Members
Posts: 64, Visits: 99
Also - it is only the parser definition that must work with JSON primitives. This is because the conversion specifier is is a string. There should be no issue with using a conversion specifier to store a JSON object.


Can you elaborate on this?

How would I go about creating a JSON parser that could store a JSON object in a property, instead of having to convert it to a stirng?

Edited 3 Years Ago by Brad Konia
LogViewPlus Support
LogViewPlus Support
Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
In general, I agree that the application should not crash like that.  However, even more important is that the log file is displayed correctly and no log entries are 'hidden' or 'lost'.  In this case, invalid JSON was detected.  LogViewPlus tried to recover by reading the next log line and combining it with the (so far) invalid JSON.  When this failed, it became impossible for the application to display the file accurately.  In this case, I believe a crash is appropriate.

In the next version of LogViewPlus, we plan to add an option to the JSON parser which allows for 'single line' JSON.  If the JSON log entry should exist on a signal line, then the scanning process can be skipped.  The JSON is still invalid, but the parser we are introducing in this release should be significantly better at handling invalid JSON.  Even if the line cannot be parsed, it can still be combined with another line so that processing can continue.  I believe your JSON object is on a single line, so this kind of logic would help.

To store a JSON object in a property, define your conversion specifier in the usual way:
{
    "message":"%m"
}


This definition can process messages like:
{
  "message": "Application initializing."
}


Or:
{
    "message": {
        "value1" : "Hello",
        "value2" : "World",
    }
}


It is necessary for the message field to be a string in the parser configuration (where the conversion specifier is defined).  However, the field value can be any JSON type.  

The above JSON will be displayed in LogViewPlus as:



Hope that helps,

Toby

Brad Konia
Brad Konia
I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)
Group: Forum Members
Posts: 64, Visits: 99
Yes, I tested it with a property containing an ojbect and it works perfectly. I was confused about this because I couldn't figure out how to get the wizard to accept an object. When I highlighted the object, it said it was not a valid primitive. It would be nice if you could make it so the wizard would accept an object, or at least mention in the error message that you can force it to accept it by making it a string.
LogViewPlus Support
LogViewPlus Support
Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Thanks for bringing this issue to my attention.  I can see why this was causing confusion and will take a look for the next release.  
Brad Konia
Brad Konia
I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)
Group: Forum Members
Posts: 64, Visits: 99
Another issue is that it treats the embedded object as a string, so when you copy the contents of the Message column, all the quotes are escaped. I can work around it by copying from the raw log entry in the bottom pane, but that's a pain, because you have to manually figure out the number of indenting levels. Could you add a setting in the conversion specifer syntax, to indicate that a field is a JSON object, not a string? Maybe appending a "j" to the specifier placeholder. For example, instead of %m, you could use %mj.
Edited 3 Years Ago by Brad Konia
LogViewPlus Support
LogViewPlus Support
Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)Prestige User (3.9K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Thanks Brad.  I still have this thread on my todo list, but I will take a look at copying the JSON data as well.

I think the "mj" is implied if you are using the JSON parser.  I suspect there is a simple way to display the string without escaping the JSON.
Brad Konia
Brad Konia
I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)I'm into this (260 reputation)
Group: Forum Members
Posts: 64, Visits: 99
I think the issue is that the conversion specifier is expecting a string, so it automatically escapes any quotes within the string. Currently it has no way of knowing that the message field contains JSON.
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Login

Explore
Messages
Mentions
Search