Parsing MySql Log File


Author
Message
Email User
Email User
Gaining Respect (189 reputation)Gaining Respect (189 reputation)Gaining Respect (189 reputation)Gaining Respect (189 reputation)Gaining Respect (189 reputation)Gaining Respect (189 reputation)Gaining Respect (189 reputation)Gaining Respect (189 reputation)Gaining Respect (189 reputation)
Group: Forum Members
Posts: 36, Visits: 106
I am having trouble parsing a MySQL log file.  A sample log entry might look like:

# Time: 2019-06-01T07:35:37.564219Z
User@Host: user[user] @ localhost []  Id: 70218
# Query_time: 0.026542  Lock_time: 0.000222 Rows_sent: 1  Rows_examined: 19608
select * from MyTable

How can I configure LogViewPlus to parse this file? 
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Hi,

So, this is not an easy file to parse as the ParserWizard cannot handle the new line very well. However, you can parse the file by configuring it manually. I am using the PatternParser with the configuration:

# Time: %d{yyyy-MM-ddTHH:mm:ss.ffffffZ}\n# User@Host: %S{User Info}\n# Query_time: %S{Query Time}\n%m%n



If you wanted, you can probably break out some of the User Info and Query Time fields – but be aware that LogViewPlus cannot process optional fields (data that is written sometimes).

Hope that helps. Let me know if you continue to have problems.

Thanks,

Toby

Jon Mikel
Jon Mikel
New Member (4 reputation)New Member (4 reputation)New Member (4 reputation)New Member (4 reputation)New Member (4 reputation)New Member (4 reputation)New Member (4 reputation)New Member (4 reputation)New Member (4 reputation)
Group: Forum Members
Posts: 1, Visits: 2
LogViewPlus Support - 6/2/2019
Hi,

So, this is not an easy file to parse as the ParserWizard cannot handle the new line very well. However, you can parse the file by configuring it manually. I am using the PatternParser with the configuration:

# Time: %d{yyyy-MM-ddTHH:mm:ss.ffffffZ}\n# User@Host: %S{User Info}\n# Query_time: %S{Query Time}\n%m%n



If you wanted, you can probably break out some of the User Info and Query Time fields – but be aware that LogViewPlus cannot process optional fields (data that is written sometimes).

Hope that helps. Let me know if you continue to have problems.

Thanks,

Toby

This is very useful. It works perfectly with the PatternParser. I have tweaked it slightly to get further insight into the fields reported in the file. Very intuitive to use...

# Time: %d{yyyy-MM-ddTHH:mm:ss.ffffffZ}\n# User@Host: %S{User Info}\n# Query_time: %s{Query Time} Lock_time: %s{Lock Time} Rows_sent: %s{Rows sent} Rows_examined: %s{Rows examined}\n%m%n

Similarly, I have created another parser for the normal MYSQL log

%d{yyyy-MM-ddTHH:mm:ss.ffffffZ}\t %s{order} %s{command}\t%S{query}%n

Using them both together in the combined view is turning out super powerful in order to find out performance issues in my backend... I cannot thank Toby enough.


LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Nice one Jon - thanks for the update!

Toby
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Login

Explore
Messages
Mentions
Search