LogViewPlus Support » LogViewPlus Support » Help & Support » Group by generic Message Template on top of an existing regex parser

Group by generic Message Template on top of an existing regex parser

Marc — Sat, 15 Feb 2025 19:59:34 GMT

Hi!
I'd like the find similar SELECT statements to the same table scattered throughout the file in repeating patterns. (See below for an example log entries)
[code]09.12.2024 18:49:28:211 [Worker-0] com.example.MyLogger call INFORMATION: SELECT RandomValueStore WHERE Value = 519342 (duration=10.1504, #rows=9)
09.12.2024 18:49:28:211 [Worker-0] com.example.MyLogger call INFORMATION: SELECT TableA WHERE Value IN (1,2,3) (duration=10.1504, #rows=9)
09.12.2024 18:49:28:211 [Worker-0] com.example.MyLogger call INFORMATION: SELECT TableA WHERE Value = 1321 (duration=1000.1504, #rows=2451)
09.12.2024 18:49:28:211 [Worker-0] com.example.MyLogger call INFORMATION: SELECT RandomValueStore WHERE Value = 1 (duration=1000.1504, #rows=2451)
09.12.2024 18:49:28:211 [Worker-0] com.example.MyLogger call INFORMATION: SELECT TableA WHERE Value IN (5,6,7) (duration=10.1504, #rows=9)
09.12.2024 18:49:28:211 [Worker-0] com.example.MyLogger call INFORMATION: SELECT TableA WHERE Value = 234 (duration=10.1504, #rows=9)
09.12.2024 18:49:28:211 [Worker-0] com.example.MyLogger call INFORMATION: SELECT RandomValueStore WHERE Value = 851 (duration=1000.1504, #rows=2451)
09.12.2024 18:49:28:222 [Worker-0] com.example.MyLogger call INFORMATION: SELECT DISTINCT CURRENT_TIMESTAMP FROM RandomValueStore (duration=2.9189, #rows=50, result=2024-12-09 18:49:28.207)
09.12.2024 18:49:31:150 [Worker-0] com.example.MyLogger execute INFORMATION: Begin transaction (duration=2.5889)
09.12.2024 18:49:31:155 [Worker-0] com.example.MyLogger execute INFORMATION: INSERT INTO RandomValueStore (ID, Value) VALUES (?, ?) (duration=4.0736)
09.12.2024 18:49:31:166 [Worker-0] com.example.MyLogger execute INFORMATION: Commit transaction (duration=10.6521)
09.12.2024 18:49:31:168 [Worker-0] com.example.MyLogger fireDeferredEvents INFORMATION: Start fire 18 deferred events
09.12.2024 18:49:31:170 [Worker-0] com.example.MyLogger fireDeferredEvents INFORMATION: Fire 18 deferred events took totally 2 ms[/code]
I've found that parsing by Message Template does an very good job.

For further analyzation I need to also to parse the duration and rows.
See https://www.logviewplus.com/forum/2358/RE-Interpret-Column-as-Number#2360 for my previous post.

Therefore I'm using the following regex parse message filter:
[code](?<SQL>(.*)(?=\(duration\=))\(duration=(?<Duration>(\d+\.\d+)?)(?=[,\)]), #rows=(?<Rows>(\d+))(?=[,\)])[/code]

But with that filter in place i lost the ability to group by the different patters in the log file, because the Message Template is now my regex.

So I thought i could add on top another automatic parse message filter like so.

But then no data is shown anymore in both filters:

Also after removing the bottom filter, my regex cannot be edited:

If I further remove also my regex filter, the message template someties stuck with my regex when grouping by message template, but I wasn't able reproduce this anymore.
I'm using the version v3.1.18.

Is this even possible to do such parsing or am I doing something wrong?
Otherwise it would be very neat to have a feature to group by automatically recognized pattern on further parsed messages.

Thank you for your help
Kind Regards
Marc

RE: Group by generic Message Template on top of an existing regex parser

LogViewPlus Support — Sat, 15 Feb 2025 19:59:34 GMT

Thanks for the feedback Marc.

Grouping is decoupled from parsing, but we can only parse the message once.

I can see where being able to parse the same message multiple times would be helpful, but it's not something we are looking at right now. There are performance and technical issues with this approach that we would need to work through. That said, it is definitely a good idea and something to keep in mind.

Thanks again,

Toby

RE: Group by generic Message Template on top of an existing regex parser

Marc — Sat, 15 Feb 2025 19:28:13 GMT

Hi Toby,

Thanks for your again quick answer.

If I may suggest a feature here:
It would be nice if we could decouple the grouping from the additional message parsing, so that we would be able to dynamically group the message further more.

Until then I stick to sorting by column which should also do the trick :-)
But non the less it is a great tool.

I wish you a nice weekend.

Kind Regards
Marc

RE: Group by generic Message Template on top of an existing regex parser

LogViewPlus Support — Sat, 15 Feb 2025 16:53:00 GMT

Hi Marc,

When you apply a message parser, the log file is reloaded and reparsed. A log entry message can only be parsed once. Here, you are overriding the initial message parse with a new message parse and then trying to also open the old one. That is causing the "cannot be edited" message because the old message parser doesn't exist anymore.

You will notice that the new message parse filter works if applied directly to the log file.

When you edit an existing parse message filter, the old filter should be deleted. I am not sure why that's not happening, but it is something I can look into as I think it is causing some confusion here.

Hope that helps,

Toby