Time offset on parser mapping


Author
Message
Patrick Dawson
Patrick Dawson
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 5, Visits: 34
Hi there. Is it somehow possible to set the time offset in the parser mappings (or by file name pattern). We have some logfiles that have timestamps in UTC (but don't use the ISO 8601 format) and other logfiles that print in local time: this way LogViewPlus cannot figure out the correct timezone. 
I already found out that I can adjust this with "time offset" manually. This way is a bit cumbersome if I open a zip file with a lot of logs and then have to adjust the time offsets manually). It would be nice to be able to configure this once per file pattern and then be done with it.

Is this possible somehow or planned as a feature in the near future?
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Hi Dawson,

I assume you are familiar with the offset target and open with offset features which allow you to specify an offset to be applied to multiple files.  These features might help, but I understand that they are not quite what you are looking for.

I can see where it would be helpful to associate a time offset with a parser.  The problem is adding the feature in such a way that it doesn't complicate the parser settings for other users.  I will have a think about how we might do this as I think an 'advanced / flexible' configuration might have other uses.

In the meantime, you should be able to achieve this with a custom post processor.

Hope that helps,

Toby
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Hi Dawson,

I just wanted to let you know that we have now released LogViewPlus v3.0.1 as a BETA release. This release allows you to define the date format with an offset.  To do this, define the date format with new -offset parameter and the number of milliseconds by which the date should be offset. For example, if you would like to offset the date by an hour, you could use something like:

%d{dd/MM/yyyy HH:mm:ss.fff -offset:3600000}

Hope that helps. Thanks again for bringing this issue to our attention.

Toby
Patrick Dawson
Patrick Dawson
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 5, Visits: 34
Hi,
Thanks for implementing a solution. I just tested version 3.0.2 but sadly found a bug when using the offset parameter:
With the following logfile:

2022.12.15-14:10:51.702 - info: Running node.exe v16.16.0. Using 'production' configuration:
{
"binaryDataFolder": "C:/ProgramData/some/product",
"log": {
"directory": "C:/ProgramData/some/product/logfiles",
"level": "info",
"timestampFormat": "YYYY.MM.DD-HH:mm:ss.SSS",
"responseTimeThreshold": 1000,
"mongoLogInterval": 60000,
"memoryMonitoring": {
"intervalMs": 1000,
"thresholdMb": 50
}
},
"import": {
"tempFolder": "C:/ProgramData/some/product/import",
"jobExpiry": 604800,
"tokenExpiry": 21600,
"pollingInterval": 1000,
"importAttempts": 3,
"cleanAttempts": 3,
"disableJSONValidation": false
},
"exportDataFolder": "C:/ProgramData/some/product/export",
"graphql": {
"introspection": false
},
"app": {
"name": "Some product"
}
}
2022.12.15-14:10:51.712 - warn: Product started

I get 2 log lines in the logviewer. When I use the offset parameter I get 2 additional lines with wrong date and time, see screenshot:

These are the the parset settings I've used:

// works
%d{yyyy.MM.dd-HH:mm:ss.fff} - %-7p %m%n
// does not work
%d{yyyy.MM.dd-HH:mm:ss.fff -offset:3600000} - %-7p %m%n

LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Hi Dawson,

I can confirm that this is a bug - thanks for bringing it to our attention.

I have identified the problem and implemented a fix.  For clarity, the output should be:


We will try and get this release out as soon as we can.  We might be able to get a BETA out this week.

Thanks again,

Toby
LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Hi Patrick,

I just wanted to let you know that we have now released LogViewPlus v3.0.3 as a BETA release.

This release should resolve the issue with date resolution when using he '-offset' parser flag as discussed above.

Hope that helps,

Toby
Patrick Dawson
Patrick Dawson
New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)New Member (21 reputation)
Group: Forum Members
Posts: 5, Visits: 34
Hi Toby,

Thank you for the quick fix. I tested the beta and it seems to work now.


LogViewPlus Support
LogViewPlus Support
Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)Supreme Being (5.3K reputation)
Group: Moderators
Posts: 1.1K, Visits: 3.7K
Glad that helped - thanks for letting me know!

Toby
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Similar Topics

Login

Explore
Messages
Mentions
Search