Date Specifier
By far, the hardest and most advanced conversion specifier is the date specifier - %d. It is the most complicated because it is doing the most difficult job. Date parsing is hard and it requires a keen eye for detail. For example, consider the date:
2014-03-23 13:46:45,566 +01:00 PM
LogViewPlus uses a standard date parsing technology developed by Microsoft called Date Format Strings. You can find out everything there is to know about date format strings by looking at Microsoft's documentation.
Let's jump right in and break our example date down into its parts:
Field
|
Example
|
Pattern
|
Notes
|
Year
|
2014
|
yyyy
|
The pattern for parsing a year is 'y', and in this case we use it four times. The number of times it is used is relevant. See below for more details.
|
Literal
|
-
|
-
|
|
Month
|
03
|
MM
|
Patterns are case sensitive. See "minute" below.
|
Literal
|
-
|
-
|
|
Day
|
23
|
dd
|
|
Literal
|
" "
|
" "
|
|
Hour
|
13
|
HH
|
We are using a 24 hour clock. For a 12 hour clock, we would use the lower case "hh". See below for more details.
|
Literal
|
:
|
:
|
|
Minute
|
46
|
mm
|
|
Literal
|
:
|
:
|
|
Second
|
45
|
ss
|
|
Litreal
|
,
|
,
|
|
Millisecond
|
566
|
fff
|
|
Literal
|
" "
|
" "
|
|
Time zone
|
+01:00
|
zzz
|
|
Literal
|
" "
|
" "
|
|
Period
|
PM
|
tt
|
|
Putting all of the pattern elements together gives us:
yyyy-MM-dd HH:mm:ss,fff zzz tt
We can use this pattern to parse our date. The final step is to pass our date conversion pattern into our conversion specifier. We do this using a special syntax of surrounding the argument in curly brackets - "{}". Our final conversion specifier is:
%d{yyyy-MM-dd HH:mm:ss,fff zzz tt}
Seems easy enough - right? The tricky thing about date format strings is that you really need to pay attention to the number of times you are using a pattern as repeating a pattern can have a very unusual effect. The tables below show the effect of using multiple patterns:
Year
Pattern
|
Matches
|
Notes
|
%y
|
9
|
%y would actually match '14' as well. The percent sign in this case is an indicator that a 'year number' may or may not be found in that position.
|
yy
|
09
|
|
yyy
|
2009
|
|
yyyy
|
2009
|
|
Month
Pattern
|
Matches
|
%M
|
9
|
MM
|
09
|
MMM
|
Sep
|
MMMM
|
September
|
Day
Pattern
|
Matches
|
%d
|
9
|
dd
|
09
|
ddd
|
Tue
|
dddd
|
Tuesday
|
Hour
Pattern
|
Matches
|
Notes
|
%h
|
9
|
'9 AM or PM' on a twelve hour clock. If your timestamp includes a period (for example, AM / PM) then you should always use a lower case 'h'.
|
hh
|
09
|
|
hhh
|
09
|
|
hhhh
|
09
|
|
%H
|
21
|
'9 PM' on a 24 hour click.
|
HH
|
21
|
|
HHH
|
21
|
|
HHHH
|
21
|
|
Minute
Pattern
|
Matches
|
%m
|
9
|
mm
|
09
|
mmm
|
09
|
mmmm
|
09
|
Seconds
Pattern
|
Matches
|
%s
|
9
|
ss
|
09
|
sss
|
09
|
ssss
|
09
|
Fractional Seconds
Pattern
|
Matches
|
Notes
|
%f
|
9
|
|
ff
|
09
|
|
fff
|
009
|
|
ffff
|
0009
|
|
f*
|
Matches 1 to 7 digits.
|
See below.
|
We always recommend matching the number of fractional seconds exactly when parsing as this reduces the likelihood of incorrect data. However, sometimes log files may use a variable number of digits. In this case, you could use the custom specifier f*. f* is used to match a variable number of fractional seconds up to 7 digits (or 100 nanoseconds). Also, if your log file always has at least 3 digits, you could use fff*. When matching digits in this way, it is always assumed that the first digit represents tenths of a second.
LogViewPlus can only represent time in memory to the 100 nanosecond level, or 7 decimal digits. For timestamps more accurate than this, you can include more fractional digits with 'f' as a parser description, but the extra accuracy will be ignored.
Time zone
Pattern
|
Matches
|
Notes
|
%z
|
+9
|
|
zz
|
+09
|
|
zzz
|
+09:00
|
|
zzzz
|
+09:00
|
|
ZZZ
|
GMT
|
LogViewPlus supports over 100 different three-letter-acronym time zones. This is indicated by an uppercase ZZZ.
|
Period
Pattern
|
Matches
|
%t
|
P
|
tt
|
PM
|
ttt
|
PM
|
tttt
|
PM
|
That is everything you need to know about parsing dates with LogViewPlus. Now, you just need to put the parts together in a way that matches the date format you use in your log files.
Here are a few examples to get you started:
Date String
|
Conversion Specifier
|
07:36:18:660761
|
%d{HH:mm:ss:ffffff}
|
2014-05-18-14.20.46.973000
|
%d{yyyy-MM-dd-HH.mm.ss.ffffff}
|
Dec 13 05:28:27
|
%d{MMM dd HH:mm:ss}
|
Sun Mar 7 16:02:00 2014
|
%d{ddd MMM d HH:mm:ss yyyy}
|
07/Mar/2004:16:06:51 -0800
|
%d{dd/MMM/yyyy:HH:mm:ss zzzz}
|
Sunday, 14 September 2014 10:50
|
%d{dddd, dd MMMM yyyy HH:mm}
|
2014-09-14T14:02:45.0174665+01:00
|
%d{yyyy-MM-ddTHH:mm:ss.fffffffzzzz}
|
Finally, if your timestamp does not include a date and your parser has not been configured to provide a date, then LogViewPlus will assume today's date. All log entries in LogViewPlus must have a date.
Using %d Without Arguments
The date specifier has a default form which does not take any arguments. You'll notice that the documentation frequently uses the date specifier without any arguments in order to simplify the discussion. When you use the date specifier without any arguments you are asking LogViewPlus to make a best effort attempt at parsing the date. To do this, it will use the same date parsing technology which is used by the basic parser. This is functionally correct and can work in many different situations. However, it has two distinct disadvantages:
1. The date and time will be parsed on a best effort basis. This means that the parsing is not guaranteed to work, and when it does work the date time value may appear incorrectly.
2. In attempting to dynamically determine the date format, LogViewPlus has to do significantly more work which may have a performance impact.
Because of these disadvantages, we recommend you specify the date format explicitly whenever possible.
Elapsed Date Times
Dates and times are occasionally represented numerically. For example, the number of milliseconds since the UNIX epoch (Jan 1, 1970). LogViewPlus supports the following built in conversion specifiers for working with numeric dates.
Conversion Specifier
|
Description
|
%d{ElapsedDateTime}
|
Used to parse any date time object that can be represented as a integer. LogViewPlus will resolve the timestamp using a protocol determined by the range of the timestamp. Possible formats include epoch ticks, epoch milliseconds and epoch seconds.
|
%d{ElapsedDateTimeDecimal}
|
Used when the timestamp is represented as a decimal value. LogViewPlus will resolve the timestamp using a protocol determined by the range of the timestamp. Possible formats include epoch ticks, epoch milliseconds and Julian dates.
|
%d{ElapsedTimeMilliseconds}
|
Used to when the log entry time is written is a decimal representing the number of milliseconds since the application started.
|
%d{ElapsedTimeSeconds}
|
Used to when the log entry time is written is a decimal representing the number of seconds since the application started.
|
%d{TAI}
|
Used to process longs which represent International Atomic Time.
|
When representing times numerically, LogViewPlus will attempt to convert the number into the correct time based on the size of the number. For example, Microsoft .Net ticks are counted as the number of 100-nanosecond intervals that have elapsed since January 1, 000. This number will be larger than the current time in milliseconds measured from Jan 1, 1970.
Metadata Date Patterns
Some log entries may be written in a format which has a timestamp but not a date. In these scenarios, the date is often extracted from the log file metadata.
Metadata Date
|
Date Used
|
datetoday
|
The current date. This setting does not use log file metadata.
|
filedate-created
|
The date the log file was created. This information may appear incorrect if the file has been copied or moved. This setting is not currently supported for remote log files.
|
filedate-modified
|
The date the log file was modified. This information may appear incorrect if the file has been copied or moved. This setting is not currently supported for remote log files.
|
filedate-namescan
|
LogViewPlus will try to parse the file name in order to extract the date. This process will use the same date parsing technology which is built into LogViewPlus. Unfortunately, a successful date parse cannot be guaranteed and there is currently no way to instruct LogViewPlus on the date format. Therefore, the date can only be extracted on a best-effort basis.
|
For example, if you wanted LogViewPlus to use the log file creation date as the date for every log entry in the log file, you could use the conversion specifier:
%d{filedate-created %H:mm:ss}
In the above example, we have simply used the metadata date pattern 'filedate-created' as the first part of our date time pattern.