Date Specifier

By far, the hardest and most advanced conversion specifier is the date specifier - %d.  It is the most complicated because it is doing the most difficult job.  Date parsing is hard and it requires a keen eye for detail.  For example, consider the date:

2014-03-23 13:46:45,566 +01:00 PM

LogViewPlus uses a standard date parsing technology developed by Microsoft called Date Format Strings.  You can find out everything there is to know about date format strings by looking at Microsoft's documentation.

Let's jump right in and break our example date down into its parts:

Field
Example
Pattern
Notes
Year
2014
yyyy
The pattern for parsing a year is 'y', and in this case we use it four times.  The number of times it is used is relevant.  See below for more details.
Literal
-
-
 
Month
03
MM
Patterns are case sensitive.  See "minute" below.
Literal
-
-
 
Day
23
dd
 
Literal
" "
" "
 
Hour
13
HH
We are using a 24 hour clock.  For a 12 hour clock, we would use the lower case "hh".  See below for more details.
Literal
:
:
 
Minute
46
mm
 
Literal
:
:
 
Second
45
ss
 
Litreal
,
,
 
Millisecond
566
fff
 
Literal
" "
" "
 
Time zone
+01:00
zzz
 
Literal
" "
" "
 
Period
PM
tt
 

Putting all of the pattern elements together gives us:

yyyy-MM-dd HH:mm:ss,fff zzz tt

We can use this pattern to parse our date.  The final step is to pass our date conversion pattern into our conversion specifier.  We do this using a special syntax of surrounding the argument in curly brackets - "{}".  Our final conversion specifier is:

%d{yyyy-MM-dd HH:mm:ss,fff zzz tt}

Seems easy enough - right?  The tricky thing about date format strings is that you really need to pay attention to the number of times you are using a pattern as repeating a pattern can have a very unusual effect.  The tables below show the effect of using multiple patterns:

Year

Pattern
Matches
Notes
%y
9
%y would actually match '14' as well.  The percent sign in this case is an indicator that a 'year number' may or may not be found in that position.
yy
09
 
yyy
2009
 
yyyy
2009
 

Month

Pattern
Matches
%M
9
MM
09
MMM
Sep
MMMM
September

Day

Pattern
Matches
%d
9
dd
09
ddd
Tue
dddd
Tuesday

Hour

Pattern
Matches
Notes
%h
9
'9 AM or PM' on a twelve hour clock.  If your timestamp includes a period (for example, AM / PM) then you should always use a lower case 'h'. 
hh
09
 
hhh
09
 
hhhh
09
 
%H
21
'9 PM' on a 24 hour click.
HH
21
 
HHH
21
 
HHHH
21
 

Minute

Pattern
Matches
%m
9
mm
09
mmm
09
mmmm
09

Seconds

Pattern
Matches
%s
9
ss
09
sss
09
ssss
09

Fractional Seconds

Pattern
Matches
Notes
%f
9
 
ff
09
 
fff
009
 
ffff
0009
 
f*
Matches 1 to 7 digits.
See below.

We always recommend matching the number of fractional seconds exactly when parsing as this reduces the likelihood of incorrect data.  However, sometimes log files may use a variable number of digits.  In this case, you could use the custom specifier f*.  f* is used to match a variable number of fractional seconds up to 7 digits (or 100 nanoseconds).  Also, if your log file always has at least 3 digits, you could use fff*.  When matching digits in this way, it is always assumed that the first digit represents tenths of a second.

LogViewPlus can only represent time in memory to the 100 nanosecond level, or 7 decimal digits.  For timestamps more accurate than this, you can include more fractional digits with 'f' as a parser description, but the extra accuracy will be ignored.

Time zone

Pattern
Matches
Notes
%z
+9
 
zz
+09
 
zzz
+09:00
 
zzzz
+09:00
 
ZZZ
GMT
 
LogViewPlus supports over 100 different three-letter-acronym time zones.  This is indicated by an uppercase ZZZ. 

Period

Pattern
Matches
%t
P
tt
PM
ttt
PM
tttt
PM

That is everything you need to know about parsing dates with LogViewPlus.  Now, you just need to put the parts together in a way that matches the date format you use in your log files.

Here are a few examples to get you started:

Date String
Conversion Specifier
07:36:18:660761
%d{HH:mm:ss:ffffff}
2014-05-18-14.20.46.973000
%d{yyyy-MM-dd-HH.mm.ss.ffffff}
Dec 13 05:28:27
%d{MMM dd HH:mm:ss}
Sun Mar 7 16:02:00 2014
%d{ddd MMM d HH:mm:ss yyyy}
07/Mar/2004:16:06:51 -0800
%d{dd/MMM/yyyy:HH:mm:ss zzzz}
Sunday, 14 September 2014 10:50
%d{dddd, dd MMMM yyyy HH:mm}
2014-09-14T14:02:45.0174665+01:00
%d{yyyy-MM-ddTHH:mm:ss.fffffffzzzz}

Finally, if your timestamp does not include a date and your parser has not been configured to provide a date, then LogViewPlus will assume today's date.  All log entries in LogViewPlus must have a date.

Using %d Without Arguments

The date specifier has a default form which does not take any arguments.  You'll notice that the documentation frequently uses the date specifier without any arguments in order to simplify the discussion.  When you use the date specifier without any arguments you are asking LogViewPlus to make a best effort attempt at parsing the date. To do this, it will use the same date parsing technology which is used by the basic parser.  This is functionally correct and can work in many different situations.  However, it has two distinct disadvantages:

1. The date and time will be parsed on a best effort basis.  This means that the parsing is not guaranteed to work, and when it does work the date time value may appear incorrectly.

2. In attempting to dynamically determine the date format, LogViewPlus has to do significantly more work which may have a performance impact.

Because of these disadvantages, we recommend you specify the date format explicitly whenever possible.

Elapsed Date Times

Dates and times are occasionally represented numerically.  For example, the number of milliseconds since the UNIX epoch (Jan 1, 1970).  LogViewPlus supports the following built in conversion specifiers for working with numeric dates.

Conversion Specifier
Description
%d{Elapsed}
Used to parse any date time object that can be represented as a long.
%d{ElapsedDecimal}
Used to parse any date time object that can be represented as a decimal.  Fractional parts will be assumed to be number of fractional seconds.
%d{TAI}
Used to process longs which represent International Atomic Time.

When representing times numerically, LogViewPlus will attempt to convert the number into the correct time based on the size of the number.  For example, Microsoft .Net ticks are counted as the number of 100-nanosecond intervals that have elapsed since January 1, 000.  This number will be larger than the current time in milliseconds measured from Jan 1, 1970.

Metadata Date Patterns

Some log entries may be written in a format which has a timestamp but not a date. In these scenarios, the date is often extracted from the log file metadata. 

Metadata Date
Date Used
datetoday
The current date.  This setting does not use log file metadata.
filedate-created
The date the log file was created. This information may appear incorrect if the file has been copied or moved. This setting is not currently supported for remote log files.
filedate-modified
The date the log file was modified. This information may appear incorrect if the file has been copied or moved. This setting is not currently supported for remote log files.
filedate-namescan
LogViewPlus will try to parse the file name in order to extract the date. This process will use the same date parsing technology which is built into LogViewPlus. Unfortunately, a successful date parse cannot be guaranteed and there is currently no way to instruct LogViewPlus on the date format.  Therefore, the date can only be extracted on a best-effort basis.

For example, if you wanted LogViewPlus to use the log file creation date as the date for every log entry in the log file, you could use the conversion specifier:

%d{filedate-created %H:mm:ss}

In the above example, we have simply used the metadata date pattern 'filedate-created' as the first part of our date time pattern.


< >