%S and %s Specifiers

To demonstrate the flexability of the %s specifiers, we are going to look at a pattern parser which can parse an IIS log file. Parsing an IIS log file is not difficult. We are using it as a simple example of a text file with structured data which is written in a way that differs from most application log files. Also, the columns used in a IIS log file differ from those typically used in an application log file - giving us a great opportunity to show off the power of the %s specifier.

One of the nice things about parsing IIS log files is that the log files often contain a comment which details the format of the log file. A sample IIS log file format is shown below. Note that the IIS log file format is a bit unusual in that all fields are separated by a single space:

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken

This format might translate into a log entry like:

2014-05-16 13:28:28 GET /dir/file.html - 80 - Mozilla - 200 0 0 39244

Now, how do we convert the format given into a pattern that pattern parser can understand. For starters, let's note 2 things. First, the log entry starts with the date - just like most of our application log files. Second, the log entry can be considered complete when we reach a new line. Therefore, we can use two common specifiers at the start and end of our log format - %d and %n. Knowing this, we could be a bit lazy and parse this file with a simple pattern:

%d %m%n

But that's not very helpful. We know the structure of the data, how can we extract more useful information? The key to parsing unusual patterns is the %s specifier first discussed in specifier basics. Using the %s specifier we can define columns like:


The above specifier will retreive the data from the "s-ip" field. Knowing this, parsing the rest of the log entry becomes trivial. The above IIS log entry can be parsed with the pattern:

%d %s{S-IP} %s{Method} %s{URI} %s{URI-Query} %s{Port} %s{Username} %s{C-IP} %s{User-Agent} %s{Referer} %s{Status} %s{Substatus} %s{Win32-Status} %s{Time-Taken}%n

This pattern may look a bit confusing at first but notice that we are only using three conversion specifiers: %d, %s and %n. The %s specifier is doing the lions share of the work we simply need to give LogViewPlus column names for the data.

Using this pattern we can load the IIS log file into LogViewPlus:

Once the log file is loaded into LogViewPlus all of the normal functionality such as text searching and data filtering will work as expected.

< >