%S and %s Specifiers

To demonstrate the flexibility of the %s specifiers, we are going to look at a pattern parser which can parse an IIS log file.  Parsing an IIS log file is not difficult.  We are using it as a simple example of a text file with structured data which is written in a way that differs from most application log files.  Also, the columns used in an IIS log file differ from those typically used in an application log file - giving us a great opportunity to show off the power of the %s specifier.

One of the nice things about parsing IIS log files is that the log files often contain a comment which details the format of the log file.  A sample IIS log file format is shown below.  The IIS log file format separates all fields with a single space:

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken

This format might translate into a log entry like:

2014-05-16 13:28:28 192.168.1.1 GET /dir/file.html - 80 - 192.168.1.1 Mozilla - 200 0 0 39244

Now, how do we convert the format given into a pattern that pattern parser can understand. For starters, let's note 2 things. First, the log entry starts with the date - just like most of our application log files.  Second, the log entry can be considered complete when we reach a new line.  Therefore, we can use two common specifiers at the start and end of our log format - %d and %n.  Knowing this, we could be a bit lazy and parse this file with a simple pattern:

%d %m%n

But that's not very helpful.  We know the structure of the data, how can we extract more useful information?  The key to parsing unusual patterns is the %s specifier first discussed in specifier basics.  Using the %s specifier we can define columns like:

%s{S-IP}

The above specifier will retrieve the data from the "s-ip" field.  Knowing this, parsing the rest of the log entry becomes trivial.  The above IIS log entry can be parsed with the pattern:

%d %s{S-IP} %s{Method} %s{URI} %s{URI-Query} %s{Port} %s{Username} %s{C-IP} %s{User-Agent} %s{Referrer} %s{Status} %s{Substatus} %s{Win32-Status} %s{Time-Taken}%n

This pattern may look a bit confusing at first but notice that we are only using three conversion specifiers: %d, %s and %n.  The %s specifier is doing most of the work, we simply need to give LogViewPlus column names for the data.

Using this pattern we can load the IIS log file into LogViewPlus:

Once the log file is loaded into LogViewPlus all of the normal functionality such as text searching and data filtering will work as expected.