Custom Readers

When learning to create custom readers for LogViewPlus it is helpful to begin with the sample code projects.  This tutorial will assume that you have downloaded and run the sample projects successfully. Please see running the samples for more information.

Custom readers are the most advanced LogViewPlus extension type. Custom readers can be used when you need to import data into LogViewPlus from a data source that is not a text file. A custom reader might be used to access the database, read information off the network, or decode the data stored in a custom binary format (for example, a Protobuf log file).

Most custom readers will be environment specific. In an effort to simplify the deployment of our custom reader, we have decided to simply create a new log entry on a timer tick. This example is not very helpful in a real-world scenario but it does effectively show how to create a custom reader while simplifying the deployment and learning curve.

If we look at the code included in the CustomReader project and remove all of the comments, we can see the custom reader implementation is divided into three parts.  First, there is the code which generates the log entries.  This code is relatively straight forward.

private volatile int _entriesProcessed;
private readonly Timer _timer;
private readonly List<LogEntry> _cache = new List<LogEntry>();
private ILogMem _memoryManager = null;

public MyReader()
{
     _timer = new Timer(OnTimerTick);
}

private void OnTimerTick(object state)
{
     if (!_tailEnabled)
          return;

     lock (_cache)
     {
          var entry = new LogEntry(_memoryManager);
          entry.Date = DateTime.Now;
          entry.Priority = "INFO";
          entry.Logger = "MyLogger";
          entry.LogFileLineNumber = ++_logLineNumber;  // Sequential line number.
          entry.Message = ++_entriesProcessed + ": " +
                     (string.IsNullOrWhiteSpace(Arguments) ? "No arguments." : Arguments);

          _cache.Add(entry);
     }
}

These methods form the core of our log reader which will manage four private variables:

_entriesProcessed - This integer will keep track of the number of log entries which have been created.  This number may be useful in certain debugging scenarios.

_timer - The timer is used to simulate new log entries coming into the system.  Note that in the above example our timer has not yet been started.

_cache - Our cache is a list of log entries that have been created. We will need to manage our cache as part of our ILogReader implementation.  Some aspects of our ILogReader implementation may be called on a separate thread. Therefore we need to control access to the cache with a lock.

_memoryManager - The ILogMem reference is new in LogViewPlus 3.0 and required when creating a new log entries.  This property received in the InitializeRead method (see below) where it is cached for future reference.  This object should only be used as a pass-through to initialize new log entries.

The main method in our custom log reader implementation is the OnTimerTick event. This is the event that will generate our new log entries. We can see from the example above that our log entries should have the columns date, priority, logger, and message.  Our custom log reader implementation is creating a simple log entry which echoes the reader configuration back to the user.

The next method in our custom log reader implementation is the GetSupportedTypes method.

public List<FieldColumnInfo> GetSupportedTypes()
{
     var retval = new List<FieldColumnInfo>();
     retval.Add(new FieldColumnInfo(ElementType.Date, true));
     retval.Add(new FieldColumnInfo(ElementType.Priority, true));
     retval.Add(new FieldColumnInfo(ElementType.Logger, true));
     retval.Add(new FieldColumnInfo(ElementType.Message, true));
     return retval;
}

The GetSupportedTypes method is required when we are implementing the IColumnManagement interface. This method is responsible for returning the column definitions for our data set.  Implementation of this interface is not strictly required, however if we do not implement this interface we will not be able to see any columns for our parsed log entries. Therefore, most real-world examples will require an implementation.

The remaining methods in our CustomReader implement the ILogReader interface.  It is important to note that the ILogReader interface is optimized for the scenario where we are reading log files in batches to allow for improved performance.

public bool AllowProgressTracking { get { return false; } }
public long ProgressPosition { get { return 0; } }
public int LineNumber { get { return _entriesProcessed; } }
public bool AllowTail { get { return true; } }
public string Arguments { get; private set; }

public bool AcceptsConfiguration()
{
     return true;
}

public void Initialize(string arguments)
{
     Arguments = arguments;
     return;
}

public List<string> GetWarnings()
{
     return null;
}

public bool HasNextBatch(long fileSize)
{
     return false;
 }

public List<LogEntry> NextBatch()
{
     lock (_cache)
     {
          var list = _cache.ToList();
          _cache.Clear();
          return list;
     }
}

public bool AtEndOfFile(long fileSize)
{
     lock (_cache)
     {
          return _cache.Count == 0;
     }
}

public List<LogEntry> InitializeRead(ILogMem memoryManager, FileInfo file, long start, long stop)
{
     _memoryManager = memoryManager; // We will need this for any new LogEntries.

     var txt = File.ReadAllText(file.FullName);
     int tickInterval = int.Parse(txt);
     _timer.Change(0, tickInterval);
     return null;
}

There are four features that the ILogReader interface will allow us to control.

Once we have built our custom log reader and copied the resulting binaries into the LogViewPlus Plugins directory, we are ready to test. To do this start LogViewPlus, open Application Settings -> Reader Mappings and add the configuration shown.

After you have saved your settings, open the MyReaderConfig.log file which is included in the sample code distribution.  Assuming tail is enabled you should see a new log entry appear approximately every second.

If you were to temporarily disable tail and then re-enable it, all of the log entries missed during the time interval will be shown.