Hello:
So whenever I leave the application open from the night before, it's unresponsive in the morning. I can see there're maybe around 20mln record in the log by that time, but I can't do anything other than ending task through the task manager. It monitors just one log file.
The inconvenience make it impossible to view earlier records after restart, as the earlier logs are not retained in my system, but overwritten.
I know this has been discussed a number of times, but I strongly believe you need to find a way to unload or otherwise hibernate earlier records, so the application doesn't hang.

Thanks for reporting this issue.  With 20 million log entries, you are likely exceeding the memory on your machine.  

We could hibernate the log entries, but the problem here is that you would run out of memory as soon as you decided to do something with those entries which required them to be reloaded into memory (like searching).  Alternatively, there would be a lot of thrashing as memory is loaded and then unloaded repeatedly. This might be fine for one search, but if you are searching repeatedly or executing a more advanced features there would be a significant CPU hit.

We could maybe try some kind of "rolling tail" feature where log entries are expired after a period if time.  For example, "remove log entries older than 'one hour'".  Would that help? 

I actually have a 32Gb memory on the machine, so this is hardly an issue. When I see the app unresponsive the total memory usage is at 78% and other applications are running fine on the same machine. At the time LogView is using around 20Gb of these.
So I don't think the problem is like that.
The expiration won't help, as the only reason I leave it running is to capture overnight records that I can otherwise miss.

I don't know if a possible solution could be to maintain a runtime dump for the tail and load it on request. I would be willing to expend time waiting for the processing rather than not having any retrospection as the records are overwritten.

This sounds like reading from your log file and writing it to a separate log file.  That might help in this case, but I am not sure that feature would be helpful for others.

I just tested with a log file with 11 million entries taking about 13 GB in memory.  Searching takes about 8 seconds.  The application is slow, but not unresponsive.  CPU usage is within expectations. 

Are you doing anything with the log file, or just loading the records?  Do you have any filters / reports / highlights running?  Is LogViewPlus able to draw on screen?  Can you minimize the window?  With such a large file, some patience may be needed.

Also, what version of LogViewPlus are you using?

Version show 3.1.
When I start the application, it's idle for a couple of hours, then the log starts to accumulate, so it picks it up, and nothing more before I first try to access it in the morning, no filters, just loading records. If it runs minimized I can't even maximize it, it's just not responding, and is shown as such in the task manager.
I can store more log files on my side, they're truncated and numbered, in 10Mb chunks, so that I can load the corresponding time ref. But it's still really not convenient as I need to manually determine first which logfile chunk contains the time ref. There's no way to monitor many files as the app hangs even faster then.

If you were to restart LogViewPlus and open all of the files which represent the data, what happens?  I am guessing this would be about 500 files totalling 5GB on disk, so loading may take awhile.   I am wondering if Windows is managing LogViewPlus memory overnight in a way that it cannot easily restore (possibly paging the memory to disk).  This is a lot of data so I would expect some slowness, but not a full application hang.

> really not convenient as I need to manually determine first which logfile chunk contains the time ref

This is a feature that has really been missing from LogViewPlus.  It would be great to point it at a directory and say "open the log entries from last Monday between 13:00 and 14:00".  It is difficult to determine what might be going wrong with the application hang as a crash dump would be extremely large.  Maybe prioritizing this feature would be an better approach?

I have learned that opening multiple files that change dynamically is slowing down the application unbearably. Moreover it becomes a torture trying to delete the files from the workspace one by one later. So I avoid doing this. 
Right now I retain 100 log files 10Mb (used to be 5Mb) each. They have an extension equal to the number like logger.log.99. The way the logger works it renames the next file, if it was logger.log.99 it becomes logger.log.100. If LogView was to monitor all this files, it would end up accumulating almost all the logs in all 100 files.

> This is a feature that has really been missing from LogViewPlus. It would be great to point it at a directory and say "open the log entries from last Monday between 13:00 and 14:00". It is difficult to determine what might be going wrong with the application hang as a crash dump would be extremely large. Maybe prioritizing this feature would be an better approach?

Definitely would be a useful feature.