I have a single log file open, and:



Why?

Hi Johnny,

On my machine, it takes about 40 seconds to open your log file.

Thanks for the additional information.  I can experiment with some of the additional details you provided to see if I can recreate the issue on my end.  It sounds like you were monitoring a remote file overnight in tail mode.  I assume the file was not rolled or recreated.

It is also worth noting that refreshing a log file is the same as closing and reopening it.  I wonder if your log file was refreshed overnight for some reason (maybe a log roll event?).

Compression can impact memory, but again - this memory should be collected.  The only significant long term memory should be the log entries.

Thanks,

Toby

Hi

Regarding the log being rolled or refreshed, it shouldn't have been. That's a separate issue I've been working on is to have the nextcloud container stop creating such enormous log files and roll them at around 5 or 10 mb instead. I *think* I have that taken care of, but it's still in the test phase. I do have to say that on smaller log sizes LVP runs great. =)

I suppose my biggest 'concern' is not really about how it performs with 1 massive log file open, since that can sort of be mitigated with rolling them in smaller sizes; but more of what might happen if I'm trying to monitor several log files at a time for debugging purposes and the total amount ends up around that 1gb+ size; I'm assuming it would still make LVP unusable due to an unresponsive UI. And I find myself working with many open log files very very frequently. Up until now I've been using VSCode to watch several logs at once, but it doesn't have all the pretty column parsing, and so forth that make it much easier to read a log. It seems to handle large and multiple files w/out issue, but again, most likely due to not really doing much parsing other than syntax highlighting.

Also just to address one thing you said, it was a local file (rather than remote). I have that log file mounted in docker so that it writes it out to a local folder, and then I just used LVP to open that local file, though yes, the container was still running all night so new items were being added (tailed). So there should be any 'overhead' in terms of maintaining a tcp connection, or anything of that nature.

Regarding mem compression, I was thinking more along the lines of compressing the actual text of the file in memory (rather than overall memory compression and reclamation). Sort of like having the log file "zipped" in memory, and you temporarily "unzip" the bits and pieces needed at the moment; for instance, to display in the table, and then recompress it as needed as well. It sounds like that could lead to a lot of cpu or what not, but with the powerhouse PCs we have these days, maybe it wouldn't be noticeable. Anyway, again, no idea if that's even possible so I'm just throwing ideas out into the ether, lol.

Apologies for the long read, I just like to give devs as much info as possible so maybe they can glean out the parts they find useful. =)

Thanks for the clarification Johnny.  Not sure where I got 'remote' from.  :-)

Given the memory you have available, you should be able to open a 1 GB file and work with it.  The issue here is the amount of memory consumed after an extended period of time.  It is not yet clear why the memory is spiking, but I am not sure you should be concerned about just opening and working with a large amount of data.  I am able to do this locally without issue.  I think the problems discussed here are more likely due to the accumulation of actions taken throughout the day.

There are several things we can do with memory management which will effectively be a trade-off of less memory for more CPU similar to the memory compression idea (a good one!  :-).  Currently, we aim to save CPU cycles, but we will be revisiting this soon. 

LogViewPlus consumes memory when opening and parsing the log file.  There is a lot of work that happens here and a lot of excess memory may be allocated which can then be reclaimed.  After parsing, everything else done by the application should be pretty trivial from a memory perspective.

Hope that helps,

Toby

Ok, TYVM, I'm just happy that I reported it and you're now aware of it. =)

If you'd like me to beta test any relevant changes, I'd be happy to.

One quick question (or request if it doesn't already exist): many similar log viewers or other apps (one example coming to mind is Discord) will have a feature that draws a horizontal line between where you left off since last viewing:



I guess maybe it just monitors if the window/tab has focus. That would be super handy when viewing logs as well, if it drew a line across the row above where you left off, so it's much faster to get your bearings when you come back.

Thanks for bringing this to our attention.  It's a problem I would love to get to the bottom of, but the first step is to recreate locally.  LogViewPlus is memory intensive, but in the next few months we will be revisiting this.  I suspect the rewrite will address this issue if we are not able to resolve it before then.  I will let you know when a beta release is available.

I can see where the line idea is helpful for chat applications, but I am not sure it would help with log files.  It is often not clear which messages have been 'read'.  LogViewPlus will tail log files and some applications can produce a lot of data.  So just because a message was displayed doesn't mean it was seen.

I think the closest thing we have to this feature is the "Clear Entries" command (F4).  This will clear all log entries from the view.  All log entries displayed after this action will therefore be new.

Hope that helps,

Toby