From #twiki IRC:

PeterThoeny: i noticed that tools/tick_twiki.pl is a resource hog
PeterThoeny: eats 7% to 40% of processor, depending on how many other processes are running
PeterThoeny: on twiki.org it runs once an hour at the hour for 3 to 4.5 min each
Lavr: A primitive "old files" wiping script probably does the job as well.
Lavr: However it does not remove stale lease files.
Lavr: But one could run a session file wiper which only looks at age once per 6 hours and the tick_ maybe once per week.
PeterThoeny: yes
PeterThoeny: or make tick_twiki.pl understand a switch, such as -no_session_cleanup or the like

I assume the processing is done because of the session cleanup, is that correct? The .lease cleanup should be fast.

I suggest to make tools/tick_twiki.pl more efficient, or is that is difficult, make a switch where one can exclude a resource intensive task and run an alternative cleanup, such as a find on timestamp with rm. This should be documented as well.

BTW, twiki.org generates about 7000 session files within 6 hours. Why so many? How long do the session files need to live?

-- PTh

The chances are that the 7000 sessions are being created by spiders.

It's a resource hog because it visits every session file, and you have lots. If you have a session timeout of 7 hours there is little point in cleaning sessions every 60 minutes. The frequency of the tick_twiki should be about the same as the timeout, I think. Personally I run tick_twiki once every 24 hours, using a niced process. However the traffic is nothing like that on t.o.

Sessions are timed out when they have not been accessed for the timeout period. You can shorten the timeout right down if you want. An hour sounds quite appropriate for t.o. Shorter timeouts and more frequent tick_twiki runs means less work per run and fewer session files.

Please go ahead an make tick_twiki more efficient. Perhaps using something other than stat to determine last file access times. Note, however, that it exists because people felt that cleaning sessions after every HTTP access was too resource hungry; Something has to do all that work to time out sessions....

CC

I think Sven runs it every hour because twiki.org run out of inodes in the /tmp dir.

Just now the script was running for 10 min.

Just for twiki.org, would find on timestamp with rm be feasable? That would certainly be much faster.

-- PTh

The real issue is the many silly session files which indeed are created by spiders. The spiders do not accept cookies so a new session is made for each hit.

So maybe instead of fighting for a few CPU cycles in a script that needs to run 1 to 24 times per day - we should think if there is a way to handle spiders differently so they so not create sessions in the first place. Maybe through some Apache magic also.

-- KJL

Just for twiki.org, would find on timestamp with rm be feasable? That would certainly be much faster - that is pretty close to what tick_twiki.pl already does. You might try tweaking TWiki::Client::expireDeadSessions to be more aggressive about dealing with "old" session files. For example, it currently always reads the session file to the last access time. This is because stat does not work on all platforms - the last access time is totally unreliable on Windows, for example. On platforms where stat does work, it could reasonably make an expiry decision based on last access time from the stat information alone.

I think it would be preferable if you could tune the session expiry by modifying the dead sessions code, so that your experience can be passed on to other users, than just tuning t.o. alone....

CC

There have been no specific proposals for improvements, so dropping this to Low and moving it to Waiting for Feedback

-- CC

Sorry, but I move it back to a normal bug report. I also do not agree that a performance issue is low priority item.

I think I gave enough feedback on the issue. IMHP the party who created the feature should make sure that a new feature performs acceptably.

Please separate a short term TWiki.org workaround and a proper performance fix for the many TWikis out there. This bug item is about the latter.

-- PTh

Peter, three points. First off, the feature does perform acceptably for most people, as you are the only person to report a problem. It is certainly acceptable to me. Secondly, it is not about the latter, you reported the issue for TWiki.org but you apparently now want a "fix" it without offering any support whatsoever. Third, the person who wrote the code is no longer with us. It came from SessionPlugin .

I'm sorry, but I'm getting increasingly irritated by reports that say "it doesn't work" from reporters who are perfectly capable of performing the small amount of debugging required to generate a meaningful and useful report. All you had to do was to confirm what is causing the slowdown on t.o., a platform to which only you have access.

While doing something else, I made the trivial change to tick_twiki to convert it to "a simple script that deletes old files" (something which you could have easily done yourself in the course of a quick debug). I have no way of determining if this makes any difference to the overall performance of the script.

I am not prepared to waste any more time on this. Discarding.

CC

I will not engange in a ping-pong, lets leave it discarded. I find this unfortunate however, I read from this that the priority on performance is low. I believe I have supplied enough data to the author of the feature to point out the bottleneck on servers with a high load.

-- PTh