TWiki 4.1.2/June 22, 2007 download of
LdapContrib/Linux/Apache 2.0.52...
I'm trying to get
LdapContrib to do auth and also create
WikiNames from another LDAP field. These things are reasonably well documented.
My problem comes when I try to enable user mapping... the code attempts to load into the cache file ALL of our LDAP directory (130,000 entries in our
case) every time I click on a function such as "Login".
After looking at the code for a couple hours, it appears that the issue is that "loadLdapMapping" in only called with $this -- rather than $this and $name
(needs to build a more specific search filter).
Am I missing something simple? I can't find any parameters to limit the cache (or turn it off). It seems that the intent would be to add individuals to the cache as they do somthing, not pull down the whole thing...
--
TWiki:Main/CrisRhea - 23 Jun 2007
I'm having the same problem. Is there a workaround or fix?
--
TWiki:Main.CurtWilhelm - 23 Jul 2007
My workaround is to set the cache refresh rate to every couple of days.
All in all,
LdapContrib is an invaluable contrib, but I find the cache implementation very strange. In my installation I'd actually have less slowdown with no caching at all than with the default cache life of 10 minutes. My LDAP server can provide a single entry with essentially no delay, but it takes 10-15 seconds to provide all 6000 entries. Causing a page load to hang that long every 10 minutes just isn't acceptable.
Really, the cache should be re-implemented so that LDAP entries are only refreshed one at a time. And until we get that feature, we need some way to turn caching off.
--
TWiki:Main.IsaacRabinovitch - 22 Aug 2007
Yes, the current
LdapContrib is (a) not caching efficiently and (b) buggy.
There have been lots of people asking me why
LdapContrib tries to download so many records at once. Let me try to
find a lame excuse explain.
TWiki (we are talking about 4.1.2, the current stable release) calls for user information internally on a couple of occasions.
A user logs in, he is authenticated agains LDAP, infact no user info is needed for that, just a bind to the server.
Once a user is logged in, his attributes are fetched from LDAP to construct his WikiName besides email info and the like.
This is one record and this information is needed on every click he does, so thats worth caching. Let's not consider the
case of multiple users acting on the site simultaneously. Surely, this is at least on LDAP record each. Let's stay with this
one user who just logged in and now finds his WikiName pointing to his TWiki homepage. So far we only did
authentication and some housekeeping. Now comes authorization: users are member of groups and there are ACLs everywhere.
To check authorization TWiki now requests more information: groups and their members. TWiki-4.1.2 constructs a user
object for every user and every group (this has been reworked in TWiki-4.2), and thereafter checks if the current user
is member of that group. Imagine that it does not suffice to only request the current user's LDAP record but those of
most if not all others now. Now comes the reason why things are so slow: while in the first use case we only needed one record for the current user to let him in and hop around, we now have the second use case for the code: to perform authorization.
The bad part is,
that this is all force through the same API.
LdapContrib can't distinguish those two use cases. It does not
know if TWiki is about to request info about one user,
or if it is in the middle of an authorization process where a bunch
of user infos are needed. TWiki is implementing the membership check on its own application level, whereas this check
could have been done on the backend level with fewer overhead.
The current caching in
LdapContrib is bad. Full stop. What it does is it requests chunks of user records while they are
requested from the backend and caches them. It does this as longs as the actually requested user record was not found.
The next call will either benefit from the work done by the previous call, or continue to request chunks of user records.
This process is done in the middle of the rendering pipeline, that is, when the user clicks around. This process is time
critical and must be as lean as possible by all means. TWiki needs more than just one user record per click. Worstcase
use case is visiting Main.TWikiGroups: all users of all groups are requested and formatted into one big
html page, thus dossing the LDAP server (if these records haven't been fully cached yet), the TWiki server constructing
perl objects for each user and last not least the user's browser that has to render the resulting large page.
Well, but that can be dealt with by removing the
%GROUPS%
tag from Main.TWikiGroups and
replace it with a paginated
%LDAP{}%
query.
So the solution for
LdapContrib on TWiki-4.1.2 is to remove the caching mechanism from the rendering pipeline
and make it a separate thing. LDAP records don't change that often that there is a need to populate the cache from
scratch every 10 minutes automatically. Much more likely, LDAP admins add a new record and then want this
change to be visible on the TWiki side. It would not harm to add an extra "Refresh LDAP cache" button to the
LDAP admin's workflow to make sure the LDAP cache of TWiki is up to date. What I am going to say is, as
TWiki-4.1.2 needs a pretty broad knowledge about its users anytime, caching should be a one shot manual thing
and not be done automatically anymore. If the LDAP data changes every 24 hours or so, go with a cronjob that
runs midnight and refreshes TWiki's LDAP cache.
As far as I know this is the best we can do wrt LDAP & TWiki-4.1.2.
--
TWiki:Main.MichaelDaum - 22 Aug 2007
Not sure I really understand that, except the part about it not getting fixed soon. So let's talk about workarounds. One idea is set the cache refresh to > 1 day, then cause a manual refresh every night when usage is low. Question: how to refresh manually? The button on
LdapContrib does nothing that I can see. It just invokes /twiki/bin/view/TWiki/LdapContrib?refreshldap=on, which returns immediately — even though a refresh on my network takes at least 30 seconds. Also, no debug output; there should be about 8000 retrieval messages.
--
TWiki:Main.IsaacRabinovitch - 25 Aug 2007
I will be working on
LdapContrib next week as I need a fixed version on september 3.
--
TWiki:Main.MichaelDaum - 25 Aug 2007
Fixed in 2.0 release.
--
TWiki:Main.MichaelDaum - 11 Oct 2007