[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Nss-mysql-devel] [Patch #49] group handling optimization + qualifiers
From: |
nobody |
Subject: |
[Nss-mysql-devel] [Patch #49] group handling optimization + qualifiers |
Date: |
Tue, 06 Nov 2001 07:22:43 -0500 |
Patch #49 has been updated.
Project:
Category: None
Status: Open
Summary: group handling optimization + qualifiers
Follow-Ups:
Date: 2001-Nov-06 12:22
By: loic
Comment:
Hi,
This is a followup to the patch #47, it obsoletes it.
I've experienced serious performance problems with a large (?) number of users
and groups. Around 300 groups and 2000 users. The symptom was that login took
around 5 sec on a PIII 500 with 256Mb RAM, all because of CPU. I analysed the
problem and found that around 300 complex SQL queries were run. It happens when
a software getgrent all entries in groups.
I made absolutely sure that my index structure was optimal, using EXPLAIN.
There was no way for me to speed up to login phase without patching nss-mysql.
I did a rather drastic choice: load all group information in memory with a
single SQL request. After implementation the gain is one order of magnitude
(0.5 sec).
That solution may be considered bad, bad, bad. After all, what's the point of
having a database if you load everything in memory each time you need it ? The
real solution would be to implement an additional library call
(groupsinwhichthisuserbelongs for instance) and arrange for login to use it.
However, this is somewhat impractical because it would require a lot of work
before it becomes POSIX, is accepted by glibc maintainers and used by the login
maintainer.
Since avoiding a sequential walk of the groups is not an option, one could
still argue that loading everything in memory is asking for troubles if the
number of groups is large. My answer to this is : we already are in trouble
when the number of groups is large. Assuming the response time grows linearly
when the number of groups grows, if you have 3000 groups, it will take 50s to
login. Sure, you won't have troubles with memory but the system will be mostly
unusable.
I focused here on login but the exact same symptoms show for various other
programs such as id.
Technically speaking the patch is both a patch to group.c (that separates
filling the group struct from the sql records from the function that runs the
sql request) and ent.c (that runs a request to get all group information and
fill all group structures in advance).
Enjoy,
-------------------------------------------------------
-------------------------------------------------------
For more info, visit:
http://savannah.gnu.org/patch/?func=detailpatch&patch_id=49&group_id=443