nss-mysql-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Nss-mysql-devel] [Patch #49] group handling optimization + qualifiers


From: nobody
Subject: [Nss-mysql-devel] [Patch #49] group handling optimization + qualifiers
Date: Tue, 06 Nov 2001 07:22:43 -0500

Patch #49 has been updated. 

Project: 
Category: None
Status: Open
Summary: group handling optimization + qualifiers

Follow-Ups:

Date: 2001-Nov-06 12:22
By: loic

Comment:
Hi,

This is a followup to the patch #47, it obsoletes it.

I've experienced serious performance problems with a large (?) number of users 
and groups. Around 300 groups and 2000 users. The symptom was that login took 
around 5 sec on a PIII 500 with 256Mb RAM, all because of CPU. I analysed the 
problem and found that around 300 complex SQL queries were run. It happens when 
a software getgrent all entries in groups. 

I made absolutely sure that my index structure was optimal, using EXPLAIN. 
There was no way for me to speed up to login phase without patching nss-mysql.

I did a rather drastic choice: load all group information in memory with a 
single SQL request. After implementation the gain is one order of magnitude 
(0.5 sec).

That solution may be considered bad, bad, bad. After all, what's the point of 
having a database if you load everything in memory each time you need it ? The 
real solution would be to implement an additional library call 
(groupsinwhichthisuserbelongs for instance) and arrange for login to use it. 
However, this is somewhat impractical because it would require a lot of work 
before it becomes POSIX, is accepted by glibc maintainers and used by the login 
maintainer.

Since avoiding a sequential walk of the groups is not an option, one could 
still argue that loading everything in memory is asking for troubles if the 
number of groups is large. My answer to this is : we already are in trouble 
when the number of groups is large. Assuming the response time grows linearly 
when the number of groups grows, if you have 3000 groups, it will take 50s to 
login. Sure, you won't have troubles with memory but the system will be mostly 
unusable.

I focused here on login but the exact same symptoms show for various other 
programs such as id.

Technically speaking the patch is both a patch to group.c (that separates 
filling the group struct from the sql records from the function that runs the 
sql request) and ent.c (that runs a request to get all group information and 
fill all group structures in advance). 

Enjoy,
-------------------------------------------------------

-------------------------------------------------------
For more info, visit:

http://savannah.gnu.org/patch/?func=detailpatch&patch_id=49&group_id=443



reply via email to

[Prev in Thread] Current Thread [Next in Thread]