Re: [Man-db-devel] Database update profiling

[Francis Giraldeau]

Le 2013-12-06 01:48, Kari Pahula a écrit :
> On Thu, Dec 05, 2013 at 03:18:29PM -0500, Francis Giraldeau wrote:
>> Hi! I did some tracing on apt-get, and one of the slow step of the
>> installation of a package on Debian/Ubuntu is the update of man database
>> with mandb.
> 
> I did some work exploring mandb's performance a year ago and managed
> to save some time on updates.  See
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=691643

I think this is interesting, because it prevent to regenerate the whole
database each time. What Colin is working on is to optimise libpipeline,
such that the whole db generation is faster, so both are complemetary.

> None of that code has yet made its way to mandb.

It's a good start, let's try make it ready.

I got some warnings when trying to build it:

check_mandirs.c: In function 'testmandirs':
check_mandirs.c:474:10: warning: ignoring return value of 'fchdir',
declared with attribute warn_unused_result [-Wunused-result]
check_mandirs.c:476:9: warning: ignoring return value of 'chdir',
declared with attribute warn_unused_result [-Wunused-result]
check_mandirs.c: In function 'mkcatdirs':
check_mandirs.c:410:11: warning: ignoring return value of 'chown',
declared with attribute warn_unused_result [-Wunused-result]
check_mandirs.c:435:13: warning: ignoring return value of 'chown',
declared with attribute warn_unused_result [-Wunused-result]
check_mandirs.c: In function 'add_dir_entries':
check_mandirs.c:378:8: warning: ignoring return value of 'chdir',
declared with attribute warn_unused_result [-Wunused-result]

Just for my understanding, the database stores strings? Why not storing
integers directly?

MYDBM_SET (content, xasprintf ("%ld", (long) time (NULL)));

I'm not a big fan of ifdef inside main code. Maybe we could isolate it
into it's own header. What is the point to use fchdir() istead of
chdir()? It doesn't seems to be related to the timestamp feature.

+#ifdef _BSD_SOURCE
+       if ( (fd = dirfd (dir)) != -1)
+               fchdir (fd);
+       else
+               chdir (path);
+#else
        chdir (path);
+#endif


I would suggest to put the timestamp testing in it's own function.

+               if (last && (stbuf.st_mtime < last
+                            || (stbuf.st_mtime == last
+                                && stbuf.st_mtim.tv_nsec < ulast))) {



>> I would be glad to help to improve the performance of mandb. For
>> instance, is there any documentation on how to setup a development
>> environment? I do find it a bit cumbersome to launch gdb on mandb as
>> root ;-)
> 
> mandb updates the user database when run as a regular user and I've
> found that quite sufficient for testing out things.  Copying
> /usr/share/man to a user location may help too, for things like
> checking how mandb acts on updated file timestamps.

Good to know, thanks!

I pushed the code to github:

https://github.com/giraldeau/man-db-2.6.3/tree/ctime

Cheers,

Francis