[PATCH] gdbm/gdbm_open (was: [BUG] gdbm , gdbm_open 1.7.3 & 1.8.0)

[Andrei Pelinescu-Onciul]

I attached 2 patches for gdbm 1.7.3 and gdm 1.8.0. The patches fix this
problem: they adjust the file_block_size to 2^n. Also if STATBLKSIZE >
4096 file_block_size will be set to 4096.


> 
> I've found the following bug in gdbm_open:
> 
> When creating a new database (GDBM_NEWDB) and when the block_size is 0
> or < 512, the file_block_size will be taken from a fstat call:
> 
>      if (block_size < 512)
>         file_block_size = STATBLKSIZE;
>       else
>         file_block_size = block_size;
> 
> Then we have:
>      dbf->header->block_size = file_block_size;
>      [...]
>       dbf->header->dir_size = 8 * sizeof (off_t);
>       dbf->header->dir_bits = 3;
>       while (dbf->header->dir_size < dbf->header->block_size)
>         {
>           dbf->header->dir_size <<= 1;
>           dbf->header->dir_bits += 1;
>         }
>        if (dbf->header->dir_size != dbf->header->block_size)
>         {
>           gdbm_close (dbf);
>           gdbm_errno = GDBM_BLOCK_SIZE_ERROR;
>           return NULL;
>         }
> 
> But if file_block_size is not of the form 2^n, it will never be equal to
> dbf->header->dir_size, and gdbm_open will fail.
> (the block size returned by stat is supposed to be the optimal size for
> disk
> i/o and not the actual filesystem blocksize.)
> 
> This happened to me on a Linux xfs filesystem, created with sunit=128
> and swidth=384 (sunit= stripe unit, swidth = stripe width, the numbers
> are 512 bytes blocks). swidth is used to report the preferred io size
> returned by the stat syscall.
> On my xfs filesystem fstat will report a 196608 io block size. 196608 =
> 3 * 65536, so it's not of the 2^n form and the piece of code above will
> always fail.
> 
> Another problem (pointed to me by Steve Lord) is if you follow blindly
> the value reported by the kernel you'll get big and inefficient dbm
> files. Perhaps it would be better if the block size will be limited on
> Linux to a maximum value (maybe 4K, 64K seems to be already too much).
> 


Andrei

--- gdbm-1.7.3/gdbmopen.c       Sun Dec 26 13:57:32 1993
+++ gdbm-1.7.3.new/gdbmopen.c   Thu Jul 12 18:00:53 2001
@@ -203,9 +203,15 @@
 
       /* Start with the blocksize. */
       if (block_size < 512)
-       file_block_size = STATBLKSIZE;
+       /* do not use block sizes > 4096 (inefficient)*/
+       file_block_size = (STATBLKSIZE < 4096)?STATBLKSIZE:4096;
       else
        file_block_size = block_size;
+       
+        /* adjust file_block_size to 2^n */
+      while ((file_block_size & (file_block_size - 1)) != 0)
+       file_block_size &= (file_block_size - 1);
+
 
       /* Get space for the file header. */
       dbf->header = (gdbm_file_header *) malloc (file_block_size);

--- gdbm-1.8.0/gdbmopen.c.orig  Thu Jul 12 18:04:42 2001
+++ gdbm-1.8.0/gdbmopen.c       Thu Jul 12 18:04:46 2001
@@ -209,9 +209,15 @@
 
       /* Start with the blocksize. */
       if (block_size < 512)
-       file_block_size = STATBLKSIZE;
+       /* do not use block sizes > 4096 (inefficient)*/
+       file_block_size = (STATBLKSIZE < 4096)?STATBLKSIZE:4096;
       else
        file_block_size = block_size;
+       
+        /* adjust file_block_size to 2^n */
+      while ((file_block_size & (file_block_size - 1)) != 0)
+       file_block_size &= (file_block_size - 1);
+
 
       /* Get space for the file header. */
       dbf->header = (gdbm_file_header *) malloc (file_block_size);