Re: system crash when much disk io

[Neal H Walfield]

> Running dselect or apt-get very often gets my hurd box on it's knees.
> When it's 'Reading Package Lists...' , the system hangs, after giving:
>
> ext2fs.static: ../../ext2fs/pager.c:396: file_pager_write_page:
> Assertion `block' failed.

This is a bug.  I have attached a patch.  Could you give it a try and
see if the bug shows up again (I have never actually seen it on my
system)?

The problem that I see is that file_pager_read_page and
file_pager_write_page both use use find_block (note: they are the only
callers of find_block) to map a page in a file to its corresponding
block on disk, however, they expect conflicting behaviors in the case
where the block is unallocated.

The find_block function does the right thing for file_pager_read_page:
when reading a page, there is no reason to have to allocate a
corresponding page on disk -- unallocated pages are just zero pages so
we make the optimization.  Thus, find_block correctly calls
ext2fs_getblk with the create flag set to zero.

In the case of file_pager_write_page, unallocated pages need to be
allocated on disk: we are about to write into the region.  The
file_pager_write_page function assumes that find_block will make sure
that a corresponding page exists or that it will return an error.
This is not, however, how find_block works nor should this behavior be
expected: the comment is quite clear.

My solution is to add another argument to find_block, CREATE, to
indicate how unallocated blocks should be handled.

libdiskfs:

2001-11-24  Neal H Walfield  <neal@cs.uml.edu>

        * pager.c (find_block): Take a new argument, CREATE.  Pass
        this to ext2_getblk instead of assuming that the callee never
        wants the indicated block block to be allocated.
        (file_pager_read_page): Conform to the new semantics of
        find_block.
        (file_pager_write_page): Likewise.


> As a workaround, I tried importing /var/lib/apt/ and /var/lib/dpkg via
> nfs.  However, this gives different problems: apt complains about
>
>  E: Couldn't make mmap of 6291457 bytes - mmap (1073741869 Operation not 
> supported)
>  E: The package lists or status file could not be parsed or opened.

The nfs client does not support mmap.  I referrer you to:
http://mail.gnu.org/pipermail/bug-hurd/2001-October/005651.html


Index: pager.c
===================================================================
RCS file: /cvsroot/hurd/hurd/ext2fs/pager.c,v
retrieving revision 1.68
diff -p -u -2 -r1.68 pager.c
--- pager.c     2001/01/07 17:03:55     1.68
+++ pager.c     2001/11/24 14:27:32
@@ -1,5 +1,5 @@
 /* Pager for ext2fs
 
-   Copyright (C) 1994,95,96,97,98,99,2000 Free Software Foundation, Inc.
+   Copyright (C) 1994,95,96,97,98,99,2000,01 Free Software Foundation, Inc.
 
    Converted for ext2fs by Miles Bader <miles@gnu.org>
@@ -122,11 +123,13 @@ free_page_buf (void *buf)
 
 /* Find the location on disk of page OFFSET in NODE.  Return the disk block
-   in BLOCK (if unallocated, then return 0).  If *LOCK is 0, then it a reader
-   lock is aquired on NODE's ALLOC_LOCK before doing anything, and left
-   locked after return -- even if an error is returned.  0 on success or an
-   error code otherwise is returned.  */
+   in BLOCK (if unallocated and CREATE is false, return 0; if
+   unallocated and CREATE is true, attempt to allocate a block).  If
+   *LOCK is 0, then a reader lock is aquired on NODE's ALLOC_LOCK
+   before doing anything, and left locked after return -- even if an
+   error is returned.  0 is returned on success otherwise, an error
+   code.  */
 static error_t
 find_block (struct node *node, vm_offset_t offset,
-           block_t *block, struct rwlock **lock)
+           block_t *block, int create, struct rwlock **lock)
 {
   error_t err;
@@ -137,9 +140,9 @@ find_block (struct node *node, vm_offset
       rwlock_reader_lock (*lock);
     }
-
+  
   if (offset + block_size > node->allocsize)
     return EIO;
 
-  err = ext2_getblk (node, offset >> log2_block_size, 0, block);
+  err = ext2_getblk (node, offset >> log2_block_size, create, block);
   if (err == EINVAL)
     /* Don't barf yet if the node is unallocated.  */
@@ -234,5 +237,5 @@ file_pager_read_page (struct node *node,
       block_t block;
 
-      err = find_block (node, page, &block, &lock);
+      err = find_block (node, page, &block, 0, &lock);
       if (err)
        break;
@@ -391,5 +395,5 @@ file_pager_write_page (struct node *node
   while (left > 0)
     {
-      err = find_block (node, offset, &block, &lock);
+      err = find_block (node, offset, &block, 1, &lock);
       if (err)
        break;