bug#10349: tail: fix --follow on FhGFS remote file systems

bug-coreutils
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#10349: tail: fix --follow on FhGFS remote file systems

From:	Jim Meyering
Subject:	bug#10349: tail: fix --follow on FhGFS remote file systems
Date:	Fri, 23 Dec 2011 19:11:22 +0100
Pádraig Brady wrote:
...
>> My original goal was to warn, for unknown file system types,
>> that the type is unknown (suggesting to report it), and that
>> tail -f is resorting to the use of polling.
>
> Oh right, warn about unknown.
> That would make sense.

Here's a proposed patch:

>From 9d8d9d7151b9d47597a9808a6dfc4e0df5af04c5 Mon Sep 17 00:00:00 2001
From: Jim Meyering <address@hidden>
Date: Thu, 22 Dec 2011 23:23:02 +0100
Subject: [PATCH] tail: with -f, use polling when a file is on an FS of
 unknown type

Before we would use inotify in that case, which would work as long
as updates were taking place locally, but not at all when remote.
Move hard-coded list of known remote FS types into a more
maintainable table in stat.c, alongside the list of FS
names and magic numbers.  Generate a new is_local_fs_type function.
* src/Makefile.am (fs-is-local.h): New rule, generated file.
* src/extract-magic: Revamp to parse local/remote keyword after
each magic number in src/stat.c's case statements.
Accept new --local option.
* src/.gitignore: Ignore the generated fs-is-local.h.
* src/tail.c [HAVE_INOTIFY]: Include fs-is-local.h.
(fremote) [HAVE_INOTIFY]: Use the new function in place of
the switch stmt with hard-coded list of FS types.
Emit a warning when processing a file on a file system of unknown type.
* NEWS (Changes in behavior): Mention it.
Suggested by Sven Breuner.
---
 NEWS              |    7 ++
 src/.gitignore    |    1 +
 src/Makefile.am   |    9 +++
 src/extract-magic |   48 ++++++++++++---
 src/stat.c        |  164 ++++++++++++++++++++++++++--------------------------
 src/tail.c        |   30 ++++------
 6 files changed, 150 insertions(+), 109 deletions(-)

diff --git a/NEWS b/NEWS
index 80b6c9a..9b96f80 100644
--- a/NEWS
+++ b/NEWS
@@ -32,6 +32,13 @@ GNU coreutils NEWS                                    -*- 
outline -*-
   [you might say this was introduced in coreutils-7.5, along with inotify
    support, but the new magic numbers weren't in the usual places then.]

+** Changes in behavior
+
+  tail -f now uses polling (not inotify) when any of its file arguments
+  resides on a file system of unknown type.  In addition, for each such
+  argument, tail -f prints a warning with the FS type magic number and a
+  request to report it to the bug-reporting address.
+

 * Noteworthy changes in release 8.14 (2011-10-12) [stable]

diff --git a/src/.gitignore b/src/.gitignore
index d397370..d25a8e5 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -31,6 +31,7 @@ false
 fmt
 fold
 fs.h
+fs-is-local.h
 getlimits
 ginstall
 groups
diff --git a/src/Makefile.am b/src/Makefile.am
index f36e138..7d33803 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -148,6 +148,7 @@ noinst_HEADERS =    \
   fiemap.h             \
   find-mount-point.h   \
   fs.h                 \
+  fs-is-local.h                \
   group-list.h         \
   ioblksize.h          \
   ls.h                 \
@@ -597,6 +598,14 @@ fs-kernel-magic: Makefile
          | LC_ALL=C sort                                               \
          > address@hidden && mv address@hidden $@

+BUILT_SOURCES += fs-is-local.h
+fs-is-local.h: stat.c extract-magic
+       $(AM_V_GEN)rm -f $@
+       $(AM_V_at)$(PERL) $(srcdir)/extract-magic --remote $(srcdir)/stat.c \
+         > address@hidden
+       $(AM_V_at)chmod a-w address@hidden
+       $(AM_V_at)mv address@hidden $@
+
 BUILT_SOURCES += fs.h
 fs.h: stat.c extract-magic
        $(AM_V_GEN)rm -f $@
diff --git a/src/extract-magic b/src/extract-magic
index 6bc054f..8a4644f 100644
--- a/src/extract-magic
+++ b/src/extract-magic
@@ -65,10 +65,15 @@ FIXME: describe

 OPTIONS:

-  Derive #define directives from specially formatted `case ...:' statements.
+  There are two modes of operation, the default, which is to emit
+  #define directives derived from specially formatted `case' statements,
+  and that with --local, which is to emit a static inline function
+  mapping S_MAGIC_* values to 1, 0, -1, corresponding to known-local,
+  known-remote/distributed/network and unknown, respectively.

-   --help             display this help and exit
-   --version          output version information and exit
+   --local    emit an is_local_fs_type function
+   --help     display this help and exit
+   --version  output version information and exit

 EOF
     }
@@ -76,8 +81,12 @@ EOF
 }

 {
+  # The default is to print S_MAGIC_* definitions.
+  my $emit_magic = 1;
+
   GetOptions
     (
+     'remote' => sub { $emit_magic = 0 },
      help => sub { usage 0 },
      version => sub { print "$ME version $VERSION\n"; exit },
     ) or usage 1;
@@ -103,31 +112,50 @@ EOF
   # Fail if there is a `case S_MAGIC_.*' line without
   # a properly formed comment.

-  print <<EOF;
+  my $map_comment = <<EOF;
+/* Map each S_MAGIC_* value to 1, 0 or -1.
+   1 if it is known to be a remote file system type,
+   0 if it is known to be a local file system type, or -1 otherwise.  */
+EOF
+  my $magic_comment = <<EOF;
 /* Define the magic numbers as given by statfs(2).
    Please send additions to address@hidden and address@hidden
    This file is generated automatically from $file. */
-
-#if defined __linux__
 EOF
+  print $emit_magic ? $magic_comment : $map_comment;
+
+  $emit_magic
+    and print "\n#if defined __linux__\n";
+  $emit_magic
+    or print "static inline int\n"
+      . "is_local_fs_type (unsigned long int magic)\n"
+      . "{\n  switch (magic)\n    {\n";

   while (defined (my $line = <FH>))
     {
       $line =~ /^[ \t]+case S_MAGIC_/
         or next;
-      $line =~ m!^[ \t]+case (S_MAGIC_\w+): /\* (0x[0-9A-Fa-f]+) \*/$!
+      $line =~
+        m!^[ \t]+case (S_MAGIC_\w+): /\* (0x[0-9A-Fa-f]+) (local|remote) \*/!
         or (warn "$ME:$file:$.: malformed case S_MAGIC_... line"),
           $fail = 1, next;
       my $name = $1;
-      my $value = $2;
-      print "# define $name $value\n";
+      my $magic = $2;
+      my $local = $3 eq 'local' ? 1 : 0;
+      print $emit_magic
+        ? "# define $name $magic\n"
+        : "      case $name: return $local;\n";
     }

-  print <<\EOF;
+  $emit_magic
+    and print <<\EOF;
 #elif defined __GNU__
 # include <hurd/hurd_types.h>
 #endif
 EOF
+  $emit_magic
+    or printf "      default: return -1;\n    }\n}\n";
+
   close FH;

   exit $fail;
diff --git a/src/stat.c b/src/stat.c
index 801073b..0a454cd 100644
--- a/src/stat.c
+++ b/src/stat.c
@@ -237,169 +237,169 @@ human_fstype (STRUCT_STATVFS const *statfsbuf)
          a comment.  The S_MAGIC_... name and constant are automatically
          combined to produce the #define directives in fs.h.  */

-    case S_MAGIC_ADFS: /* 0xADF5 */
+    case S_MAGIC_ADFS: /* 0xADF5 local */
       return "adfs";
-    case S_MAGIC_AFFS: /* 0xADFF */
+    case S_MAGIC_AFFS: /* 0xADFF local */
       return "affs";
-    case S_MAGIC_AFS: /* 0x5346414F */
+    case S_MAGIC_AFS: /* 0x5346414F remote */
       return "afs";
-    case S_MAGIC_ANON_INODE_FS: /* 0x09041934 */
+    case S_MAGIC_ANON_INODE_FS: /* 0x09041934 local */
       return "anon-inode FS";
-    case S_MAGIC_AUTOFS: /* 0x0187 */
+    case S_MAGIC_AUTOFS: /* 0x0187 local */
       return "autofs";
-    case S_MAGIC_BEFS: /* 0x42465331 */
+    case S_MAGIC_BEFS: /* 0x42465331 local */
       return "befs";
-    case S_MAGIC_BFS: /* 0x1BADFACE */
+    case S_MAGIC_BFS: /* 0x1BADFACE local */
       return "bfs";
-    case S_MAGIC_BINFMT_MISC: /* 0x42494E4D */
+    case S_MAGIC_BINFMT_MISC: /* 0x42494E4D local */
       return "binfmt_misc";
-    case S_MAGIC_BTRFS: /* 0x9123683E */
+    case S_MAGIC_BTRFS: /* 0x9123683E local */
       return "btrfs";
-    case S_MAGIC_CGROUP: /* 0x0027E0EB */
+    case S_MAGIC_CGROUP: /* 0x0027E0EB local */
       return "cgroupfs";
-    case S_MAGIC_CIFS: /* 0xFF534D42 */
+    case S_MAGIC_CIFS: /* 0xFF534D42 remote */
       return "cifs";
-    case S_MAGIC_CODA: /* 0x73757245 */
+    case S_MAGIC_CODA: /* 0x73757245 remote */
       return "coda";
-    case S_MAGIC_COH: /* 0x012FF7B7 */
+    case S_MAGIC_COH: /* 0x012FF7B7 local */
       return "coh";
-    case S_MAGIC_CRAMFS: /* 0x28CD3D45 */
+    case S_MAGIC_CRAMFS: /* 0x28CD3D45 local */
       return "cramfs";
-    case S_MAGIC_CRAMFS_WEND: /* 0x453DCD28 */
+    case S_MAGIC_CRAMFS_WEND: /* 0x453DCD28 local */
       return "cramfs-wend";
-    case S_MAGIC_DEBUGFS: /* 0x64626720 */
+    case S_MAGIC_DEBUGFS: /* 0x64626720 local */
       return "debugfs";
-    case S_MAGIC_DEVFS: /* 0x1373 */
+    case S_MAGIC_DEVFS: /* 0x1373 local */
       return "devfs";
-    case S_MAGIC_DEVPTS: /* 0x1CD1 */
+    case S_MAGIC_DEVPTS: /* 0x1CD1 local */
       return "devpts";
-    case S_MAGIC_ECRYPTFS: /* 0xF15F */
+    case S_MAGIC_ECRYPTFS: /* 0xF15F local */
       return "ecryptfs";
-    case S_MAGIC_EFS: /* 0x00414A53 */
+    case S_MAGIC_EFS: /* 0x00414A53 local */
       return "efs";
-    case S_MAGIC_EXT: /* 0x137D */
+    case S_MAGIC_EXT: /* 0x137D local */
       return "ext";
-    case S_MAGIC_EXT2: /* 0xEF53 */
+    case S_MAGIC_EXT2: /* 0xEF53 local */
       return "ext2/ext3";
-    case S_MAGIC_EXT2_OLD: /* 0xEF51 */
+    case S_MAGIC_EXT2_OLD: /* 0xEF51 local */
       return "ext2";
-    case S_MAGIC_FAT: /* 0x4006 */
+    case S_MAGIC_FAT: /* 0x4006 local */
       return "fat";
-    case S_MAGIC_FHGFS: /* 0x19830326 */
+    case S_MAGIC_FHGFS: /* 0x19830326 remote */
       return "fhgfs";
-    case S_MAGIC_FUSEBLK: /* 0x65735546 */
+    case S_MAGIC_FUSEBLK: /* 0x65735546 remote */
       return "fuseblk";
-    case S_MAGIC_FUSECTL: /* 0x65735543 */
+    case S_MAGIC_FUSECTL: /* 0x65735543 remote */
       return "fusectl";
-    case S_MAGIC_FUTEXFS: /* 0x0BAD1DEA */
+    case S_MAGIC_FUTEXFS: /* 0x0BAD1DEA local */
       return "futexfs";
-    case S_MAGIC_GFS: /* 0x1161970 */
+    case S_MAGIC_GFS: /* 0x1161970 remote */
       return "gfs/gfs2";
-    case S_MAGIC_GPFS: /* 0x47504653 */
+    case S_MAGIC_GPFS: /* 0x47504653 remote */
       return "gpfs";
-    case S_MAGIC_HFS: /* 0x4244 */
+    case S_MAGIC_HFS: /* 0x4244 local */
       return "hfs";
-    case S_MAGIC_HPFS: /* 0xF995E849 */
+    case S_MAGIC_HPFS: /* 0xF995E849 local */
       return "hpfs";
-    case S_MAGIC_HUGETLBFS: /* 0x958458F6 */
+    case S_MAGIC_HUGETLBFS: /* 0x958458F6 local */
       return "hugetlbfs";
-    case S_MAGIC_INOTIFYFS: /* 0x2BAD1DEA */
+    case S_MAGIC_INOTIFYFS: /* 0x2BAD1DEA local */
       return "inotifyfs";
-    case S_MAGIC_ISOFS: /* 0x9660 */
+    case S_MAGIC_ISOFS: /* 0x9660 local */
       return "isofs";
-    case S_MAGIC_ISOFS_R_WIN: /* 0x4004 */
+    case S_MAGIC_ISOFS_R_WIN: /* 0x4004 local */
       return "isofs";
-    case S_MAGIC_ISOFS_WIN: /* 0x4000 */
+    case S_MAGIC_ISOFS_WIN: /* 0x4000 local */
       return "isofs";
-    case S_MAGIC_JFFS: /* 0x07C0 */
+    case S_MAGIC_JFFS: /* 0x07C0 local */
       return "jffs";
-    case S_MAGIC_JFFS2: /* 0x72B6 */
+    case S_MAGIC_JFFS2: /* 0x72B6 local */
       return "jffs2";
-    case S_MAGIC_JFS: /* 0x3153464A */
+    case S_MAGIC_JFS: /* 0x3153464A local */
       return "jfs";
-    case S_MAGIC_KAFS: /* 0x6B414653 */
+    case S_MAGIC_KAFS: /* 0x6B414653 remote */
       return "k-afs";
-    case S_MAGIC_LUSTRE: /* 0x0BD00BD0 */
+    case S_MAGIC_LUSTRE: /* 0x0BD00BD0 remote */
       return "lustre";
-    case S_MAGIC_MINIX: /* 0x137F */
+    case S_MAGIC_MINIX: /* 0x137F local */
       return "minix";
-    case S_MAGIC_MINIX_30: /* 0x138F */
+    case S_MAGIC_MINIX_30: /* 0x138F local */
       return "minix (30 char.)";
-    case S_MAGIC_MINIX_V2: /* 0x2468 */
+    case S_MAGIC_MINIX_V2: /* 0x2468 local */
       return "minix v2";
-    case S_MAGIC_MINIX_V2_30: /* 0x2478 */
+    case S_MAGIC_MINIX_V2_30: /* 0x2478 local */
       return "minix v2 (30 char.)";
-    case S_MAGIC_MINIX_V3: /* 0x4D5A */
+    case S_MAGIC_MINIX_V3: /* 0x4D5A local */
       return "minix3";
-    case S_MAGIC_MQUEUE: /* 0x19800202 */
+    case S_MAGIC_MQUEUE: /* 0x19800202 local */
       return "mqueue";
-    case S_MAGIC_MSDOS: /* 0x4D44 */
+    case S_MAGIC_MSDOS: /* 0x4D44 local */
       return "msdos";
-    case S_MAGIC_NCP: /* 0x564C */
+    case S_MAGIC_NCP: /* 0x564C remote */
       return "novell";
-    case S_MAGIC_NFS: /* 0x6969 */
+    case S_MAGIC_NFS: /* 0x6969 remote */
       return "nfs";
-    case S_MAGIC_NFSD: /* 0x6E667364 */
+    case S_MAGIC_NFSD: /* 0x6E667364 remote */
       return "nfsd";
-    case S_MAGIC_NILFS: /* 0x3434 */
+    case S_MAGIC_NILFS: /* 0x3434 local */
       return "nilfs";
-    case S_MAGIC_NTFS: /* 0x5346544E */
+    case S_MAGIC_NTFS: /* 0x5346544E local */
       return "ntfs";
-    case S_MAGIC_OPENPROM: /* 0x9FA1 */
+    case S_MAGIC_OPENPROM: /* 0x9FA1 local */
       return "openprom";
-    case S_MAGIC_OCFS2: /* 0x7461636f */
+    case S_MAGIC_OCFS2: /* 0x7461636f remote */
       return "ocfs2";
-    case S_MAGIC_PROC: /* 0x9FA0 */
+    case S_MAGIC_PROC: /* 0x9FA0 local */
       return "proc";
-    case S_MAGIC_PSTOREFS: /* 0x6165676C */
+    case S_MAGIC_PSTOREFS: /* 0x6165676C local */
       return "pstorefs";
-    case S_MAGIC_QNX4: /* 0x002F */
+    case S_MAGIC_QNX4: /* 0x002F local */
       return "qnx4";
-    case S_MAGIC_RAMFS: /* 0x858458F6 */
+    case S_MAGIC_RAMFS: /* 0x858458F6 local */
       return "ramfs";
-    case S_MAGIC_REISERFS: /* 0x52654973 */
+    case S_MAGIC_REISERFS: /* 0x52654973 local */
       return "reiserfs";
-    case S_MAGIC_ROMFS: /* 0x7275 */
+    case S_MAGIC_ROMFS: /* 0x7275 local */
       return "romfs";
-    case S_MAGIC_RPC_PIPEFS: /* 0x67596969 */
+    case S_MAGIC_RPC_PIPEFS: /* 0x67596969 local */
       return "rpc_pipefs";
-    case S_MAGIC_SECURITYFS: /* 0x73636673 */
+    case S_MAGIC_SECURITYFS: /* 0x73636673 local */
       return "securityfs";
-    case S_MAGIC_SELINUX: /* 0xF97CFF8C */
+    case S_MAGIC_SELINUX: /* 0xF97CFF8C local */
       return "selinux";
-    case S_MAGIC_SMB: /* 0x517B */
+    case S_MAGIC_SMB: /* 0x517B remote */
       return "smb";
-    case S_MAGIC_SOCKFS: /* 0x534F434B */
+    case S_MAGIC_SOCKFS: /* 0x534F434B local */
       return "sockfs";
-    case S_MAGIC_SQUASHFS: /* 0x73717368 */
+    case S_MAGIC_SQUASHFS: /* 0x73717368 local */
       return "squashfs";
-    case S_MAGIC_SYSFS: /* 0x62656572 */
+    case S_MAGIC_SYSFS: /* 0x62656572 local */
       return "sysfs";
-    case S_MAGIC_SYSV2: /* 0x012FF7B6 */
+    case S_MAGIC_SYSV2: /* 0x012FF7B6 local */
       return "sysv2";
-    case S_MAGIC_SYSV4: /* 0x012FF7B5 */
+    case S_MAGIC_SYSV4: /* 0x012FF7B5 local */
       return "sysv4";
-    case S_MAGIC_TMPFS: /* 0x01021994 */
+    case S_MAGIC_TMPFS: /* 0x01021994 local */
       return "tmpfs";
-    case S_MAGIC_UDF: /* 0x15013346 */
+    case S_MAGIC_UDF: /* 0x15013346 local */
       return "udf";
-    case S_MAGIC_UFS: /* 0x00011954 */
+    case S_MAGIC_UFS: /* 0x00011954 local */
       return "ufs";
-    case S_MAGIC_UFS_BYTESWAPPED: /* 0x54190100 */
+    case S_MAGIC_UFS_BYTESWAPPED: /* 0x54190100 local */
       return "ufs";
-    case S_MAGIC_USBDEVFS: /* 0x9FA2 */
+    case S_MAGIC_USBDEVFS: /* 0x9FA2 local */
       return "usbdevfs";
-    case S_MAGIC_V9FS: /* 0x01021997 */
+    case S_MAGIC_V9FS: /* 0x01021997 local */
       return "v9fs";
-    case S_MAGIC_VXFS: /* 0xA501FCF5 */
+    case S_MAGIC_VXFS: /* 0xA501FCF5 local */
       return "vxfs";
-    case S_MAGIC_XENFS: /* 0xABBA1974 */
+    case S_MAGIC_XENFS: /* 0xABBA1974 local */
       return "xenfs";
-    case S_MAGIC_XENIX: /* 0x012FF7B4 */
+    case S_MAGIC_XENIX: /* 0x012FF7B4 local */
       return "xenix";
-    case S_MAGIC_XFS: /* 0x58465342 */
+    case S_MAGIC_XFS: /* 0x58465342 local */
       return "xfs";
-    case S_MAGIC_XIAFS: /* 0x012FD16D */
+    case S_MAGIC_XIAFS: /* 0x012FD16D local */
       return "xia";

 # elif __GNU__
diff --git a/src/tail.c b/src/tail.c
index 1f619f3..4581845 100644
--- a/src/tail.c
+++ b/src/tail.c
@@ -54,6 +54,7 @@

 /* inotify needs to know if a file is local.  */
 # include "fs.h"
+# include "fs-is-local.h"
 # if HAVE_SYS_STATFS_H
 #  include <sys/statfs.h>
 # elif HAVE_SYS_VFS_H
@@ -896,26 +897,21 @@ fremote (int fd, const char *name)
     }
   else
     {
-      switch (buf.f_type)
+      switch (is_local_fs_type (buf.f_type))
         {
-        case S_MAGIC_AFS:
-        case S_MAGIC_CIFS:
-        case S_MAGIC_CODA:
-        case S_MAGIC_FUSEBLK:
-        case S_MAGIC_FUSECTL:
-        case S_MAGIC_GFS:
-        case S_MAGIC_GPFS:
-        case S_MAGIC_FHGFS:
-        case S_MAGIC_KAFS:
-        case S_MAGIC_LUSTRE:
-        case S_MAGIC_NCP:
-        case S_MAGIC_NFS:
-        case S_MAGIC_NFSD:
-        case S_MAGIC_OCFS2:
-        case S_MAGIC_SMB:
+        case 0:
           break;
-        default:
+        case -1:
+          error (0, 0, _("unrecognized file system type 0x%08lx for %s. "
+                         "please report this to %s. reverting to polling"),
+                 buf.f_type, quote (name), PACKAGE_BUGREPORT);
+          /* Treat as "remote", so caller polls.  */
+          break;
+        case 1:
           remote = false;
+          break;
+        default:
+          assert (!"unexpected return value from is_local_fs_type");
         }
     }
 # endif
--
1.7.8.1.367.g25ecc
[Prev in Thread]
Current Thread
[Next in Thread]
bug#10349: tail: fix --follow on FhGFS remote file systems, (continued)
Prev by Date: bug#10355: Add an option to {md5,sha*} to ignore directories
Next by Date: bug#10349: tail: fix --follow on FhGFS remote file systems
Previous by thread: bug#10349: tail: fix --follow on FhGFS remote file systems
Next by thread: bug#10349: tail: fix --follow on FhGFS remote file systems
Index(es):
- Date
- Thread