Re: `tail -f` intermittency

[Bob Proulx]

dae3 wrote:
> Eric Blake <address@hidden> wrote:
> > [please post with a valid email address]
> 
> On Usenet? No way.

You could just note that you are reading the mail from an archive and
therefore CC's to you are not necessary nor desired.  In my case I am
subscribed and so CC's to me are not necessary nor desired either.

> > Unfortunately, that breaks threading and makes replying to you very
> > awkward.  The newsgroup that you are referring to is merely a gateway
> > around a mailing list, and you picked the wrong gateway.
> 
> Where does one learn which is the "right" gateway?

For tail you would look at the --version and --help output.  --version
because people sometimes are using a different version than they think
they are using.  --help because it says address@hidden is the
place to report bugs.  Also reading the FAQ is useful.

After knowing that address@hidden is the right mail address
then you should email there.  If you know that all gnu bug reporting
addresses (with the exception of only a very few) are all mailing
lists at lists.gnu.org then you could avoid actually generating an
email yourself (which requires a valid email from address) and instead
finding a web interface.  There are several popular mail archive web
interfaces.  Use a web search tool to find them.

If you are using a vendor's distribution then it is unlikely that you
will want to report the bug upstream.  Because upstream reports, if it
really is a bug, will usually end up asking you to try some source
code patch.  Which means you need to be prepared to compile and test a
source code patch to the latest upstream release.  You probably don't
want to do that if you are running a distro stable version for
example.  Distro stable versions will almost always lab behind
upstream due to differing release schedules.  Reporting the problem to
your vendor's bug tracking system would be better in that case because
then the maintainers there will know what version you are using and
will be able to be a good go-between for source code patches if
needed.  Also a number of distros introduce patched versions of
upstream tools and a significant number of bugs have been found in the
introduced patches.

> > If you want a newsgroup gateway around the coreutils mailing list, you
> > can use http://news.gmane.org/gmane.comp.gnu.coreutils.bugs among
> > others.
> 
> I use Gmane for lots of other MLs. But Gmane asks for confirmation that
> one is allowed to post, which for most MLs means one is subscribed. In
> fact, I would have used Gmane if I had been able to subscribe OR if I'd
> known subscription wasn't necessary for this particular ML.

Subscription is not necessary for any of the gnu.org bug reporting
mailing lists.  (This is authoritative information as I am one of the
mailing list helpers who moderates messages from non-subscribers.)

> > Meanwhile, most GNU mailing lists do not require you to be a
> > subscriber to post - there is a human moderator in the loop which
> > allows non-subscribers posts through, so waiting for a confirmation
> > mail is not necessary.
> 
> Thanks for the info. It's VERY helpful that this list's website not only
> has no such notice, but even ASKS you to subscribe and tells you to wait
> for a freaking confirmation email that will never get sent.

What web site are you referring to?  Can you post the reference URL so
that it can be corrected?  What I see is on this page where it does
state that postings are moderated.

  http://www.gnu.org/software/coreutils/

  It has been necessary to moderate the Coreutils mailing lists to
  prevent the flood of spam.  Postings to the lists are held for release
  by the list moderator.  Sometimes the moderators are unavailable for
  brief periods of time.  Please be patient when posting.  If you don't
  see the message in the list archive then it did not get posted.

It also says this:

  Enhancement Requests and Bug Reports

  If you would like any new feature to be included in future versions of
  Coreutils, please send a request to <address@hidden>. 

Which says nothing about needing to be subscribed in order to post
there.  I just made that explicit.

> The changelogs say nothing about tail's -f flag, which means I'm not
> going to compile and install a newer version.

I haven't been following tail -f behavior closely either and therefore
may easily have missed changes there but I don't recall major changes
in behavior there between 5.97 and the latest.  So I doubt you would
see any changes by trying the latest.  I don't see any indications
that it would.  Therefore if I were you I would stick with your distro
version too.

However, if you really can reproduce a case where tail -f misses data
then that is very valuable.  Since tail -f is apparently working for
others it makes it well nigh impossible to guess at how it might be
failing for you in your environment.  A failing test case is very,
very, very valuable in making forward progress on hard bug reports.
Having a test case would allow testing if the bug in question exists
in any particular version or if it is fixed at some version.  It
allows determining if a source code patch fixes the problem.  I just
can't stress enough how important it is to be able to recreate
problems in order to be able to fix problems.

> I just wanted to report that `tail -f` - at least when running in the
> background - doesn't always output the latest additions to the watched
> file.
> 
> For example, I always have `tail -f /var/log/httpd/error_log` running
> in the background. I just discovered new (not necessarily recent)
> additions to the 'error_log' file that were never reported by `tail`.
> This has happened to me several other times.

I am guessing that your issue may be happening when the log file is
truncated or rotated.  Things are pretty simple when the file is
simply growing.  But enter log rotation and life gets interesting.  I
would guess that the worst case would be data appended to the file
immediately after it has been rotated.  Can you confirm this?

Please read the tail documentation which describes the various
'follow' behaviors.  You may want to select a different one.  Such as
'tail -F' (aka 'tail --follow=name --retry') instead.

  info coreutils 'tail invocation'

  `-f'
  `--follow[=HOW]'
       Loop forever trying to read more characters at the end of the file,
       presumably because the file is growing.  If more than one file is
       given, `tail' prints a header whenever it gets output from a
       different file, to indicate which file that output is from.

       There are two ways to specify how you'd like to track files with
       this option, but that difference is noticeable only when a
       followed file is removed or renamed.  If you'd like to continue to
       track the end of a growing file even after it has been unlinked,
       use `--follow=descriptor'.  This is the default behavior, but it
       is not useful if you're tracking a log file that may be rotated
       (removed or renamed, then reopened).  In that case, use
       `--follow=name' to track the named file by reopening it
       periodically to see if it has been removed and recreated by some
       other program.

       No matter which method you use, if the tracked file is determined
       to have shrunk, `tail' prints a message saying the file has been
       truncated and resumes tracking the end of the file from the
       newly-determined endpoint.

       When a file is removed, `tail''s behavior depends on whether it is
       following the name or the descriptor.  When following by name,
       tail can detect that a file has been removed and gives a message
       to that effect, and if `--retry' has been specified it will
       continue checking periodically to see if the file reappears.  When
       following a descriptor, tail does not detect that the file has
       been unlinked or renamed and issues no message;  even though the
       file may no longer be accessible via its original name, it may
       still be growing.

       The option values `descriptor' and `name' may be specified only
       with the long form of the option, not with `-f'.

       If `POSIXLY_CORRECT' is set, the `-f' option is ignored if no FILE
       operand is specified and standard input is a FIFO or a pipe.

  `-F'
       This option is the same as `--follow=name --retry'.  That is, tail
       will attempt to reopen a file when it is removed.  Should this
       fail, tail will keep trying until it becomes accessible again.

  `--retry'
       This option is useful mainly when following by name (i.e., with
       `--follow=name').  Without this option, when tail encounters a
       file that doesn't exist or is otherwise inaccessible, it reports
       that fact and never checks it again.

Bob