[Bug-wget] exit status

[Micah Cowan]

So, had some thoughts about exit status for recursive fetches. I just
posted the following comment to the bug report at
<https://savannah.gnu.org/bugs/index.php?23348>:

Wget should definitely return an error upon any type of failure for any
explicitly-specified URL arguments. And, as someone else mentioned, for
conditions related to actual problems, such as disk or network errors.
But then, what about network errors on non-explicitly-specified hosts
when -H is in force?

Things we'd ideally distinguish in error codes, in rough order of priority:

* write failure/system errors
* any sort of error preventing retrieval of explicitly-requested
  URL (not found, auth required, forbidden)
* network or lookup failure for an URL we didn't explicitly request, but
on a host corresponding to an URL we requested.
* network or lookup failure, but for a host that doesn't correspond to
  an explicitly-requested URL (when -H is in effect)
* any other sort of error preventing retrieval of recursively-found
  links
* robots/no-follow forbids

Could potentially use a bitmask for these, since in the case of
non-fatal errors, more than one might apply. Of course, certain errors
are important enough that we wouldn't really care that much whether or
not some of the lower-priority ones occurred. Using a bitmask would mean
we could only distinguish at most 8 different types of error.

Note that distinguishing errors based on a "host corresponding to an URL
we requested" would require significant adjustments, whereas "a URL we
requested" is itself an easy thing to determine, since it will be the
only URL passed to retrieve_tree() (things do get a little more
complicated when that URL had already reached the blacklist, having been
found via recursion).

Thoughts?

-mjc