[bug #62586] Warning given every time wget runs: “.netrc:..: unknown token” when a password contains quotes

[Eo Koochu]

Follow-up Comment #2, bug #62586 (project wget):

[comment #1 comment #1:]
> As you mention, there is no standard/spec. And this puts this issue into an
unsolvable state. Too many clients have their own, slightly diverting
implementation. Whatever any client will change, it likely breaks backwards
compatibility.
> 

I believe there’s been a misunderstanding about what the bug actually is.
The fact that wget parses .netrc differently than other apps is not the bug
(as every app is free to choose their own syntax, unfortunately).

I see two bugs in wget:

bug 1) wget incorrectly assumes that all other apps require the same syntax
for the password string.  So when a record appears in .netrc that exists for
fetchmail (not for wget), wget should not be making noise about it (unless
perhaps wget is asked to use a password it can’t parse for a host it’s
connecting to).  

bug 2) the wget man page neglects to specify the syntax of the .netrc file. So
not only is wget enforcing a particular syntax, it doesn’t document what
syntax it expects.

bug 3?) wget treats a password with many quote chars as a syntax error, when
cURL (for example) would not. (see below)

> I looked into netkit and GNU inetutils' ftp command. In both cases, the
.netrc parser allows surrounding double quotes and supports primitive escaping
using \ (the following char is taken as-is). No hint for a support of
bash-style quoting.
> 

Someone is wrong here. The fetchmail man page states that fetchmail follows
the same syntax rules as netkit-ftp. And the developer confirms this here:

https://bugs.launchpad.net/ubuntu/+source/fetchmail/+bug/1976361

I can also say for certain that using a backslash to escape quote characters
does not work for fetchmail.

> Wget (and nowadays also curl) does basically the same thing (but not 100%
identical).
> 

cURL simply takes all chars in the linefeed-terminated password field to be
the literals. So if a password is simple but has surrounding double quotes,
even then cURL treats the quotes themselves as part of the password. There is
also no concept of escaping characters with cURL’s parser.

If that’s similar to wget’s netrc parser, then perhaps whoever updates the
man page of one might consider updating the man page of the other.  (Because
cURL is also undocumented in this regard IIRC).

But I suspect there must be a significant difference between cURL and wget’s
parser because the password that wget said was a syntax error is one that cURL
would consider syntactically correct. cURL would simply treat all the quote
chars as literally being part of the password.  So if wget is expected to
behave the same in that regard, then there is a 3rd bug in wget.

> Interestingly, both ftp clients I looked at use a character/stream based
parser. This implies that quoted password can span multiple lines - the only
way to include e.g. a linefeed into the password (\ escaping doesn't allow
this).
> 
> Fetchmail (and also wget) uses a line-oriented parser that doesn't allow
linefeed in a password at all. And it uses what you call 'bash-style' quoting

>

This seems contradictory. That is, with the bash-style quoting it would
certainly be possible to include linefeeds. But I should clarify that
fetchmail does not document the password spec rules. It’s merely my
experimentation & testing that led me to characterize fetchmail as expecting
“bash-style quoting”. So it’s quite possible that you’re right on
that… that fetchmail quits when a linefeed is encountered, in which case
it’s wrong for me to call it bash—style quoting. Not sure what else to
call it then. But notibly, just as there are many ways to quote a string in
bash, the same password can also have many different representations.  E.g.
the following are the same result in fetchmail IIUC:

compl3x"'"p4SS'"w0rd%here'
"compl3x'"p4SS'"'w0rd%here
compl3x"'"p4SS'"'w0rd%here

> - something that is absolutely unneeded when it comes to embedding quotes in
passwords.
> 

There are two typical ways to represent embedded quotes:

1) Surrounding all single quotes with double quotes, and all double quotes
with single quotes.

2) Using an escape character, such as a backslash. But if the escape character
chosen is valid in a password string (such as backslash), then the parser must
also make an exception for the escape char itself (\\ → \).

Using one approach means you don’t need the other approach.

> My conclusion is that it is too late to set up a spec, and that fetchmail is
at least incompatible with other netrc parsers (to say it nicely).
> 

I disagree, but it doesn’t matter because the purpose of this bug report was
not to get a standard spec.

It would be possible for the netrc-ftp project to declare a spec in order to
get all future apps consistent. Existing apps could be regarded as
non-conformant but it wouldn’t necessarily require action; they would likely
be grandfathered & tolerated.

In any case, wget should document its own spec for what’s expected in the
.netrc file.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?62586>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/