Re: [Bug-wget] [PATCH] Update HSTS info

[Ander Juaristi]

Hi Daniel,

I've amended the previous patches based on your feedback. Well, the first patch 
only. I re-attach the second one without modifications.

On 08/10/2015 06:06 PM, Daniel Kahn Gillmor wrote:
> There is no pointer or reference here to what the "correct HSTS database
> format used by Wget" is.  providing a brief pointer (e.g. "look at file
> doc/FOO in the wget source" or "see https://example.org/wget-hsts-db";)
> would be useful.

I have now added a concise description of how the HSTS database should look 
like.

> Is it possible that a user would want to decouple reading from
> (respecting) an HSTS file and writing to it?

During development, neither Tim nor I ever talked about separating read and 
write. It just didn't happen.

But I guess you're right. There should be a way of separating reads and writes. 
The default behaviour should still be kept the same, but there should be a way 
of decoupling those processes, as you say. It should not be prohibitively 
expensive, and someone might benefit from such features.

I've been thinking about a couple of possibilities here, based on the examples 
you provided. Discussion starts here ;-)

    * Make the HSTS database read-only. Load the HSTS entries contained there 
but do not rewrite the file. This could be governed by an extra command-line 
switch, like '--hsts-read-only'. It could be passed together with 
'--hsts-file', for example:

        wget --hsts-read-only --hsts-file=~/my-fixed-hsts-file

    * Do not use an HSTS database at all. HSTS would be handled internally but 
there would be no interaction with the file system. Something like 
'--no-hsts-file'.

    * <Your option here>

> This sounds like there might still be a race condition where one
> process's changes get clobbered.  can we make any atomicity guarantees
> for users who might be concerned about this?

You're right. My fault not to take this into account. This could be fixed with 
flock/fcntl. I think they're both in gnulib.

These last two issues require code changes. I'll take the responsibility to fix 
them, but outside of GSoC. The first one requires a bit of consensus before 
coding anything, but the second one seems a bit more straightforward. For now, 
I attach the two previous patches. The first one (the HSTS docs) amended with 
dkg's suggestions. If there're no more complaints, I think they can be pushed, 
because Wget's behaviour has not changed yet. When we implement any of the 
ideas proposed above, we'll update the docs.

> I'd normally characterize these threats as "privacy concerns", not
> necessarily "security concerns".  users of wget might understand them
> most closely as offering "cookie-equivalent" mechanisms for some
> http/https clients.

I think that's a widespread misconception. It's true many users map HSTS to HTTP cookies, 
but IMO that's a mistake. HTTP cookies and HSTS might be similar in some aspects, but 
they're two mechanisms that were designed for different purposes. HTTP cookies bridge the 
gap between HTTP's traditional stateless nature, and the stateful needs of modern 
Internet users. On the other hand, HSTS was conceived as a workaround for some security 
threats. It's true that these threats might more specifically target privacy, but I think 
it's an error for us, GNU developers, to keep on feeding the "HSTS == cookie" 
misconception.


Regards,
- AJ

0001-Extra-debug-traces-for-HSTS.patch
Description: Text Data

0002-Updated-HSTS-documentation.patch
Description: Text Data