[Bug-wget] ping (Re: I am seeing problems with wget-1.14.96-38327 doing gnutls secure sessions.)

[SciFi]

ping

I guess I need to remind about this bug,
I haven't opened a real bugzilla report, tho.
Shall I?

FWIW, I've changed to the timeout=0 setting,
which did let the httpS code work.
I'll need to have a non-infinite setting
for some projects I have that use wget.

And I've hand-applied the patch below.
No ill effects there.

Happy Holidays!


On Mon, 04 Nov 2013 21:24:32 +0100, Tim Rühsen wrote:
> 
> Hi Sci-Fi @ hush.ai, found a prob on your XPI (nice rhyme !)
> 
> You problem is reproducable here by using
>       -e timeout=20 -e check-certificate=off 
> 
> A workaround is 
>       -e timeout=0
> 
> It must be some sort of regression, as you say.
> I have no time to dig, but maybe my observation might help someone to find it.
> 
> 
>> Certificates loaded: -1250
> ? Holy sheepshit, what is this ?
> GNUTLS_E_UNIMPLEMENTED_FEATURE returned by 
> gnutls_certificate_set_x509_system_trust().
> 
> Fixed in attached patch.
> 
> Tim
> 
> 
> Am Montag, 4. November 2013, 16:36:56 schrieb SciFi:
>> Hi,
>> 
>> (I am still here, still running OSX 10.6.8
>>  with all security updates etc.)
>> 
>> I've compiled the 1.14.96-38327 tarball here.
>> 
>> With it, I'm suddenly getting retries when I need to
>> fetch something with https
>> (while regular http seems ok)
>> no matter what server I need to pull from.
>> 
>> I also updated gnutls to 3.2.6
>> and nettle to 2.7
>> just in case
>> but no help in this regard.
>> 
>> For example, here's a wget of
>> the nightly Enigmail build
>> 
>> in debug mode:
>> > $ wget -d 
>> > https://www.enigmail.net/download/nightly/enigmail-nightly-all.xpi DEBUG
>> > output created by Wget 1.14.96-38327 on darwin10.8.0.
>> > 
>> > URI encoding = ‘UTF-8’
>> > --2013-11-04 10:06:45-- 
>> > https://www.enigmail.net/download/nightly/enigmail-nightly-all.xpi
>> > Certificates loaded: -1250
>> > Resolving www.enigmail.net (www.enigmail.net)... 217.26.54.154
>> > Caching www.enigmail.net => 217.26.54.154
>> > Connecting to www.enigmail.net (www.enigmail.net)|217.26.54.154|:443...
>> > connected. Created socket 4.
>> > Releasing 0x01091670 (new refcount 1).
>> > WARNING: No certificate presented by www.enigmail.net.
>> > 
>> > ---request begin---
>> > GET /download/nightly/enigmail-nightly-all.xpi HTTP/1.1
>> > User-Agent: Wget/1.14.96-38327 (darwin10.8.0)
>> > Accept: */*
>> > Host: www.enigmail.net
>> > Connection: Keep-Alive
>> > 
>> > ---request end---
>> > HTTP request sent, awaiting response... Read error (Success.) in headers.
>> > Retrying.
>> > 
>> > --2013-11-04 10:06:47--  (try: 2) 
>> > https://www.enigmail.net/download/nightly/enigmail-nightly-all.xpi Found
>> > www.enigmail.net in host_name_addresses_map (0x1091670)
>> > Connecting to www.enigmail.net (www.enigmail.net)|217.26.54.154|:443...
>> > connected. Created socket 4.
>> > Releasing 0x01091670 (new refcount 1).
>> > WARNING: No certificate presented by www.enigmail.net.
>> > 
>> > ---request begin---
>> > GET /download/nightly/enigmail-nightly-all.xpi HTTP/1.1
>> > User-Agent: Wget/1.14.96-38327 (darwin10.8.0)
>> > Accept: */*
>> > Host: www.enigmail.net
>> > Connection: Keep-Alive
>> > 
>> > ---request end---
>> > HTTP request sent, awaiting response... Read error (Success.) in headers.
>> > Retrying.
>> > 
>> > --2013-11-04 10:06:49--  (try: 3) 
>> > https://www.enigmail.net/download/nightly/enigmail-nightly-all.xpi Found
>> > www.enigmail.net in host_name_addresses_map (0x1091670)
>> > Connecting to www.enigmail.net (www.enigmail.net)|217.26.54.154|:443...
>> > connected. Created socket 4.
>> > Releasing 0x01091670 (new refcount 1).
>> > WARNING: No certificate presented by www.enigmail.net.
>> > 
>> > ---request begin---
>> > GET /download/nightly/enigmail-nightly-all.xpi HTTP/1.1
>> > User-Agent: Wget/1.14.96-38327 (darwin10.8.0)
>> > Accept: */*
>> > Host: www.enigmail.net
>> > Connection: Keep-Alive
>> > 
>> > ---request end---
>> > HTTP request sent, awaiting response... Read error (Success.) in headers.
>> > Retrying.
>> > 
>> > ^C
>> 
>> I can fetch this file ok
>> with 1.14.96-38327
>> if I use plain http.  ;)
>> 
>> 
>> I saved the current stable 1.14 build of wget
>> and it fetches from https ok.
>> So this might be a regression of some sort.
>> 
>> My ~/.wgetrc (for all wget versions/sessions shown here):
>> > $ cat ~/.wgetrc
>> > tries = 0
>> > continue = on
>> > timestamping = on
>> > timeout = 20
>> > waitretry = 5
>> > random_wait = on
>> > #inet4_only = on
>> > #prefer_family = IPv4
>> > retry_connrefused = on
>> > check-certificate = off
>> > trust-server-names = on
>> > #content-on-error = on
>> > auth-no-challenge = on
>> > ca-certificate = /usr/local/share/wget/cacert.pem
>> > robots = off
>> > #load-cookies = /Users/scifi/Library/Application
>> > Support/Camino/cookies.txt
>> 
>> My compile parms:
>> > $ wget --version
>> > GNU Wget 1.14.96-38327 built on darwin10.8.0.
>> > 
>> > +digest +https +ipv6 +iri +large-file +nls +ntlm +opie +ssl/gnutls
>> > 
>> > Wgetrc:
>> >     /Users/scifi/.wgetrc (user)
>> >     /usr/local/etc/wgetrc (system)
>> > 
>> > Locale:
>> >     /usr/local/share/locale
>> > 
>> > Compile:
>> >     gcc-4.2 -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc"
>> >     -DLOCALEDIR="/usr/local/share/locale" -I. -I../lib -I../lib
>> >     -I/usr/local/ssl/include -I/usr/X11/include -I/usr/local/include
>> >     -I/WhichXcode/Headers/FlatCarbon -I/usr/include
>> >     -I/usr/local/include -Os -mtune=core2 -march=core2
>> >     -force_cpusubtype_ALL -arch i386
>> > 
>> > Link:
>> >     gcc-4.2 -Os -mtune=core2 -march=core2 -force_cpusubtype_ALL -arch
>> >     i386 -Os -mtune=core2 -march=core2 -force_cpusubtype_ALL -arch i386
>> >     -L/usr/local/lib -L/usr/local/lib -liconv -L/usr/local/lib -lintl
>> >     -Wl,-framework -Wl,CoreFoundation -lnettle -L/usr/local/lib
>> >     -lgnutls -L/usr/local/ssl/lib -L/usr/local/lib/libquicktime
>> >     -L/usr/X11/lib -lnettle -lhogweed -lgmp /usr/lib/libz.dylib
>> >     -lp11-kit -lintl /usr/lib/libpthread.dylib -lz -L/usr/local/ssl/lib
>> >     -L/usr/local/lib/libquicktime -L/usr/local/lib -L/usr/X11/lib
>> >     -L/usr/lib -lidn -lpcre ftp-opie.o gnutls.o http-ntlm.o
>> >     ../lib/libgnu.a
>> > 
>> > Copyright (C) 2011 Free Software Foundation, Inc.
>> > License GPLv3+: GNU GPL version 3 or later
>> > <http://www.gnu.org/licenses/gpl.html>.
>> > This is free software: you are free to change and redistribute it.
>> > There is NO WARRANTY, to the extent permitted by law.
>> > 
>> > Originally written by Hrvoje Niksic <address@hidden>.
>> > Please send bug reports and questions to <address@hidden>.
>> 
>> Of course I would much-rather use Secure mode
>> rather than open-clear mode
>> if for no other reason than to
>> tell TPTB to stop spying on everyone.
>> If ya git my gist.
>> ;)
>> 
>> 
>> FWIW, thanks for keeping this project alive.

> From 60ee1abcad86dbeb542688d46983512b59ab2c85 Mon Sep 17 00:00:00 2001
> From: Tim Ruehsen <address@hidden>
> Date: Mon, 4 Nov 2013 21:22:41 +0100
> Subject: [PATCH] fix number of certificates in debug msg
> 
> ---
>  src/ChangeLog | 4 ++++
>  src/gnutls.c  | 4 ++--
>  2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/src/ChangeLog b/src/ChangeLog
> index 42ce3e4..2c87ee8 100644
> --- a/src/ChangeLog
> +++ b/src/ChangeLog
> @@ -1,3 +1,7 @@
> +2013-11-04  Tim Ruehsen  <address@hidden>
> +
> +     * gnutls.c (ssl_init): fix number of certificates in debug msg
> +
>  2013-11-02  Giuseppe Scrivano  <address@hidden>
>  
>       * http.c (gethttp): Increase max header value length to 512.
> diff --git a/src/gnutls.c b/src/gnutls.c
> index 9b4b1ec..715aadb 100644
> --- a/src/gnutls.c
> +++ b/src/gnutls.c
> @@ -104,6 +104,8 @@ ssl_init (void)
>     * Also use old behaviour if the CA directory is user-provided.  */
>    if (ncerts <= 0)
>      {
> +      ncerts = 0;
> +
>        ca_directory = opt.ca_directory ? opt.ca_directory : "/etc/ssl/certs";
>        if ((dir = opendir (ca_directory)) == NULL)
>          {
> @@ -118,8 +120,6 @@ ssl_init (void)
>            size_t dirlen = strlen(ca_directory);
>            int rc;
>  
> -          ncerts = 0;
> -
>            while ((dent = readdir (dir)) != NULL)
>              {
>                struct stat st;