Re: [Bug-wget] Anyone want to add libcares support to wget?

[Tim Ruehsen]

On Friday 18 March 2016 08:58:49 Ben Greear wrote:
> On 03/18/2016 02:33 AM, Tim Ruehsen wrote:
> > Tim
> > 
> > On Thursday 17 March 2016 09:36:47 Ben Greear wrote:
> >> On 03/17/2016 08:58 AM, Giuseppe Scrivano wrote:
> >>> Hi Ben,
> >>> 
> >>> Ben Greear <address@hidden> writes:
> >>>>> Did you consider using a container (e.g. docker) for such a task ?
> >>>>> Easy
> >>>>> to set up and you'll have your feature not only for wget. IMO, that is
> >>>>> much more flexible. (It was Giuseppe's idea during a private talk).
> >>>> 
> >>>> Containers will not work for me.  I need to scale to thousands of
> >>>> instances on modest hardware.  I'm certain the libcares and binding
> >>>> approach will work because we do similar things with curl and other
> >>>> programs already.
> >>> 
> >>> how modest is this hardware?
> >>> 
> >>> I tried to build a minimal Docker container for wget on top of Alpine:
> >>> 
> >>> Dockerfile:
> >>> 
> >>> FROM alpine
> >>> RUN apk add --update wget
> >>> WORKDIR /out
> >>> ENTRYPOINT ["/usr/bin/wget"]
> >>> 
> >>> and simply running (192.168.1.13 is an internal DNS server):
> >>> 
> >>> docker run --dns=192.168.1.13 --rm -v $(pwd):/out:Z wget wdserver
> >>> 
> >>> takes around 5M, plus the docker daemon.
> >>> 
> >>> If this is not enough, you can even just run wget in a chroot, and
> >>> provide a different /etc/resolv.conf.
> >> 
> >> I really am not at all interested in LD_PRELOAD, docker, chroot, or any
> >> other such a thing.  I have 15+ years invested in my current
> >> architecture,
> >> and it works very well for me.  It absolutely cannot be easier to manage
> >> docker instances on random platforms vs using a few cmd-line switches to
> >> wget.
> >> 
> >> I know that this feature will not be useful for most people, but then
> >> again, none of us can know what some user might be able to do with such
> >> a feature.>> 
> >>   I know that some users of curl + libcares wanted the ability to specify
> >> 
> >> certain DNS servers to get around oppressive firewalls (on systems they
> >> could not have root privs), for instance.
> >> 
> >> I don't mind making the changes to wget.  I thought if someone was
> >> interested in a project, then I'd be happy to work with them (and pay
> >> them,
> >> etc).
> > 
> > Why not put an offer here (with an exact description of what to deliver,
> > OS, etc) ?
> 
> A lot of lists do not like such postings, so I was trying to be a bit
> discreet.
> 
> Anyway, I'll pay $500 USD for a working implementation, and $1000 if someone
> can get it coded up well enough that it is accepted upstream.  If someone
> thinks this is not an appropriate amount, let me know your suggestions.
> 
> Features:
> 
>   * Ability to link with libcares for all DNS lookup requests.
>   * Ability to specify DNS server list for libcares.  At least IPv4.  IPv6
> is nice but not required.
>   * Ability to bind libcares to a local IP address.  Ability to also bind to
> local network interface is nice but not required.  libcares library already
> supports this, see 'curl' for examples of how this was done.
>   * All of this configurable with cmd-line arguments.
>   * Ability to specify which libcares to include and link against (with
> ./configure option or similar)
> 
>   * OS:  Linux, modern Fedora.

Thanks. I appreciate putting cards on the table.

I suggest that anyone interested should not just start programming but talk 
with Ben before. It would be wasted time if several people work on a solution 
and just the first one gets paid. Just said this to avoid tears.

Regards, Tim

signature.asc
Description: This is a digitally signed message part.