[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Parallel downloads
From: |
Pierre Neidhardt |
Subject: |
Re: Parallel downloads |
Date: |
Wed, 13 Nov 2019 20:34:45 +0100 |
Mark H Weaver <address@hidden> writes:
> For these reasons, I'm inclined to think that parallel downloads is the
> wrong approach. If a single download process is not making efficient
> use of the available bandwidth, I'd be more inclined to look carefully
> at why it's failing to do so. For example, I'm not sure if this is the
> case (and don't have time to look right now), but if the current code
> waits until a NAR has finished downloading before asking for the next
> one, that's an issue that could be fixed by use of HTTP pipelining,
> without multiplying the memory usage.
I think so too.
More generally, the Guix daemon jobs can be categorized in 2: downloads
and builds. These 2 categories demands almost complementary hardware
resources for the local machine.
With that in mind, we could have 2 pipelines: one for the builds and one
for the downloads. This, I think, is general enough that it could be
used by default and improve performance for everyone, regardless of your
Internet bandwidth or CPU power.
Cheers!
--
Pierre Neidhardt
https://ambrevar.xyz/
signature.asc
Description: PGP signature
- Re: Parallel downloads, (continued)
- Re: Parallel downloads, Ludovic Courtès, 2019/11/12
- Re: Parallel downloads, John Soo, 2019/11/12
- Re: Parallel downloads, zimoun, 2019/11/12
- Re: Parallel downloads, Efraim Flashner, 2019/11/13
- Re: Parallel downloads, zimoun, 2019/11/13
- Re: Parallel downloads, Leo Famulari, 2019/11/12
- Re: Parallel downloads, Ludovic Courtès, 2019/11/17
- Re: Parallel downloads, Mark H Weaver, 2019/11/13
- Re: Parallel downloads, Pierre Neidhardt, 2019/11/13
- Re: Parallel downloads, Leo Famulari, 2019/11/13
- Re: Parallel downloads,
Pierre Neidhardt <=
- Re: Parallel downloads, Ludovic Courtès, 2019/11/17
- Re: Parallel downloads, Bengt Richter, 2019/11/06