Re: Emacs HTTP libraries [was: Re: How to contribute new package to GNU ELPA?]

[Philip K.]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Philip K." <philipk@posteo.net>
>> Cc: arthur.miller@live.com, adam@alphapapa.net, rms@gnu.org,
>>      emacs-devel@gnu.org
>> Date: Tue, 22 Dec 2020 11:38:31 +0100
>> 
>> >> In that case, gnutls-negotiate seems to be the most suspicious function,
>> >> using over 50%-60% of the CPU time, at least on my machine. This also
>> >> makes sense, as TLS sites seem to take longer to load than regular,
>> >> non-encrypted ones.
>> >
>> > Please show the code you profiled and the fully expanded profile.
>> 
>> I sadly coudln't reproduce it, but this time the critical section looked
>> something like this:
>> 
>>         - url-retrieve-synchronously                               212  52%
>>          - url-retrieve                                            149  36%
>>           - url-retrieve-internal                                  149  36%
>>            - url-http                                              136  33%
>>             - url-http-find-free-connection                        135  33%
>>              - url-open-stream                                     135  33%
>>                 open-network-stream                                134  33%
>> 
>> when evaluating 
>> 
>>      (url-retrieve-synchronously "http://textboard.org/sexp/prog/index";)
>> 
>> in the *scratch* buffer. I used emacs -Q (GNU Emacs 27.1 (build 1,
>> x86_64-redhat-linux-gnu, GTK+ Version 3.24.22, cairo version 1.16.0) of
>> 2020-08-21), but I don't know why that should make any
>> difference. I repeated the same test on the master branch and the
>> results were basically the same (±5%).
>> 
>> Either way, this simple request took over 2.5 minutes, whereas curl
>> requires a quarter of a second. Note that this is even unencrypted, so
>> this is not even taking the encryption overhead into account.
>
> That's strange, because I get here much faster times: 0.6 sec (with 3
> GC cycles) on the first attempt, and less than 0.1 sec afterwards.

Why did it take three GC cycles? benchmark-run says

    (130.773534384 0 0.0)

and 

    (129.933227402 0 0.0)

for the first two runs, and

    (0.044592597 0 0.0)

for the third one, a bit later.

> How come it's so slow on your system?

I can't say for sure, but I don't think it is due to some specific local
configuration. I reran the code on a university computer a few
kilometres away, and the behaviour stays basically the same, even though
their network is a lot better than mine. But when I ran it on a VPS in
Berlin (I'm currently in southern Germany), it was a lot faster, despite
probably running the same build (whatever it in the Debian Buster
repository). I looked up where textboard.org is located, and it seems to
be in France, or it at the very least has to pass through some Proxy in
France. Another interesting observation is that connections from my home
to my university are basically instantaneous using Eww, and vice
versa. The Server in Berlin takes a lot longer to connect to my local
network and my university server. The only exception to this rule is
that eww doesn't seem to take that long to connect to my VPS.

None of these effects are observable using curl.

I guess this should be reported as a proper bug, instead of being
discussed here.

-- 
        Philip K.