bug#56342: TRAMP (sh) issues way too many commands, thus being very slow over high-ping networks

[Paul Pogonyshev]

> I will see whether I can do something along these lines.

Thank you. I'm just spewing out ideas here, you know the code better and what will and will not work.

> Perhaps changing the order: First try

> to insert the file contents, and if this errs out, a call with
> file-exists-p in order to raise the error. I'll play with this.

Yes, this would be a better way from my point of view. I don't care how slowly (unless it is like 50 times slower) it fails, that is an unlikely case. The most likely one, when the file exists, should be the benchmark.

> Doing it asynchronously would require a second connection to the remote
> host. Performance would rather degrade.

Maybe not really asynchronously, just let it return early, not waiting for the result? I'm not sure how TRAMP receives responses, but can it just keep executing commands sequentially, as now, but give control back to the caller in case of commands where the result doesn't really matter (cleanup, e.g. deleting a temporary file). Of course, this means that if an "important" command is issued right away, it has to wait for the response to the previous inessential command. But when such an inessential command is the last in a batch, this waiting would be effectively merged with Emacs' idling in the normal UI command loop, making things more responsive for the user.

> One thing you could do w/o code change is to change the value of
> remote-file-name-inhibit-cache.

Thank you for the hint, will try.

Paul

On Sun, 3 Jul 2022 at 20:47, Michael Albinus <michael.albinus@gmx.de> wrote:

Paul Pogonyshev <pogonyshev@gmail.com> writes:

Hi Paul,

> Maybe you could then single out commands that are supposed to return
> result nearly-instantly anyway, e.g. `stat' or `test' (though the
> latter I have suggested to replace with `stat' before). If they fail
> to produce the result in 10 seconds, the connection can be assumed
> dead, just as with `echo are you awake'. Commands that involve
> transmitting (potentially) large amount of data this or that way can
> work as now. So, you would usually avoid one extra command (which with
> a high ping means sth. like 0.2 s, already pretty noticeable in UI)
> and achieve the same result.
>
> Sanity checks are good. but not if they visibly slow down normal
> operation.

I will see whether I can do something along these lines.

> It might be easier to convince the rest of Emacs developers to use an
> extra variable to possibly skip `file-exists-p' on only remote files
> than to remove the call altogether. Though in either case it is
> required that `file-missing' error is handled properly which is,
> incidentally, easier to test if the call to `file-exists-p' is
> dropped. A target here would be `insert-file-contents' and whatever it
> calls, and this is a part of Emacs, not some external library.

Tramp has an own implementation of insert-file-contents, called
tramp-handle-insert-file-contents. And indeed, file-exists-p is called
in order to raise the file-missing error if needed. I have no idea how
we could generate this otherwise. Perhaps changing the order: First try
to insert the file contents, and if this errs out, a call with
file-exists-p in order to raise the error. I'll play with this.

OTOH, with longer cache expiration time (see below), file-exists-p
doesn't cost a roundtrip.

> Yet another idea: removing temporary should be done asynchronously (I
> haven't checked, maybe it's already the case, but likely not). The
> caller doesn't really care about call result and even if it has
> succeeded. Again, not sure if this is easy to achieve interface-wise,
> maybe it's from a higher level. But I suspect it's something like
> `with-temp-file' and can be optimized: one-time optimization are worth
> it even if that costs readability.

Doing it asynchronously would require a second connection to the remote
host. Performance would rather degrade.

One thing you could do w/o code change is to change the value of
remote-file-name-inhibit-cache. Its default value is 10, meaning caches
expire already after 10 seconds. With your slow connection, a higher
value should help already.

> Paul

Best regards, Michael.