Re: merge pick and scan

[Philipp Takacs]

[2022-04-21 14:49] Eric Gillespie <epg@pretzelnet.org>
> And it gets even worse: first you have to wait for pick to slowly
> search ALL THE FILES (within limiting message range you may have
> given it if you have any idea and often I did not), and then you
> wait for scan to slowly readdir everything, and THEN you finally
> get your results.  What I really want (and I doubt I'm alone) is
> scan lines as soon as a message is a HIT, so I can interrupt when
> the message I'm looking for comes across, without waiting for any
> further work.

It's even worse, because pick don't read a message once, it reads the
message for eatch action. There is a optimization to not perform eatch
action on the message, but this depends on the given actions and the
messages. So for example have the following command:

> pick -from bla -and -subject blub

All mails from bla will read twice. So if you know you have only a few
messages with the subject blub and many messages from bla, switching the
actions will increase the performance of pick. To avoid this pick could
read the message per field[0] and check for a match.

Yes the actions do early return, but this also depend on the messages
and the libc/fs could do some read optimication. But in general pick
does many IO-operations which aren't necesary.

In general there could be a pattern to read a mail/header before
processing and store the required fields. All the follow up processing
could be done without IO operation.

> On a modern filesystem on SSD in 2022 maybe nobody cares anymore.
> But over NFS in 2010, this mattered a lot.

I think this still matters, not in the way that we must optimice every
operation, but nmh sould avoid wasting time.

Philipp

[0] I only think of header search, because body search don't work
    good on most mails I get (MIME).