lynx-dev Keys, an attempt to understand

[Klaus Weide]

It seems we have a kind of mess with various ways in which keyboard
input is mapped at various levels.  So let me try to take a step back,
and try to understand what was originally there, and why; how it was
meant to be used; and why we have a problem now.  I am trying to
discover and express some underlying principles, which involves
guessing; you may not agree with that analysis, but then please give
your interpretation.

Basically and "originally", we have two levels of mapping:

        1.                  2.                    3.

        byte sequence  ---> lynxkeycode      ---> lynxactioncode
                                         (or ---> lynxeditactioncode)

(I am making these terms up here, they don't necessarily correspond
to the terms used in code or documentation)
Examples:

        "a"            ---> 'a'               ---> 48
                                                   LYK_ADD_BOOKMARK

        "\033[B"       ---> 0x101             ---> 25
        "\033OB"            DNARROW                LYK_NEXT_LINK
        "\033B"

Let's look at this in some more detail, right to left.  The
lynxaction code is symbol standing for a number assigned by the lynx
code, to represent a distinct action to be performed.  The lynxkeycode
is a number (and possibly symbol) assigned by the lynx code to
represent distinct keys.  The 2. -> 3. mapping mechanism is simply
using the lynxkeycode as an index into a table of lynxactioncodes,
but there are more than one of these tables and they may change on
the fly depending on context (DIRED, links-are-numbered state,...).
There is also a table mapping lynxkeycodes to line editor actions
(LYE_* instead of LYK_*), which is used (of course) for line editing.

Column 1. has the keyboard input in the form Lynx really receives it.
In the age of terminal, that meant function keys (in a wide sense, not
just Fnn but everything that needs more to represent than a char) are
sequences of bytes read.  The 1. -> 2. mapping isn't done in Lynx via
a table or sequence of tables, but, in its simplest (actually not so
simple) form, by a function reading chars, recognizing well-known
sequences, and returning the appropriate lynxkeycode.

Is this two-level procedure necessary, or is it already too
complicated in principle?  I say it is A Good Thing, and think that
should be obvious enough so I won't go into (probable) motivation.
Note that the 2. -> 3. mappings are the province of user preference,
while the 1. -> 2. mappings are in the area of basic
getting-it-to-work functionality - they don't need to change with user
preference, but need to be adapted as new terminals arrive on the
scene.

Now a still closer look at those intermediate lynxkeycodes.
I regard the following properties as essential:

(1) The lynxkeycodes are assigned by lynx.
(2) Each lynxkeycode represents a distinct key.
(3) Distinct keys have different lynxkeycodes.

    With "distinct keys" I mean an abstraction that works across
    systems - an "F1" key is the same key at this level of abstraction,
    whether pressing it sends "\033[[A" or "\033OP" or notifies the
    program in some other way of being pressed; a key labelled "Help"
    is the same key as the one labelled "Hilfe" on a different keyboard;
    but the Help key is not the same as the F1 key.

With essential I don't mean "necessary to work", or that the lynxkeycodes
were ever actually fully implemented that way; but rather what is part of
their indented function, what they were meant to be.

I regard the following as near-essential:

(4) There is a special value indicating "an unrecognized key was pressed".
    In the Lynx code that value is 0.
(5) There is a special value DO_NOTHING indicating "no key was pressed,
    after all".  Of course no such "No key" key exists, but a function
    sometimes wants to tell another "pretend no key was pressed".  This
    value should be distinct from that for unrecognized key.

Then there are some "It makes so much sense, no reason to think about
changing it" properties - wheter essential or not:

(6) the space of lynxkeycodes encompasses all one-byte values whether
    there is always a key that can send them or not, those chars are
    maped trivially into the space of lynxkeycode values.

In contrast, the following observations can be made in the Lynx code.
These are accidental properties - they could be changed (some have already)
without changing the basic function of the lynxkeycodes.

(7) The lynxkeycodes are assigned sequentially.
(8) DO_NOTHING is the "last" value in some sense.
    In principle, there isn't any requirement that DO_NOTHING has any
    specific value.

I leave out examples of deviations from the ideal here, as well as probable
reasons for them and for the "accidental" properties.  We could get into
that later.

Then along comes newer curses.  (I don't pretend that this is the historical
squence, I am imposing some logical order.)  There is keypad(), providing for
recognition of keys such that Lynx dosen't need to know the details.  It comes
with its own space of codes/symbols KEY_*.  Not a problem. it fits into the
picture as

                    - - - - - > - - - - -
       1.        /        1a.             \ 2.                3.
       byte sequence ---> curseskeycode --> lynxkeycode  ---> lynxactioncode


Example: (assuming "\033[B" is recognize by curses, while "\033B" isn't)

       "\033B"   - - - - - > - - - - - -
                                         \
       "\033[B"      ---> e.g.0402      --> 0x101        ---> 25
                          KEY_DOWN          DNARROW           LYK_NEXT_LINK

The KEY_* codes just provide another way for getting the lynxkeycode, Lynx
doesn't need to know the actual values assigned by curses.h to KEY_* macros
(since mapping -> 2. is done in switch code, not table).  There's also
still the old recognition for well-known keys curses may be ignorant about.
The KEY_* curseskeycodes have the same function for the curses library
as the lynxkeycodes for Lynx - an abstraction of possible keys - but the
values for the two kinds are assigned by different parties as they choose.

Then comes slang.  For the longest time, this hasn't changed anything for
key mapping (using the old 1. -> 2. method).  Slang keycodes (yet another
namespace) seem to be used now, I haven't looked too closely, but as far
as Lynx is concerned this fits in in the same way as the 1a of curses.

Then comes the recently discovered (for me :) ) customizable "lynx-keymaps"
file.  (Again a reminder that this is not historical sequence.)  This
provides yet another way to map byte squences to lynxkeycodes, either
directly or indirectly via KEY_* curseskeycodes or some symbolic names.
It is still another way of 1. -> 2. mapping.

Then comes Lynx for DOS, in various flavours.  Suddenly input isn't a
sequence of bytes, but some extended codes gotten somehow from the systems.
Codes for keys beyond single bytes are provided in some form, assigned by
the libraries used in the various flavours in incompatible ways (between
each other as well as anything already in the Lynx code).

The logical way to deal with this would have been to treat these new kinds
of codes as yet more variations on the 1a. theme: Something not assigned by
us, which has to be translated to how we represent keys.  But this is not
what happened.  Instead, the space of new codes was identified with the
space of lynxkeycodes.  This breaks several of the properties of
lynxkeycodes, essential and otherwise, and ever since there was a mess...

I can understand why things were done this way:
- A lot more keys wanted mapping than had lynxkeycodes.
- Hooking into the existing table mechanism by just extending existing
  tables was convenient.  Especially since there was no table for
  1. -> 2. or 1a. -> 2. that could be hooked into.
- Only a few lynxkeycodes were used beyond 0100.
- The supposed "right" mechanism isn't spelled out anywhere.
- Historically lynxkeycodes haven't been added to with definitions
  for new keys, as one would expect from (my understanding of) their
  basic function.

So I am not blaming anyone.  Instead the question is what to do now.
I can see three possibilities:
1.) Continue a before; muddle along somehow, solve problems as they
    come up.  Likely there'll be the same kind of discussion again
    when the next person wants to add a new code for some reason.
2.) Add another mapping level, those new codes -> lynxkeycodes.
3.) Keep on identifying those new codes == lynxkeycodes, but make
    it work consistently somehow, and spell out the new rules.
    At least this seems to require moving the traditionally assigned
    lynxkeycodes from 0x100 to some other area, or use some offset for
    the new codes' positions in the table; some kind of assignment
    policy or convention.

  Klaus