bug#73798: 31.0.50; ERC 5.7: New extensibility focused match API

[J.P.]

Tags: patch

This follows loosely from bug#73580 and bug#68265.

It's well known that a favorite pastime of ERC users is griping about
its lack of an easily exploitable API for analyzing and manipulating
message content. The current landscape is indeed regrettably sparse,
with avenues either too rough and tumble, like at the protocol handling
level, or too zoomed out and refined, like at the user options level.
The traditional go-to solution (one certainly not known for its stellar
UX) lies somewhere in between. Often called the "message insertion"
phase (by me), this somewhat nebulous and often feared limbo period
between message receipt and display is home to the infamous
`erc-insert-modify-hook' and friends, where a good many modules,
including `match', do their dirty work.

Anyone familiar with the aforementioned bug#68265 will know that it's
very much about offering access before initial formatting takes place,
meaning just prior to insertion. Another TBD bug to be opened eventually
will explore the flip side: an easier way to examine and influence a
message's fate during insertion (e.g., determining which insertion hooks
run and how their arguments and return values take shape). As mentioned,
this current bug is meant to address modification immediately after
insertion, while a message's content and format are less violently in
flux but still being finalized in the narrowed buffer.

Here's the proposed documentation:


  File: erc.info, Node: Match API, Next: Options, Prev: Module Loading,
  Up: Advanced Usage

  5.6 Match API
  =============

  This section describes the low-level ‘match’ API introduced in ERC 5.7.
  For basic, options-oriented usage, please see the doc strings for option
  ‘erc-pal-highlight-type’ and friends in the ‘erc-match’ group.
  Unfortunately, those options often prove insufficient for more granular
  filtering and highlighting needs, and advanced users eventually outgrow
  them.  However, under the hood, those options all use the same
  foundational ‘erc-match’ API, which centers around a ‘cl-defstruct’
  “type” of the same name:

   -- Struct: erc-match predicate spkr-beg spkr-end body-beg sender nick
            command handler

       This is a ‘cl-struct’ type that contains some handy facts about the
       message being processed.  That message's formatted body occupies
       the narrowed buffer when ERC creates and provides access to each
       ‘erc-match’ instance.  To use this interface, you add a
       “constructor”-like function to the list ‘erc-match-types’:

        -- User Option: erc-match-types

            A hook-like list of functions, where each accepts the
            parameters named above as an ‘&rest’-style plist and returns a
            new ‘erc-match’ instance.  A function can also be a
            traditional ‘cl-defstruct’-provided constructor belonging to a
            “subtype” you've defined.

       The only slot you definitely need to specify is ‘predicate’.  Both
       it and ‘handler’ are functions that take a single argument: the
       instance itself.  As its name implies, ‘predicate’ must return
       non-‘nil’ if ‘handler’, whose return value ERC ignores, should run.

       A few slots, like ‘spkr-beg’, ‘spkr-end’, and ‘nick’, may surprise
       you.  The first two are ‘nil’ for non-chat messages, like those
       displayed for ‘JOIN’ events.  The ‘nick’ slot can likewise be ‘nil’
       if the sender of the message is a domain-style host name, such as
       ‘irc.example.org’, which it often is for informational messages,
       like ‘*** #chan was created on 2023-12-26 00:36:42’.

       To locate the start of the just-inserted message, use ‘body-beg’, a
       marker indicating the beginning of the message proper.  Don't
       forget: all inserted messages include a trailing newline.  If you
       want to extract just the message body's text, use the function
       ‘erc-match-get-message-body’:

        -- Function: erc-match-get-message-body match

            Takes an ‘erc-match’ instance and returns a string containing
            the message body, sans trailing newline and any leading
            speaker or decorative component, such as ‘erc-notice-prefix’.

  Although module authors may want to subclass this struct, everyday users
  can just instantiate it directly (it's “concrete”).  This is especially
  handy for one-off tasks or simple customizations in your ‘init.el’.  To
  do this, define a function that invokes its constructor:

       (require 'erc-match)

       (defvar my-mentions 0)

       (defun my-match (&rest plist)
         (apply #'erc-match
                :predicate (lambda (_) (search-forward "my-project" nil t))
                :handler (lambda (_) (cl-incf my-mentions))
                plist))

       (setopt erc-match-types (add-to-list 'erc-match-types #'my-match)
               erc-prompt (lambda () (format "%d!" my-mentions)))

  Here, the user could just as well shove the incrementer into the
  ‘predicate’ body, since ‘handler’ is set to ‘ignore’ by default
  (however, some frown at the notion of a predicate exhibiting side
  effects).  Likewise, the user could also choose to concentrate only on
  chat content by filtering out non-‘PRIVMSG’ messages via the slot
  ‘command’.

     For a detailed example showing how to use this API for more involved
  matching that doesn't involve highlighting, see the ‘notifications’
  module, which lives in ‘erc-desktop-notifications.el’.  Ignore the parts
  that involve adapting the global setup (and teardown) business to a
  buffer-local context.  Since your module is declared ‘local’, as per the
  modern convention, you won't be needing such code, so feel free to use
  utility functions like ‘erc-match-add-local-type’ directly in your
  module's definition.

  5.6.1 Highlighting
  ------------------

  Third-party modules likely want to manage and apply faces themselves.
  However, in a pinch you can just piggyback atop the highlighting
  functionality already provided by ‘match’ to support its many high-level
  options.

       (require 'erc-match)

       (defvar my-keywords
         `((foonet ("#chan" ,(rx bow (or "foo" "bar" "baz") eow)))))

       (defface my-face
         '((t (:inherit font-lock-constant-face :weight bold)))
         "My face.")

       (defun my-match (&rest plist)
         (apply #'erc-match-opt-keyword
                :data (and-let* ((chans (alist-get (erc-network) my-keywords))
                                 ((cdr (assoc (erc-target) chans)))))
                :face 'my-face
                plist))

       (setopt erc-match-types (add-to-list 'erc-match-types #'my-match))

  Here, the user leverages a handy subtype of ‘erc-match’, called
  ‘erc-match-opt-keyword’, which actually descends directly from another,
  intermediate ‘erc-match’ type:

   -- Struct: erc-match-traditional category face data part

       Use this type or one of its descendants (see below) if you want
       ‘erc-text-matched-hook’ to run alongside (after) the ‘handler’
       slot's default highlighter, ‘erc-match-highlight’, on every match
       for which the ‘category’ slot's value is non-‘nil’ (it becomes the
       argument provided for the hook's MATCH-TYPE parameter).

       Much more important, however, is ‘part’.  This slot determines what
       portion of the message is being highlighted or otherwise operated
       on.  It can be any symbol, but the ones with predefined methods are
       ‘nick’, ‘message’, ‘all’, ‘keyword’, ‘nick-or-keyword’, and
       ‘nick-or-mention’.

       The default handler, ‘erc-match-highlight’, does its work by
       deferring to a purpose-built “method” meant to handle ‘part’-based
       highlighting:

        -- Method on erc-match-traditional: erc-match-highlight-by-part
                 instance part

            You can override this method by “specializing” on any
            subclassed ‘erc-match-traditional’ type and/or non-reserved
            PART, such as one known only to your ‘init.el’ or (informally)
            associated with your package by its library “namespace”.

  You likely won't be needing these, but for the sake of completeness,
  other options-based types similar to ‘erc-match-opt-keyword’ include
  ‘erc-match-opt-current-nick’, ‘erc-match-opt-fool’, ‘erc-match-opt-pal’,
  and ‘erc-match-opt-dangerous-host’.  (If you're familiar with this
  module's user options, you'll notice some parallels here.)

     And, finally, here's a more elaborate, module-like example demoing
  highlighting based on the ‘erc-match-traditional’ type:

       ;; -*- lexical-binding: t; -*-

       (require 'erc-match)
       (require 'erc-button)

       (defvar my-keywords
         `((foonet ("#chan" ,(rx bow (or "foo" "bar" "baz") eow)))))

       (defface my-keyword '((t (:underline (:color "tomato" :style wave))))
         "My face.")

       (defun my-get-keyword ()
         (and-let* ((chans (alist-get (erc-network) my-keywords))
                    ((cdr (assoc (erc-target) chans))))))

       (cl-defstruct (my-match (:include erc-match-opt-keyword
                                         (part 'keyword)
                                         (data (my-get-keyword))
                                         (face 'my-keyword))
                               (:constructor my-match)))

       (setopt erc-match-types (add-to-list 'erc-match-types #'my-match))

       (cl-defmethod erc-match-highlight-by-part ((instance my-match)
                                                  (_ (eql keyword)))
         "Highlight keywords by merging instead of clobbering."
         (dolist (pat (my-match-data instance))
           (goto-char (my-match-body-beg instance))
           (while (re-search-forward pat nil t)
             (erc-button-add-face (match-beginning 0) (match-end 0)
                                  (my-match-face instance)))))

  (Note that in the method body, you _could_ technically skip to the
  beginning of the last match for the first go around because the match
  data from the ‘predicate’ is still fresh.)


Some canned Q&As:

1. ERC is already famously bogged down by ill-conceived contracts, why
   add another?

   This isn't just some new feature. It's a revamping and refactoring of
   the `match' library that exposes much needed foundational seams to
   users while encapsulating some compat-related business and other
   unruly minutiae, such as text-property twiddling. It also includes a
   partial refactoring of the `notifications' module (housed in
   erc-desktop-notifications.el) that makes heavy use of this new API
   and thus serves as a reference implementation for certain flavors of
   non-highlight-centric matching.

   But TBF, the "extensibility focused" bit from this bug's subject line
   is somewhat of a stretch. In reality, such extensibility is merely a
   knock-on effect of bending over backward to prioritize compatibility.
   But that doesn't mean it can't also help tamp down on contracts
   proliferation in ERC. For those unfamiliar, users often rightly call
   for some existing option to be expanded or spun off into a variant
   more attuned to a specific context: e.g., option erc-foo needs to be
   network/channel/nickname aware. One way to push back on such demands
   is to point to a not-too-painful existing workaround.

2. Why not combine the predicate and handler slots of match objects?

   We certainly could (and very well still might), in which case this
   would end up looking more like a traditional hook arrangement. The
   original idea was to promote code reuse and separation of concerns by
   enforcing a bit of structure. However, it's certainly true that hook
   members could just as easily perform the predicate and handler logic
   themselves.
   
3. Why the separate message-body getter instead of a dedicated slot?

   To retain that state in match objects would mean having to update it
   between handler runs because users are invited to modify the text.
   (This will be mentioned in a code comment.)

4. Why not EIEIO?

   While such a move might help "consumers" of the API to some degree,
   the expanded feature set won't really have the same impact on
   implementation and upkeep of the library. And though ERC does
   currently pull in dependencies that in turn require EIEIO, that may
   not always be so.

(Note that while the attached patches target ERC 5.7, they don't include
changes to bump the version, etc.)

For anyone interested in this bug, feedback of any form is always
welcome.

Thanks!


In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.43, cairo version 1.18.0) of 2024-10-07 built on localhost
Repository revision: ff4de9eff30ade164655354f71e7cbca48135858
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12401002
System Description: Fedora Linux 40 (Workstation Edition)

Configured using:
 'configure --enable-check-lisp-object-type --enable-checking=yes,glyphs
 'CFLAGS=-O0 -g3'
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr compile comint ansi-osc ansi-color ring comp-run
bytecomp byte-compile comp-common rx emacsbug message mailcap yank-media
puny dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived
epg rfc6068 epg-config gnus-util text-property-search time-date subr-x
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo gtk
x-toolkit xinput2 x multi-tty move-toolbar make-network-process
native-compile emacs)

Memory information:
((conses 16 59412 9167) (symbols 48 6747 0) (strings 32 16822 4165)
 (string-bytes 1 492565) (vectors 16 11411)
 (vector-slots 8 139223 13079) (floats 8 21 4) (intervals 56 248 0)
 (buffers 984 11))

0001-5.7-Use-speaker-prefix-end-marker-in-ERC-insertion-h.patch
Description: Text Data

0002-5.7-Introduce-lower-level-erc-match-API.patch
Description: Text Data

0003-5.7-Use-erc-match-type-API-for-erc-desktop-notificat.patch
Description: Text Data