rx and rx-to-string handle (regexp VAR) differently

[Kévin Le Gouguec]

Hello Emacs!

I am using (rx-define NAME (regexp STRING-VAR)) to give friendlier names
to regular expressions that I have lying around, written out manually
without rx.  To my surprise, rx accepts NAME, but rx-to-string does not.

For example:

  (setq foo-regexp "fo+")
  (rx-define foo (regexp foo-regexp))
  (message "(rx foo): %s"
           (rx foo))
  (message "(rx-to-string '(seq foo) t): %s"
           (rx-to-string '(seq foo) t))

This yields:

  (rx foo): fo+
  rx--translate-regexp: rx ‘regexp’ form with non-string argument

The backtrace from rx-to-string suggests that the symbol foo-regexp is
not substituted for its value:

  Debugger entered--Lisp error: (error "rx ‘regexp’ form with non-string 
argument")
    signal(error ("rx ‘regexp’ form with non-string argument"))
    error("rx `regexp' form with non-string argument")
⇒   rx--translate-regexp((foo-regexp))
⇒   rx--translate-form((regexp foo-regexp))
⇒   rx--translate((regexp foo-regexp))
⇒   rx--translate-symbol(foo)
⇒   rx--translate(foo)
    mapcar(rx--translate (foo))
    rx--translate-seq((foo))
    rx--translate-form((seq foo))
    rx--translate((seq foo))
    rx-to-string((seq foo) t)
    (message "(rx-to-string '(seq foo) t): %s" (rx-to-string '(seq foo) t))

I can circumvent this by running rx-define through eval:

  (setq foo-regexp "fo+")
  (eval `(rx-define foo (regexp ,foo-regexp)))
  (message "(rx foo): %s"
           (rx foo))
  (message "(rx-to-string '(seq foo) t): %s"
           (rx-to-string '(seq foo) t))

AFAIU this ensures that foo-regexp is substituted for its value before
rx-define is called, so both rx and rx-to-string end up processing a
(regexp "fo+") form.  This does yield:

  (rx foo): fo+
  (rx-to-string ’(seq foo) t): fo+

Is there a bug to file here, or have I missed something in the
documentation that explains why rx can work with (regexp VAR) but
rx-to-string cannot?  "(elisp) Rx Constructs" says:

> ‘(regexp EXPR)’
> ‘(regex EXPR)’
>      Match the string regexp that is the result from evaluating the Lisp
>      expression EXPR.  The evaluation takes place at call time, in the
>      current lexical environment.

Re(-re)ⁿ-reading that last sentence makes me tentatively conclude that
foo-regexp is not in rx-to-string's "current lexical environment", but
somehow is in rx's?  "(elisp) Lexical Binding" makes me doubt that
interpretation:

>    Here is how lexical binding works.  Each binding construct defines a
> “lexical environment”, specifying the variables that are bound within
> the construct and their local values.  When the Lisp evaluator wants the
> current value of a variable, it looks first in the lexical environment;
> if the variable is not specified in there, it looks in the symbol’s
> value cell, where the dynamic value is stored.

I don't see anything in there that suggests that things would be
different for rx or rx-to-string; AFAIU in those (message …) forms,
there is no "binding construct" at play (assuming this designates let &
friends), so in both cases the "dynamic value" of foo-regexp should be
retrieved, which I'd expect to be the value set by the (setq …) form?

(FWIW, using defvar instead of setq yields the same results)


Anyhoo; for now, I have my (eval …) workaround (which admittedly is not
perfect, since (rx foo) will not be updated if I change foo-regexp; it's
good enough for my purposes though), so I am mainly bringing this up
because I figure either (a) there's a bug to be reported, or(/and) (b)
I'm about to get schooled on some aspect of ELisp I don't fully grasp.
TBH at this stage (b) sounds more interesting.


At any rate, thanks for your time 🙏


(If that matters: I can reproduce this with 28.2, 29.0.90 from ≈1 month
ago, 30.0.50 from ≈3 weeks ago)