Re: OSC8 hyperlink support in ncurses

[Thomas Dickey]

On Fri, Aug 25, 2023 at 03:25:10AM -0500, G. Branden Robinson wrote:
> Hi Thomas,
> 
> At 2023-08-23T15:25:24-0400, Thomas Dickey wrote:
> > On Wed, Aug 23, 2023 at 03:21:43AM -0500, G. Branden Robinson wrote:
> > > There was a humdinger of an argument about this on Egmont Koblinger's
> > > Gist about this feature.
> > > 
> > > https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
> [...]
> > But this is the developer responsible for the feature that caused
> > gnome-terminal to run out of memory while I ran an ncurses
> > test-program.
> > 
> > See the end of this section:
> > 
> > https://invisible-island.net/ncurses/ncurses-slang.html#compare_picsmap
> 
> Ah, ha.  Yes, you've had that story up for a long time--I remember
> reading it, but having no idea of the principals behind
> the...entertainment.
> 
> > (coincidentally, he fixed _that_ bug two weeks after I wrote that
> > section)
> 
> We take bug reports as we find them.  :P

In this case, it was coincidence. Though my page was being reviewed,
and it was visible on the Internet, it wasn't advertised -- it was
one of several things that I did in developing ncurses 6.1: making
test-programs for extended colors, validating them on various terminals,
etc.  Actually I'd noticed the problem in August, and with one distraction
and another documented the problem in my page on November 3, for the
"dots" program.

I noticed the (coincidental report) in my routine snapshot of
https://gist.github.com/XVilka/8346728

I have a few hundred Git clones, and pull those about once a week,
but the gist for this wouldn't come via wget (and wasn't in Git).
So I snapshotted the page from Firefox and kept my own Git repo for it,
until its owner trashed it.  (The suggested replacement site has similar
problems, as you might have noticed).

As I recall it, when I began this one, the Internet Archive had trouble
with it too, but see in responding that it has a few snapshots (not as
many as I).  So I can offer you a URL:

https://web.archive.org/web/20171119155902/https://gist.github.com/XVilka/8346728

which contains this comment:

        <p>Up to today <code>VTE</code> contained a significant memory
        leak if the colon form was used (<a
        href="https://github.com/neovim/neovim/issues/7573";>1</a>, <a
        href="https://bugzilla.gnome.org/show_bug.cgi?id=790539";
        rel="nofollow">2</a>, <a
        href="https://git.gnome.org/browse/vte/commit/?id=dda73cc";
        rel="nofollow">3</a>).  Embarrassing.</p>

Searching on bugzilla.gnome.org no longer works, but at the moment I
am (still) able to visit its URLs, and match that up with the Git:

        commit dda73cc07250ea324b4227907197c39b93fcd365
        Author: Christian Persch <chpe@src.gnome.org>
        Date:   Sat Nov 18 18:40:03 2017 +0100

            matcher: Fix memory leak

            Don't leak the GValueArray.

            https://bugzilla.gnome.org/show_bug.cgi?id=790539
 
Seeing the comment about GValueArray, I recalled reading a discussion between
Egmont and Christian which said that one had noticed a memory leak, but that
it wasn't something that they need not fix at that time.  At that time,
I took a look for it, but didn't find that.  Since searches are no longer
working, it would be a lot harder (for me) to get that information.

The neovim report explains the problem, which lets us find the change
which produced it:

        commit b8a021330c0375ef6411f9abf2e4f717d0a5fb1d
        Author: Egmont Koblinger <egmont@gmail.com>
        Date:   Mon Jan 13 17:29:26 2014 +0100

            Allow ":" as subparameter delimiter in SGR color sequences.

            https://bugzilla.gnome.org/show_bug.cgi?id=685759

So that was in the code for more than three and a half years.

OSC8 is more complicated to implement, will rely upon being developed
by _several_ developers, etc.

(I take it for granted that my audience knows how to use the hash to
find the relevant commits).

> > > I offer that link mainly for the edification of bystanders; I trust
> > > you've already read and considered that material.
> > 
> > yes, I've had a few years to consider it.  Now that it's becoming more
> > prevalent, others are thinking about it as well.  It would be nice to
> > have the full presentation for this:
> > 
> > https://www.theregister.com/2023/08/09/ansi_escape_sequence_risks/
> 
> Yes.  This is the sort of thing Ingo was alluding to.  I went off on my
> own rant about it on the groff list.
> 
> https://lists.gnu.org/archive/html/groff/2023-08/msg00106.html

I like mailing list archives because they're likely to have related mails
from people which I didn't collect, though (in a recent work) I've noticed
that they're not always complete, omitting some message that I'd like to
quote.
 
> If OSC 8 fails--even if hyperlinks in terminal windows are discarded as
> a pointless fad--I will still be glad I added the `MR` macro to groff
> for the three other reasons I did so.
> 
> (groff) NEWS:
>   Inclusion of the `MR` macro was prompted by its introduction to
>   Plan 9 from User Space's troff in August 2020.  Its purpose is to
>   ameliorate several long-standing problems with man page cross
>   references: (1) the package's lack of inherent hyperlink support for
>   them; (2) false-positive identification of strings resembling man page
>   cross references (as can happen with "exit(1)", "while(1)",
>   "sleep(5)", "time(0)" and others) by terminal emulators and other
>   programs; (3) the unwanted intrusion of hyphens into man page topics,
>   which frustrates copy-and-paste operations (this problem has always
>   been avoidable through use of the \% escape sequence, but cross
>   references are frequent in man pages and some page authors are
>   inexpert *roff users); and (4) deep divisions in man page maintenance
>   communities over which typeface should be used to set the man page
>   topic (italics, roman, or bold).
> 
> > > In the meantime it would be helpful if you could add a terminfo
> > > capability so that applications using terminfo but not curses per se
> > > can pay their money and take their chances, as with groff's
> > > grotty(1), for which Lennart Jablonka is preparing patches to make
> > > the program a terminfo application (at long last, one might say).
> > 
> > Nicolas has a feature for this which his users can configure :-)
> 
> I take it you mean that the `Hls` capability that tmux recognizes, we
> could similarly handle via the user_caps(5) approach?

yes - it's called user_caps because users can use the feature

ncurses uses the feature too, of course.
 
> > > We'd like to be able to ask terminfo if the terminal description
> > > supports OSC 8, but we can't.
> > 
> > Actually, that's mostly due to Egmont's interference,
> > as you're probably aware.
> 
> I actually am not.  But I am always interested to hear stories that can
> inform my expectations of who will be fruitful to work with--or won't.

yes... but Egmont's not the only nuisance (and it takes work to document
these things).  Although I have lots of notes, he's not been mentioned on
any of the pages I've written this year.
 
> That goes for plagiarists, too.  I appreciate your pages documenting
> your encounters with them.

:-)

-- 
Thomas E. Dickey <dickey@invisible-island.net>
https://invisible-island.net

signature.asc
Description: PGP signature