Re: Using -dshow-available-fonts with LilyPond v2.25.12 throws error with non–UTF-8 encoded font info

[Nate Whetsell]

Thanks for making the merge request. Unfortunately, it didn’t fix the issue, 
although it really seems like it should have. I’m starting to think this may be 
a bug in Guile.

With the patch applied, running `lilypond -dshow-available-fonts` outputs:

GNU LilyPond 2.25.12 (running Guile 3.0)
ERROR: In procedure ly:font-config-display-fonts:
Throw to key `decoding-error' with args `("scm_from_stringn" "input locale 
conversion error" 92 #vu8([very long bytevector])

The “very long bytevector” is the bytes of the C string that’s being converted. 
In this C string, there appears to be a single encoding issue related to a 
macOS system font that causes conversion to fail. Attached is a much reduced 
(though still long) LilyPond file that demonstrates an issue (but maybe not the 
exact same issue). In the LilyPond file, I’m converting a bytevector to a 
string using bytevector->string 
<https://www.gnu.org/software/guile/manual/html_node/Representing-Strings-as-Bytes.html#index-bytevector_002d_003estring>
 (which is documented 
<https://www.gnu.org/software/guile/manual/html_node/Conversion-to_002ffrom-C.html#index-scm_005ffrom_005fstringn>
 as the Scheme interface to scm_from_stringn).

When I use a conversion strategy of 'substitute, the conversion works as 
expected (with a � where the encoding issue occurs). When I use the default 
strategy ('error, presumably), LilyPond outputs:

string-test.ly:3:2: error: Guile signaled an error for the expression beginning 
here
#
 (begin
Throw to key `decoding-error' with args `("scm_from_utf8_stringn" "input locale 
conversion error" 2 #vu8([long bytevector])

For what it’s worth, I tried—

SCM string_port = scm_open_output_string ();
scm_set_port_encoding_x (string_port, scm_from_utf8_string ("UTF-8"));
scm_set_port_conversion_strategy_x (string_port, scm_from_utf8_symbol 
("substitute"));
SCM scheme_string = scm_from_port_stringn (str.c_str (), str.length (), 
string_port);
scm_close_port (string_port);
scm_write_line (scheme_string, port);

—only to realize that scm_from_port_stringn 
<https://codeberg.org/rlb/guile/src/branch/main/libguile/strings.c#L1726> is a 
convenience function that calls scm_from_stringn (so it results in the same 
error). (This is based on the bytevector->string implementation 
<https://codeberg.org/rlb/guile/src/branch/main/module/ice-9/iconv.scm#L63>, 
which uses read-string 
<https://codeberg.org/rlb/guile/src/branch/main/module/ice-9/rdelim.scm#L152> 
instead of something like scm_from_port_stringn.)

It seems like there’s something weird happening with the conversion strategy 
passed to scm_from_stringn. I’m wondering if a conversion strategy from an 
input port is being used instead of the conversion strategy passed to 
scm_from_stringn.

Cheers,
Nate



> On Jan 9, 2024, at 2:49 AM, Werner LEMBERG <wl@gnu.org> wrote:
> 
> 
>> On macOS Ventura v13.6.3 (and almost certainly other versions), the
>> string produced when using -dshow-available-fonts need not be
>> encoded as UTF-8.
> 
> Thanks, should be fixed with this MR:
> 
>  https://gitlab.com/lilypond/lilypond/-/merge_requests/2231
> 
> Out of interest: Which font shows this behaviour?
> 
> I suggest you contact the FontConfig people and report the issue, too;
> assuming that the font is valid it should be possible to convert the
> non-UTF8 data to UTF8.
> 
> 
>    Werner