bug#22667: string->bytevector encoding error

[Mark H Weaver]

Josep Portella Florit <address@hidden> writes:

> 'string->bytevector' throws 'encoding-error' when the encoding is
> "UTF-16" and the length of the string is greater than 128.  The same
> for the encoding "UTF-32" and a string of length greater than 64.
>
> Tested on Guile 2.0.11 and 2.1.2 with the same result.
>
> How to reproduce:
>
> (use-modules (ice-9 iconv))
>
> ;; These expressions evaluate as expected:
> (string->bytevector (make-string 128 #\A) "UTF-16")
> ;; => #vu8(254 255 0 65 0 65 ...)
> (string->bytevector (make-string 64 #\A) "UTF-32")
> ;; => #vu8(0 0 254 255 0 0 0 65 0 0 0 65 ...)
>
> ;; These expressions throw 'encoding-error':
> (string->bytevector (make-string 129 #\A) "UTF-16")
> (string->bytevector (make-string 65 #\A) "UTF-32")

Thee bug is in 'display_string_using_iconv', introduced in commit
f4bc4e5934, which improperly assumes that a 256-byte buffer
(encoded_output) will be large enough to hold the results of converting
up to 256 bytes of UTF-8 (utf8_buf) to an arbitrary encoding using
'iconv'.  When this assumption fails, the call to 'iconv' on line 997

      done = iconv (id->output_cd, &input, &input_left,
                    &output, &output_left);

reports a failure due to the output buffer being too small, and the code
fails to handle this case.

To be continued...

     Mark