Re: Why does using aset sometimes output raw bytes?

[Stephen Berman]

On Sun, 09 Dec 2018 19:47:03 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: help-gnu-emacs@gnu.org
>> Date: Sun, 09 Dec 2018 18:32:26 +0100
>> 
>> >> why are raw bytes inserted only with some
>> >> multibyte strings (e.g. with "äöüß" but not with "ſðđŋ")?
>> >
>> > Because ſ doesn't fit in a single byte, so when you insert it, the
>> > entire string is made multibyte, and then the other characters are
>> > inserted into a multibyte string.
>> 
>> This seems to imply that ä, ö, ü and ß do fit in a single byte?  Yet
>> (multibyte-string-p "äöüß") returns t.  So I still don't understand.
>
> Look at the codepoints: the above are all less than FF hex, so they
> can fit in a single byte.  By contrast, ſ is 17F hex, more than a
> single byte can hold.  So inserting ſ into a unibyte string _must_
> first make that string multibyte, whereas inserting ä etc. can leave
> it unibyte.
>
> Why (multibyte-string-p "äöüß") returns t is an unrelated issue: it
> has to do with how the Lisp reader reads the string.  The result is a
> multibyte string, where ä is represented by its UTF-8 sequence and not
> by its single-byte codepoint E4 hex.  If you want a unibyte string
> with these bytes, use (multibyte-string-p "\344\366\374\337") instead.

Thanks for the very clear and enlightening explanations; I feel I
understand this better now.

>> >> is there some way in Lisp to say "treat the value of s0 as multibyte
>> >> (regardless of what characters it contains)"?
>> >
>> > Not that I know of, no.  And I don't really understand how could such
>> > a thing exist: how do you "treat as multibyte" an arbitrary byte that
>> > is beyond 127 decimal?
>> 
>> Actually, for the code I was experimenting with, it seems to suffice to
>> use (make-string len 128) as the input to aset (before, I had used
>> (make-string len 32), which led to raw bytes being displayed).
>
> Not sure I understand what you mean by "suffice".  Feel free to ask
> questions if there are some left.

I was experimenting with aset to make random permutations of a string
and didn't understand why there were sometimes raw bytes in the result
(which also led to args-out-of-range errors), but using (make-string len
128) as the container for the permutations prevents that.  And with your
above explanations I now think I understand why.

Steve Berman