[Octave-bug-tracker] [bug #57107] regexp functions fail on ISO-8859 input

[Markus Mützel]

Update of bug #57107 (project octave):

                  Status:                    None => In Progress            
        Operating System:               GNU/Linux => Any                    

    _______________________________________________________

Follow-up Comment #17:

The attached patch is a lot cleaner validating UTF-8 encoded strings.
It still isn't wired in to do anything meaningful though. But it should be
safe enough for testing.

The question remains where we want this conversion from invalid UTF-8 to valid
UTF-8 to happen?
It might be surprising to a user if the string they read from a file wasn't
byte identical to the content of the file.
At the same time, there are probably a lot of places inside Octave (not only
regexp*) where we would need to check if char arrays contained valid UTF-8
before using them safely.
On the other hand, there is nothing that would prevent a user from creating
invalid UTF-8 manually (e.g. assigning "a=char(181)"). So validating strings
that are read from a file wouldn't suffice anyway.

Thus, the best option I see at the moment is identifying the critical places
(e.g. just before passing the strings to PCRE) and validate them using the new
function "validate_u8".

(file #47746)
    _______________________________________________________

Additional Item Attachment:

File name: bug57107_validate_u8.patch     Size:3 KB
    <https://savannah.gnu.org/file/bug57107_validate_u8.patch?file_id=47746>



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?57107>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/