Re: [Chicken-hackers] CR #1142 and upcoming changes

[Felix Winkelmann]

> To avoid doing this again soon, I think the other change you suggested
> should definitely be included: the reworking of internal libraries by
> splitting them up.  Perhaps you already assumed this would be included,
> I don't think I have seen this mentioned yet so I wanted to put it out
> there.

Yes, that was implied.
> 
> No problem @ bignums.  I don't know much about unicode, so that should
> probably be looked at by someone else.  A simple thing we could include
> would be to reject all strings that have invalid UTF-8 encoding, like
> Postgres does that.  I always really appreciated this feature of Postgres:
> it ensures that you don't get invalid data in your system, and prevents
> pollution and getting in "character set hell", like happens all the time
> in MySQL: 
> https://www.bluebox.net/insight/blog-article/getting-out-of-mysql-character-set-hell
> 
> Adding rejection of non-UTF-8 strings would make the transition to a
> "full" Unicode system less painful (and perhaps make it possible to do
> it in a non-breaking way).  I'm not sure how difficult this would be,
> though: all string mutation procedures should have a check that they
> won't create invalid strings by setting characters (bytes).

Sounds like a good first step, even though I personally would prefer
UCS-4 strings (constant lookup + modification and so on). But that
seems to be unpopular, AFAICT...


felix