Re: [MIT-Scheme-devel] Why does URL:ENCODE-STRING not encode & and ; etc.?

[Taylor R Campbell]

   Date: Wed, 02 May 2012 13:21:32 +0200
   From: <address@hidden>

   I may misunderstand things here, but isn't URL:ENCODE-STRING meant to
   propertly encode characters that are special to urls? Examples would be
   =, & and ; (which should all be encoded). However, 

   > (url:encode-string "a=b&b=c;c=d") 

   just echoes that string literally ("a=b&b=c;c=d") without any encoding.
   This is by design, as these characters are explicitly added to
   URL:CHAR-SET:UNESCAPED in runtime/url.scm. 

`URL encoding' is a pile of lossage because the concept of a URI was
designed for Perl-type regexp-scraping garbage rather than as an
encoded data container, which is how the runtime's URL abstraction
(incorrectly) treats the concept.

The characters `=', `&', and `;' have no special interpretation in any
of the generic URI syntax.  URL:ENCODE-STRING chooses some random set
of characters to encode for hysterical raisins (and is not used by
anything else in the runtime's URL abstraction), but every different
application needs to worry about different rules for encoding because
they don't compose.

For example, if I have an alist (("foo" "x=y") ("bar" "p&q")), and I
want to map it into an application/x-www-form-urlencoded string, I get
`foo=x%3dy&bar=p%26q'.  But if I want to put that into the query
string of a URI, it has to be `http://x.com/?foo=x%3dy&bar=p%26q' --
note that the percent signs did *not* get encoded here.  And it's
incorrect to decompose the URI by percent-decoding before decoding the
application/x-www-form-urlencoded text -- if we did that, we'd get out
`foo=x=y&bar=p&q', which is a bad application/x-www-form-urlencoded
string.

If you send `http://x.com/?foo=x%253dy&bar=p%2526q' -- the obvious
sensible thing to do -- to any existing agent, designed bug-for-bug
compatibly with brain-damaged Perl garbage, it'll get confused and the
alist that comes out will be (("foo" "x%3dy") ("bar" "p%26q")), which
is wrong.