Re: Decoding URLs input

[Jean Louis]

* Yuri Khan <yuri.v.khan@gmail.com> [2021-07-03 23:17]:
> I just fed it some percent-encoded sequences that I knew would result
> in invalid UTF-8 when decoded. If it were doing a full decode, I
> expected it to signal an error. It didn’t.
> 
> > The docstring is poor, it says like: "Remove %XX embedded spaces, etc in a
> > URL." -- with "remove" I don't expect converting UTF-8 into bytes.

> Yeah, that is bad. If I see “remove %xx” in a docstring, I expect
> (string= (f "Hello%20World") "HelloWorld").

Yes. Can you correct that docstring maybe?

> > I am now solving the issue that spaces are converted to plus sign and
> > that I have to convert + signs maybe before:
> > (decode-coding-string (url-unhex-string "Hello+There") 'utf-8)
> > but maybe not before, maybe I leave it and convert later.
> 
> You have to replace them before percent-decoding. If you try it after
> percent-decoding, you will not be able to distinguish a + that encodes
> a space from a + that you just decoded from %2B. Luckily, spaces never
> occur in a valid encoded query string; if they did and had some
> meaning, you’d have to decode + *at the same time* as %xx.

Exactly. I just did not yet get into that analysis and thanks for your
quick one! Now it is clear I have to do it.

> By the way, you’re in for some unspecified amount of pain by trying to
> implement a web application without a framework. (And by a framework I
> mean a library that would give you well-tested means to encode/decode
> URL parts, HTTP headers, gzipped request/response bodies, base64,
> quoted-printable, application/x-www-form-urlencoded,
> multipart/form-data, json, …) CGI is not nearly as simple as it
> initially appears to be when you read a hello-cgi-world tutorial.

Definitely not as simple. Though for the specific need it may be very
compact. There also exist simple Emacs CGI libraries though nearly not
as comprehensive as you mentioned it.

The Double Opt-In already works with cosmetic errors. It will soon be
perfected with these information.

Double Opt-In is to receive subscription, redirect to the subscription
confirmation page that in turn could redirect to sales page or be
sales page or other page; to send email to subscriber to confirm, to
receive confirmation and dispatch the Emacs hash to administrator; to
receive unsubscribe request without any hesitations and dispatch to
administrator; offer to visitor to subscribe again.

I am designing that to be offline just as I have been doing it long
time before, for years. Database is not online. No people's data
should be ever released online. This is for business secret
purposes. One can see that databases leak all the time on
raidforums.com

And practically it works well, it generates relations. I consider it
one of most important scripts.

The old Perl Form script type I have long converted to Common Lisp.

For my specific need:

- encode/decode URL parts is resolved as I only receive URL and
  dispatch simpler confirmation URLs;

- HTTP headers, I just use these and nothing more so far:

(defun rcd-cgi-headers (&optional content-type)
  "Prints basic HTTP headers for HTML"
  (let ((content-type (or content-type "text/html")))
    (princ (format "Content-type: %s\n\n" content-type))))

(defun rcd-cgi-redirect (url)
  "Redirect to URL."
  (princ (concat "Location: " url "\n\n"))
  (unless (eq major-mode 'emacs-lisp-mode)
    (kill-emacs 0)))

- gzipped request/response bodies -- hmm, I think I will not get
  gzipped request, I have no idea right now. Responses definitely not,
  as the only response is either error or redirect to a page. I will
  keep redirect pages in the URL itself to have the script totally on
  its own without anything much hard coded inside.

- I would like to have a line encrypted request. I have used Tiny
  Encryption in Perl and it worked well.

  Do you know any single line encryption for emacs?
  Maybe I can use OpenSSL. I need a stream cipher.
  https://en.wikipedia.org/wiki/Tiny_Encryption_Algorithm#Versions

  I don't know how to use this one, but that maybe what I could use:
  https://github.com/skeeto/emacs-chacha20

  as for URL subscribe requests it is best to have just one string
  encrypted, like doi.cgi?njadsjnasnfdkjsbfbsfbhj

  that nothing is shown to user. That is how I have been doing it
  before with Perl.

  I may simply use external program and encrypt URL requests similar
  like:

  echo '(mid 1 eid 2 cid 3 tile "Subscribe to business")' | openssl enc 
-ChaCha20 -e -k some -pbkdf2 -base64 -A
  
U2FsdGVkX192ic8hOU15mR6zjoYK/rpRA/NkgHohy6eO2A+W8EHuopAigBcc57wKR/sxMYqPV1ESYEY523DS/h0=
  as the idea is to keep the script free of hard coding. It should
  only authorize email addresses that server is in charge of.

  Maybe I can do that with gnutls- functions? I just don't know
  how. That would be better as not to have external dependencies.
  
- base64 functions exist in Emacs already? Any problem? I will not
  need it.

- quoted-printable - they also exist in Emacs.

- application/x-www-form-urlencoded -- yes, definitely, up

- multipart/form-data, json -- json functions are there in Emacs,
  though I think I will not need it.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/