bug#38012: 26.2; Gnus can't handle headers with broken idna

[Niels Möller]

Today, I received an email with some strange headers, including

  Cc: gmp-devel@gmplib.org, niels mller <nisse@lysator.liu.xn--se.-wwb>
  To: torbjrn granlund <tg@gmplib.xn--org.-3hc>

I'm not that familiar with the IDNA specs, but I suspect these headers
are bogus.

I read email in Gnus, and when I try to open this mail by pressing RET
in the *Summary* buffer, I get an error 

  puny-decode-string: Args out of range: "se", 2

The buffer shown to me then starts with 95 lines of email headers with
no fontification or hiding of the uninteresting ones, and actual message
not visible at all without scrolling down.

After enabling debug-on-error, and evaluating the
puny-decode-string-internal defun (to get the interpreted rather than
byte-compiled version), I get the below backtrace.

I get the same error by simply evaluating (puny-decode-string "xn--se").
My understanding is that it's right for puny-decode-string to signal an
error, but Gnus needs to handle that error more gracefully. E.g.,
leaving the broken header as-is, maybe showing soem warning, and, most
importantly, continue to process and beautify the remaining headers and
the body of the message as usual.

Regards,
/Niels

Debugger entered--Lisp error: (args-out-of-range "se" 2)
  aref("se" 2)
  (puny-decode-digit (aref encoded ic))
  (setq digit (puny-decode-digit (aref encoded ic)))
  (progn (setq digit (puny-decode-digit (aref encoded ic))) (setq ic (1+ ic)) 
(setq i (+ i (* digit w))) (setq t1 (cond ((<= k bias) puny-tmin) ((>= k (+ 
bias puny-tmax)) puny-tmax) (t (- k bias)))))
  (progn (progn (setq digit (puny-decode-digit (aref encoded ic))) (setq ic (1+ 
ic)) (setq i (+ i (* digit w))) (setq t1 (cond ((<= k bias) puny-tmin) ((>= k 
(+ bias puny-tmax)) puny-tmax) (t (- k bias))))) (>= digit t1))
  (while (progn (progn (setq digit (puny-decode-digit (aref encoded ic))) (setq 
ic (1+ ic)) (setq i (+ i (* digit w))) (setq t1 (cond ((<= k bias) puny-tmin) 
((>= k (+ bias puny-tmax)) puny-tmax) (t (- k bias))))) (>= digit t1)) (setq w 
(* w (- puny-base t1)) k (+ k puny-base)))
  (progn (while (progn (progn (setq digit (puny-decode-digit (aref encoded 
ic))) (setq ic (1+ ic)) (setq i (+ i (* digit w))) (setq t1 (cond ((<= k bias) 
puny-tmin) ((>= k ...) puny-tmax) (t (- k bias))))) (>= digit t1)) (setq w (* w 
(- puny-base t1)) k (+ k puny-base))) nil)
  (let ((old-i i) (w 1) (k puny-base) digit t1) (progn (while (progn (progn 
(setq digit (puny-decode-digit (aref encoded ic))) (setq ic (1+ ic)) (setq i (+ 
i (* digit w))) (setq t1 (cond (... puny-tmin) (... puny-tmax) (t ...)))) (>= 
digit t1)) (setq w (* w (- puny-base t1)) k (+ k puny-base))) nil) (setq out 
(1+ (buffer-size))) (setq bias (puny-adapt (- i old-i) out (= old-i 0))))
  (while (< ic (length encoded)) (let ((old-i i) (w 1) (k puny-base) digit t1) 
(progn (while (progn (progn (setq digit (puny-decode-digit ...)) (setq ic (1+ 
ic)) (setq i (+ i ...)) (setq t1 (cond ... ... ...))) (>= digit t1)) (setq w (* 
w (- puny-base t1)) k (+ k puny-base))) nil) (setq out (1+ (buffer-size))) 
(setq bias (puny-adapt (- i old-i) out (= old-i 0)))) (setq n (+ n (/ i out)) i 
(mod i out)) (goto-char (point-min)) (forward-char i) (insert (format "%c" n)) 
(setq i (1+ i)))
  (let ((encoded (buffer-substring (if (search-backward "-" nil (quote move)) 
(1+ (point)) (point)) (point-max))) (ic 0) (i 0) (bias puny-initial-bias) (n 
puny-initial-n) out) (delete-region (point) (point-max)) (while (< ic (length 
encoded)) (let ((old-i i) (w 1) (k puny-base) digit t1) (progn (while (progn 
(progn (setq digit ...) (setq ic ...) (setq i ...) (setq t1 ...)) (>= digit 
t1)) (setq w (* w (- puny-base t1)) k (+ k puny-base))) nil) (setq out (1+ 
(buffer-size))) (setq bias (puny-adapt (- i old-i) out (= old-i 0)))) (setq n 
(+ n (/ i out)) i (mod i out)) (goto-char (point-min)) (forward-char i) (insert 
(format "%c" n)) (setq i (1+ i))))
  (progn (insert string) (let ((encoded (buffer-substring (if (search-backward 
"-" nil (quote move)) (1+ (point)) (point)) (point-max))) (ic 0) (i 0) (bias 
puny-initial-bias) (n puny-initial-n) out) (delete-region (point) (point-max)) 
(while (< ic (length encoded)) (let ((old-i i) (w 1) (k puny-base) digit t1) 
(progn (while (progn (progn ... ... ... ...) (>= digit t1)) (setq w (* w ...) k 
(+ k puny-base))) nil) (setq out (1+ (buffer-size))) (setq bias (puny-adapt (- 
i old-i) out (= old-i 0)))) (setq n (+ n (/ i out)) i (mod i out)) (goto-char 
(point-min)) (forward-char i) (insert (format "%c" n)) (setq i (1+ i)))) 
(buffer-string))
  (unwind-protect (progn (insert string) (let ((encoded (buffer-substring (if 
(search-backward "-" nil ...) (1+ ...) (point)) (point-max))) (ic 0) (i 0) 
(bias puny-initial-bias) (n puny-initial-n) out) (delete-region (point) 
(point-max)) (while (< ic (length encoded)) (let ((old-i i) (w 1) (k puny-base) 
digit t1) (progn (while (progn ... ...) (setq w ... k ...)) nil) (setq out (1+ 
(buffer-size))) (setq bias (puny-adapt (- i old-i) out (= old-i 0)))) (setq n 
(+ n (/ i out)) i (mod i out)) (goto-char (point-min)) (forward-char i) (insert 
(format "%c" n)) (setq i (1+ i)))) (buffer-string)) (and (buffer-name 
temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert 
string) (let ((encoded (buffer-substring (if ... ... ...) (point-max))) (ic 0) 
(i 0) (bias puny-initial-bias) (n puny-initial-n) out) (delete-region (point) 
(point-max)) (while (< ic (length encoded)) (let ((old-i i) (w 1) (k puny-base) 
digit t1) (progn (while ... ...) nil) (setq out (1+ ...)) (setq bias 
(puny-adapt ... out ...))) (setq n (+ n (/ i out)) i (mod i out)) (goto-char 
(point-min)) (forward-char i) (insert (format "%c" n)) (setq i (1+ i)))) 
(buffer-string)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer 
(set-buffer temp-buffer) (unwind-protect (progn (insert string) (let ((encoded 
(buffer-substring ... ...)) (ic 0) (i 0) (bias puny-initial-bias) (n 
puny-initial-n) out) (delete-region (point) (point-max)) (while (< ic (length 
encoded)) (let (... ... ... digit t1) (progn ... nil) (setq out ...) (setq bias 
...)) (setq n (+ n ...) i (mod i out)) (goto-char (point-min)) (forward-char i) 
(insert (format "%c" n)) (setq i (1+ i)))) (buffer-string)) (and (buffer-name 
temp-buffer) (kill-buffer temp-buffer)))))
  puny-decode-string-internal("se")
  puny-decode-string("xn--se")
  mapconcat(puny-decode-string ("xn--se" "-wwb") ".")
  puny-decode-domain("xn--se.-wwb")
  article-decode-idna-rhs()
  run-hooks(gnus-article-decode-hook)
  gnus-request-article-this-buffer(5302 "nnml:mail.gmp-devel")
  gnus-article-prepare(5302 nil)
  gnus-summary-display-article(5302 nil)
  gnus-summary-select-article(nil force)
  gnus-summary-show-article(nil)
  funcall-interactively(gnus-summary-show-article nil)
  call-interactively(gnus-summary-show-article nil nil)
  command-execute(gnus-summary-show-article)



In GNU Emacs 26.2 (build 1, amd64-portbld-freebsd11.2, GTK+ Version 3.24.9)
Recent messages:
Fetching headers for nnml:mail.gmp-devel...done
Sorting threads...done
Generating summary...done
No more unseen articles
Mark saved where search started
Auto-saving...done
Auto-saving...done
Quit
Type "q" in help window to delete it.


Configured using:
 'configure --disable-build-details --localstatedir=/var --without-gpm
 --with-x --enable-acl --without-cairo --with-dbus --with-gconf
 --with-gif --with-gnutls --with-gsettings --with-x-toolkit=gtk3
 --with-jpeg --with-json --with-file-notification=kqueue --with-lcms2
 --with-m17n-flt --with-imagemagick --with-mailutils --with-modules
 --with-sound=oss --with-libotf --with-png --with-toolkit-scroll-bars
 --with-rsvg --with-threads --with-tiff --with-xft --with-xim
 --with-xml2 --with-xpm --without-xwidgets --x-libraries=/usr/local/lib
 --x-includes=/usr/local/include --prefix=/usr/local
 --mandir=/usr/local/man --disable-silent-rules
 --infodir=/usr/local/share/emacs/info/
 --build=amd64-portbld-freebsd11.2 'CFLAGS=-O2 -pipe
 -fstack-protector-strong -isystem /usr/local/include
 -fno-strict-aliasing ' 'CPPFLAGS=-isystem /usr/local/include' 'LDFLAGS=
 -fstack-protector-strong -L/usr/local/lib ''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GCONF GSETTINGS GLIB
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Summary

Minor modes in effect:
  gnus-agent-summary-mode: t
  gnus-mailing-list-mode: t
  type-break-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow bbdb-message nnir emacsbug mm-archive map eieio-opt speedbar
sb-image ezimage dframe misearch multi-isearch jka-compr find-func
cl-print debug cus-edit cus-start cus-load thingatpt help-fns radix-tree
shr svg xml dom browse-url sort ansi-color gnus-cite mail-extr
gnus-async gnus-bcklg qp cl-extra help-mode gnus-ml disp-table
gnus-topic cursor-sensor nndraft nnmh bbdb-gnus bbdb-mua bbdb-com
nnagent nnml network-stream nsm starttls gnus-agent gnus-srvr gnus-score
score-mode nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime
smime dig mailcap nntp gnus-cache gnus-sum gnus-group gnus-undo
gnus-start gnus-cloud nnimap nnmail mail-source tls gnutls utf7 netrc
nnoo parse-time gnus-spec gnus-int gnus-range message sendmail rmc puny
dired dired-loaddefs format-spec rfc822 mml mml-sec epa derived epg
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader gnus-win gnus nnheader gnus-util rmail rmail-loaddefs rfc2047
rfc2045 ietf-drums mail-utils mm-util mail-prsvr wid-edit term/screen
term/xterm xterm time-date elec-pair bookmark pp recurse cl bbdb
timezone type-break info finder-inf bbdb-autoloads package easymenu
epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind kqueue lcms2 dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 288125 36650)
 (symbols 48 84381 1)
 (miscs 40 189 601)
 (strings 32 94330 10295)
 (string-bytes 1 2753763)
 (vectors 16 27725)
 (vector-slots 8 686345 18342)
 (floats 8 249 707)
 (intervals 56 4052 0)
 (buffers 992 29))

-- 
Niels Möller. PGP-encrypted email is preferred. Keyid 368C6677.
Internet email is subject to wholesale government surveillance.