Re: [VM] bug in mime encoding of cached data

[Julian Bradfield]

On 2014-10-27, Julian Bradfield <address@hidden> wrote:
[ story of non-ascii data in cached summaries being corrupted ]

> I was still loading somewhere an ancient attempt to fix up one of the
> old bugs.
>
> Forget I spoke!
>
>

I spoke too soon. Even after clearing out my old failed attempts to
fix the problem, I still have a problem.

However, now I know why.

It's a FSFmacs/XEmacs difference.

In XEmacs, #'princ, and hence #'format, do what they say do, namely
output the printed representation to an output stream. Hence, text
properties on strings are lost.

In FSFmacs, #'princ builds a string representation (complete with
inherited properties), and then prints it to a stream only if the
output stream is an i/o stream; otherwise it inserts the strings into
the output buffer/string.

Thus the summary line, which is ultimately built by calling #'format
on various mime-decoded strings, doesn't contain the text properties
necessary for mime-reencoding to work.

I have applied the following patch to my own copy of vm, which
appears, so far, to deal with the issue.

*** vm-summary.el       2014/11/16 11:47:50     1.4
--- vm-summary.el       2014/11/23 20:16:36     1.5
***************
*** 954,965 ****
                         (setq token ''mark)
                       (setq sexp (cons (list 'vm-su-mark
                                              'vm-su-message) sexp)))))
!             (cond ((and (not token) vm-display-using-mime)
!                    ;; strings might have been already mime-decoded,
!                    ;; but there is no harm in doing it again. USR, 2010-05-13
!                    (setcar sexp
!                            (list 'vm-decode-mime-encoded-words-in-string
!                                  (car sexp)))))
              (cond ((and (not token) (match-beginning 1) (match-beginning 2))
                     (setcar sexp
                             (list
--- 954,967 ----
                         (setq token ''mark)
                       (setq sexp (cons (list 'vm-su-mark
                                              'vm-su-message) sexp)))))
!             ;; we're going to encode them, so don't decode them.
!             ;; JCB 2014-11-23
! ;;          (cond ((and (not token) vm-display-using-mime)
! ;;                 ;; strings might have been already mime-decoded,
! ;;                 ;; but there is no harm in doing it again. USR, 2010-05-13
! ;;                 (setcar sexp
! ;;                         (list 'vm-decode-mime-encoded-words-in-string
! ;;                               (car sexp)))))
              (cond ((and (not token) (match-beginning 1) (match-beginning 2))
                     (setcar sexp
                             (list
***************
*** 990,999 ****
                                               (match-beginning 4)
                                               (match-end 4)))))))
              ;; Why do we reencode decoded strings?  USR, 2010-05-12
! ;;          (cond ((and (not token) vm-display-using-mime)
! ;;                 (setcar sexp
! ;;                         (list 'vm-reencode-mime-encoded-words-in-string
! ;;                               (car sexp)))))
              (setq sexp-fmt
                    (cons (if token "" "%s")
                          (cons (substring format
--- 992,1003 ----
                                               (match-beginning 4)
                                               (match-end 4)))))))
              ;; Why do we reencode decoded strings?  USR, 2010-05-12
!             ;; Because XEmacs #'format doesn't preserve text properties
!             ;; JCB 2014-11-23
!             (cond ((and (not token) vm-display-using-mime)
!                    (setcar sexp
!                            (list 'vm-reencode-mime-encoded-words-in-string
!                                  (car sexp)))))
              (setq sexp-fmt
                    (cons (if token "" "%s")
                          (cons (substring format