bug#34404: 26.1; Finalizer in hash table run early?

[Braun Gábor]

Hi,

The following test is reported as failed in the *ert* buffer
but I think it should pass.

------------- File test.el ------------------
;; -*- lexical-binding: t; -*-
(require 'ert)

(ert-deftest finalizer-in-weak-hash-table ()
  "Test finalizers in weak hash tables"
  (skip-unless (fboundp 'make-finalizer))
  (let ((finalizer-run nil))
    (let ((table (make-hash-table :weakness 'key)))
      (let ((key (make-symbol "key")))
        ;; Without wrapper (ignore ...) the test passes, why?
        (ignore
         (puthash
          key
          (make-finalizer (lambda () (setq finalizer-run t)))
          table))
        (garbage-collect)
        ;; With just one (garbage-collect), the test passes.
        (garbage-collect)
        (message "Finalizer: %s" (gethash key table))
        (should-not finalizer-run)
        ;; Prevent optimizing out early.
        (message "Finalizer: %s" (gethash key table)))
      (garbage-collect)
      ;;(garbage-collect)
      (should finalizer-run))))
----------------------------------------------------------------

Start emacs with the command line:

emacs -Q -l test.el --eval '(ert t)'

In buffer *ert* press "m" with the cursor on the single test.
The following buffers are shown:

-------------- buffer *ert* ------------------------------------
Selector: t
Passed:  0
Failed:  1 (1 unexpected)
Skipped: 0
Total:   1/1

Started at:   2019-02-09 13:53:14+0100
Finished.
Finished at:  2019-02-09 13:53:15+0100

F

F finalizer-in-weak-hash-table
    Test finalizers in weak hash tables
    (ert-test-failed
     ((should-not finalizer-run)
      :form finalizer-run :value t))


----------------------------------------------------------------


------------------ buffer *ERT Messages* -----------------------
Messages for test ‘finalizer-in-weak-hash-table’:
Finalizer: #<finalizer used>
----------------------------------------------------------------

Discussion:

The finalizer since its creation is always reachable in the inner-most 
(let...)
via (gethash key table), so it shouldn't be run until the last
(message...) there, which accesses the finalizer this way.
However, both the message made in buffer *ERT Messages* as well as the 
failed
form in buffer *ert* indicates that the finalizer has been run; that is
the only way the local variable finalizer-run can have a value t.

(The (message...) after the failed test (should-not...) is there to rule
out the destruction of local variables key and table early as a form of
optimization, which would justify running the finalizer before the 
test.)

As some of the comments in the test indicate,
small changes cause the test to pass:
like omitting the wrapper (ignore...) around (puthash...)
or just one (garbage-collection) before (should-not...) causes the test
to pass.

System information:

In GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.2)
 of 2018-12-26, modified by Debian built on x86-ubc-01
Windowing system distributor 'The X.Org Foundation', version 
11.0.12003000
System Description:     Debian GNU/Linux buster/sid

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Finalizer: #<finalizer used>
Ran 1 tests, 0 results were as expected, 1 unexpected
Making completion list...

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-
lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/
usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-
lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/
usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-3ThesY/emacs-26.1+1=. -fstack-
protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: hu_HU.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ERT-View

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt bytecomp
byte-compile cconv dired dired-loaddefs format-spec rfc822 mml mml-sec
password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils cl-seq thingatpt cl-extra help-mode cl-macs gv ert pp
find-func ewoc easymenu debug cl-loaddefs cl-lib elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 107828 8705)
 (symbols 48 21253 1)
 (miscs 40 63 154)
 (strings 32 30662 1206)
 (string-bytes 1 809252)
 (vectors 16 16338)
 (vector-slots 8 511222 11260)
 (floats 8 60 178)
 (intervals 56 366 8)
 (buffers 992 14))


Best wishes,

     Gábor Braun