Re: [PATCH v2 3/3] Start opportunistic GC timer at startup

[Stefan Monnier]

> +(defun gc-start-opportunistic ()
> +  "Start an opportunistic GC which may run at some point in the future.
> +
> +This function may perform a single GC at some point while Emacs
> +is idle, with the goal of improving interactive performance by
> +avoiding GC while the user is actively interacting with Emacs.
> +
> +The higher `gc-opportunistic-eager-factor' is, the more likely
> +this function is to actually perform a GC.  Note that increasing
> +this variable can worsen performance by performing excessive GCs.
> +If `gc-opportunistic-eager-factor' is nil, this function will do
> +nothing.
> +
> +If `gc-opportunistic-eager-factor' is non-nil at Emacs startup,
> +this function will be run by an idle timer.  Such a timer can
> +also be started after Emacs startup with `run-with-idle-timer'."
> +  (when gc-opportunistic-eager-factor
> +    (when gc-next-opportunistic-timer
> +      (cancel-timer gc-next-opportunistic-timer))
> +    (setq gc-next-opportunistic-timer
> +          (run-with-idle-timer
> +           gc-estimated-time nil
> +           (lambda ()
> +             (if (garbage-collect-maybe gc-opportunistic-eager-factor)
> +                 (setq gc-opportunistic-performed
> +                       (1+ gc-opportunistic-performed))))
> +           ))))
> +
>  (defun command-line ()
>    "A subroutine of `normal-top-level'.
>  Amongst another things, it parses the command-line arguments."
> @@ -1420,6 +1467,15 @@ please check its value")
>                (eq face-ignored-fonts old-face-ignored-fonts))
>        (clear-face-cache)))
>  
> +  ;; Start opportunistic GC (after loading the init file, so we obey
> +  ;; its settings).  This is desirable for two reason:
> +  ;; - It reduces the number of times we have to GC in the middle of
> +  ;;   an operation.
> +  ;; - It means we GC when the C stack is short, reducing the risk of false
> +  ;;   positives from the conservative stack scanning.
> +  (when gc-opportunistic-eager-factor
> +    (run-with-idle-timer 1 t #'gc-start-opportunistic))

This looks a bit complicated: why use 2 timers?

I currently use the following instead:

    (defvar gc--opportunistic-counter 0)
    
    (defun gc--opportunistic ()
      "Run the GC during idle time."
      ;; This is good for two reason:
      ;; - It reduces the number of times we have to GC in the middle of
      ;;   an operation.
      ;; - It means we GC when the C stack is short, reducing the risk of false
      ;;   positives from the conservative stack scanning.
      (when (garbage-collect-maybe 3)
        (setq gc--opportunistic-counter (1+ gc--opportunistic-counter))
        ;; Recalibrate the timer.
        (cancel-function-timers #'gc--opportunistic)
        (run-with-idle-timer
         ;; FIXME: Magic formula!
         (+ 1 (* 10 (/ gc-elapsed gcs-done))) t #'gc--opportunistic)))
    (run-with-idle-timer 1 t #'gc--opportunistic)

Also notice that I use a different idle time (I think we should impose
a minimum of at the very least 0.2s, I used 1s), and I think
we may want to play with that as well.

When I looked at `gc--opportunistic-performed` I wasn't sure what to
think of it, when comparing it to `gcs-done`: I don't want this to be
too high (because that means we're doing way more GCs) and I don't want
it to be too low either (because it likely means it's ineffective).

Maybe what I want to compare it against is the number of commands
during which the GC is run exactly once?  The idea is to filter out
those commands which are compute-intensive and hence run the GC
internally (for which an opportunistic GC is of no use anyway).

So I added:

    (defvar gc--opportunistic-single-gc-cmds 0)
    (let ((last-gcs nil))
      (add-hook 'pre-command-hook (lambda () (setq last-gcs gcs-done)))
      (add-hook 'post-command-hook
                (lambda ()
                  (if (eq (1- gcs-done) last-gcs)
                      (setq gc--opportunistic-single-gc-cmds
                            (1+ gc--opportunistic-single-gc-cmds))))))

So far, this seems to indicate that it's working:
OT1H `gc--opportunistic-single-gc-cmds` stays much lower than
`gc--opportunistic-counter`, indicating that we seem to be able to avoid
GC in many cases, and OTOH `gc--opportunistic-counter` is much lower
than `gcs-done` indicating that we're not excessively increasing the
number of GCS.


        Stefan