bug#70423: 29.3; cc-mode hang at 100% cpu and consuming all available memory

[pillowtrucker]

Greetings!

I have been able to replicate a (relatively) minimum environment for this — 
hyperbole + clang-format.el.
I think lsp-mode aggravates the issue somehow, but is not necessary to enter 
the infinite loop. Without lsp-mode, however, the memory usage seems flat, as 
opposed to ballooning when it is present.
I would also guess that if you are able to plug in that regexp into cc-mode 
without installing all of hyperbole.el, you would probably get the same effect, 
but I didn't know how to do that. 


You will need gzip and the clang-format utility for this from your linux 
distribution or compiled from source. Preferably version 18.1.3 of the llvm 
suite, but it might work with earlier versions. I don't think /how/ it's 
formatting the file matters as much as the fact of it being altered to some 
degree.

Please follow these steps:
1. start with an empty directory
$ export MYDEBUGDIR=emacs-debugging
2. use the attached init.el file as follows:
$ emacs-29.3 --init-directory=${MYDEBUGDIR} -l init.el
3. Wait for the script to reach the point where it's in an infinite loop (check 
top/htop for emacs with 100% usage, maybe wait a few seconds or minutes to make 
sure it is not just taking a long time to run the formatting) and press C-g to 
send "quit".

This results in a hang and the debugger should show something like this:

Debugger entered--Lisp error: (quit)
  #f(compiled-function () #<bytecode -0x7e8f9110f9b66d2>)()
  syntax-ppss()
  beginning-of-defun-raw(nil)
  beginning-of-defun()
  c-get-fallback-scan-pos(18908)
  c-parse-state-get-strategy(18908 1)
  c-parse-state-1()
  c-parse-state()
  c-laomib-invalidate-cache(18908 18910)
  c-before-change(18908 18910)
  clang-format--replace(18908 2 "\n")
  apply(clang-format--replace (18908 2 "\n"))
  clang-format-region(1 18910 nil nil)
  clang-format-buffer()
  load-with-code-conversion("/home/wrath/emacs-debugging2/init.el" 
"/home/wrath/emacs-debugging2/init.el" nil t)
  command-line-1(("-l" "emacs-debugging2/init.el"))
  command-line()
  normal-top-level()

or maybe you will see:

Debugger entered--Lisp error: (quit)
  re-search-backward("^\\s(\\|\\(?:^[ 
\11]*\\(template\\s-*<[^>;.{}]+>\\s-*\\)?\\(..." nil move 1)
  beginning-of-defun-raw(nil)
  beginning-of-defun()
  c-get-fallback-scan-pos(18908)
  c-parse-state-get-strategy(18908 1)
  c-parse-state-1()
  c-parse-state()
  c-laomib-invalidate-cache(18908 18910)
  c-before-change(18908 18910)
  clang-format--replace(18908 2 "\n")
  apply(clang-format--replace (18908 2 "\n"))
  clang-format-region(1 18910 nil nil)
  clang-format-buffer()
  load-with-code-conversion("/home/wrath/emacs-debugging2/init.el" 
"/home/wrath/emacs-debugging2/init.el" nil t)
  command-line-1(("-l" "emacs-debugging2/init.el"))
  command-line()
  normal-top-level()

Either way it does seem like the problem lies somewhere in cc-mode choking 
either on the file itself or on that regular expression.

I license the attached init.el under the terms of the AGPL-3 or later version. 
The drm_formats.cpp file is part of the MIR Server project and is licensed 
under the terms of the GPL-v3. Both licenses attached. I'm not sure if I am 
allowed to re-license the MIR file as AGPL, but I believe they are in any case 
compatible to be distributed together.

Thank you for having a look at this issue, it has been plaguing me since 
november and even making rust look like an ever so slightly more pleasant 
alternative to dealing with constant hangs/crashes and having to restore 
frame/window layouts when editing C/C++..

Kind regards,
M.

On Tuesday, April 16th, 2024 at 7:59 PM, Alan Mackenzie <acm@muc.de> wrote:

> Hello.
> 

> Thanks for taking the trouble to submit this bug report.
> 

> On Tue, Apr 16, 2024 at 13:36:16 +0000, 1stmil.eth wrote:
> 

> > Among other scenarios, when clang-format runs as a hook on file save
> > when I make any (even noop) edit in
> > https://github.com/pillowtrucker/mir/blob/main/src/platform/graphics/drm_formats.cpp
> > this file, for example, emacs enters an infinite loop consuming 100%
> > of a cpu core and all memory up to gc threshold.
> 

> > It is not always the regex function on the top of the stack but it is 
> > always a call around those functions
> > "beginning-of-defun-raw"
> > "beginning-of-defun"
> > "c-get-fallback-scan-pos"
> > (gdb) bt full
> 

> 

> Yes. c-parse-state is known for sometimes looping - it is an awkward
> spot where doing something rigorusly is likely to be (far) too slow, but
> the ad hoc speed ups occasionally go wrong. That seems to be the
> problem here.
> 

> [ The gdb backtrace (snipped) unfortunately isn't very helpful, here. ]
> 

> > Inside of emacs I get something like this:
> 

> > Debugger entered--Lisp error: (quit) re-search-backward("^\\s(\\|\\(?:^[ 
> > \11]\\(template\\s-<[^>;.{}]+>\\s-\\)?\\(\\(\\(auto\\|const\\|explicit\\|extern\\s-+\"[^\"]+\"\\|extern\\|friend\\|inline\\|mutable\\|overload\\|register\\|static\\|typedef\\|virtual\\)\\s-+\\)\\(\\([[<a-zA-Z][]a-zA-Z0-9]\\(::[]_a-zA-Z0-9]+\\)?\\s-<[<>a-zA-Z0-9
> >  
> > ,]+>\\s-[&]\\|[[<a-zA-Z][]_<>a-zA-Z0-9]\\(::[[<a-zA-Z][]<>a-zA-Z0-9]+\\)?\\s-[&]\\)[&
> >  
> > \11\n\15]+\\)\\)?\\(\\(::\\|[[<a-zA-Z][]a-zA-Z0-9]\\s-<[^>;{}]+>\\s-[&]*::\\|[[<a-zA-Z][]~<>a-zA-Z0-9]\\s-[&]::\\)\\s-\\)?\\(operator\\s-[^
> >  \11\n\15:;.,?~{}]+\\(\\s-*\\[\\]\\)?\\|[~<a-zA-Z][^][ 
> > \11:;.,~{}()\177]\\|[&]?\\([~<a-zA-Z][a-zA-Z0-9]\\s-<[^>;{}]+[ 
> > \11\n\15>]*>\\|[~<a-zA-Z][~<>a-zA-Z0-9]\\)\\)\\s-\\(([^{;])\\(\\(\\s-+const\\|\\s-+mutable\\)?\\(\\s-[=:][^;{]+\\)?\\)?\\)\\s-*\\)\\s("
> >  nil move 1)
> > beginning-of-defun-raw(nil)
> > beginning-of-defun()
> > c-get-fallback-scan-pos(18875)
> > c-parse-state-get-strategy(18875 13474)
> > c-parse-state-1()
> > c-parse-state()
> > c-laomib-invalidate-cache(18875 18880)
> > c-before-change(18875 18880)
> 

> 

> The functions thus far suggest that your file is somewhat unusual; it
> has, perhaps, long stretches of text without braces, or long brace
> blocks.
> 

> Could you possibly send me a copy of the above C++ file, or point me to
> an open website (not github) where I could download a copy? I have no
> access to github, since I am not prepared to let probably hostile
> scripts controlled by Microsoft run on my machine.
> 

> > lsp--apply-text-edit-replace-buffer-contents(#<hash-table equal 2/2 
> > 0x120ad92d>)
> > #f(compiled-function (edit) #<bytecode -0x9b8e84b9515e2ea>)(#<hash-table 
> > equal 2/2 0x120ad92d>)
> > lsp--apply-text-edits((#<hash-table equal 2/2 0x12062b03>) format)
> > lsp-format-buffer()
> > run-hooks(before-save-hook)
> > basic-save-buffer(t)
> > save-buffer(1)
> > funcall-interactively(save-buffer 1)
> > #<subr call-interactively>(save-buffer nil nil)
> > call-interactively@ido-cr+-record-current-command(#<subr 
> > call-interactively> save-buffer nil nil)
> > apply(call-interactively@ido-cr+-record-current-command #<subr 
> > call-interactively> (save-buffer nil nil))
> > call-interactively(save-buffer nil nil)
> > command-execute(save-buffer)
> 

> > This is not the only scenario when this happens. I can also sometimes
> > trigger the same thing by trying to jump to the definition of a symbol
> > from a .cpp/c++-header file into a system header (all of which are on
> > a read-only filesystem on my computer), for example one of the SDL
> > headers. The main difference is that the buffer-switching gets mixed
> > into the lot and emacs doesn't even respond to C-g and I have to kill
> > -SIGUSR2 it to get it to start responding again, but I'm pretty sure
> > it's the same bug - output from this scenario is here:
> > https://0x0.st/X-RT.txt
> 

> 

> What would be most helpful here would be a reliable recipe to reproduce
> the bug without needing other libraries such as lsp, preferably starting
> from emacs -Q.
> 

> > I'm sorry about the <optimized out> everywhere and lack of inlined
> > sources, but I've already rebuilt it with nixos's enableDebugging
> > flag, which should have enabled -Og and -g3 (I think it did do the
> > latter but failed on the optimizations and left it at O2) and I've
> > already spent most of my day debugging this and it would take another
> > full day to figure out how to properly override the byzantine nixpkgs
> > emacs infrastructure with the right optimization flag and to store the
> > right source tree.
> 

> 

> Yes, bugs are like that sometimes. I'm afraid I don't know anything
> about nixos.
> 

> > In GNU Emacs 29.3 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
> > version 1.18.0, Xaw3d scroll bars)
> > Windowing system distributor 'The X.Org Foundation', version 11.0.12101012
> > System Description: NixOS 24.05 (Uakari)
> 

> 

> [ .... ]
> 

> --
> Alan Mackenzie (Nuremberg, Germany).

init.el
Description: Text Data

COPYING-AGPL
Description: Binary data

COPYING.GPL3
Description: Binary data

publickey - pillowtrucker@proton.me - 0x40BB2176.asc
Description: application/pgp-keys

signature.asc
Description: OpenPGP digital signature