Re: Memory usage report

[Ihor Radchenko]

> Where do you see evidence for a significant memory in
> the heap?

top shows more than 200Mb memory increase.

> So the interesting question is: where are those 7GB used?

1Gb in lisp objects and 20%*8Gb=1.6Gb total. I am not sure about those
extra 0.6Gb, but answering about the 1Gb in lisp objects would also
help. That's why I asked earlier about buffer-local variable summary and
summary about library variables.

> And that is actually the bottom line here: the GBytes of memory used
> by such large Emacs sessions don't seem to come from Lisp objects,
> they are used up in some other way.  The question is how and where in
> the code this happens.

While I would also like to answer this question, I also have large
fraction of lisp objects. As I mentioned in previous emails (sorry if I
was not clear), I would appreciate some way to understand why my **lisp
object** memory grew from 283Mb right after loading my org files to 1Gb
later.

P.S.

I tried to use memory profiler to understand the memory usage, but the
memory profiler report appear to show peak memory usage (there tend to
be functions with many let-bindings). So, it seems not be to useful for
me (correct me if I am wrong).

Best,
Ihor


Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@gmail.com>
>> Cc: larsi@gnus.org, emacs-devel@gnu.org
>> Date: Sat, 19 Sep 2020 00:14:34 +0800
>> 
>> Before:
>> Total in lisp objects: 77.4MB (live 47.7MB, dead 29.7MB)
>> Buffer ralloc memory usage:
>> 16 buffers
>> 39.3kB total (23.6kB in gaps)
>> 
>> After:
>> Total in lisp objects:  283MB (live  218MB, dead 64.7MB)
>> Buffer ralloc memory usage:
>> 47 buffers
>> 6.79MB total (78.6kB in gaps)
>> 
>> This is not all the memory increase. Some part seems to be in heap.
>
> I see almost all of the 200MB of additional memory in Lisp objects:
> mainly conses, the rest in vectors and intervals (i.e. text
> properties).  Where do you see evidence for a significant memory in
> the heap?
>
> I guess you could ask Org developers a question regarding this high
> memory usage.  In any case, this is not what people report in
> bug#43389, or so it seems.
>
>> Also, the memory usage is even worse when org-mode is using overlays (I
>> am on org-mode branch using text properties instead of overlays in org).
>
> Looks like you have 30 Org buffers whose total size is below 10MB?  If
> they use a lot of text properties and overlays, it's a small wonder
> you get high memory usage.  I don't necessarily see a problem here.
>
>> The memory consumption gets even worse after I use Emacs for several
>> hours. Recently, it tend to settle around 20% (of 8Gb) with lisp objects
>> taking around 1Gb.
>
> So the interesting question is: where are those 7GB used?
>
>> This last part is the most annoying and also difficult to track - I have
>> no easy way to know if it is also caused by org-mode or it is something
>> else.
>
> The only way to investigate this is to use a good memory-mapping
> utility and perhaps also a debugging malloc library.
>
> And that is actually the bottom line here: the GBytes of memory used
> by such large Emacs sessions don't seem to come from Lisp objects,
> they are used up in some other way.  The question is how and where in
> the code this happens.