Re: Memory usage report

[Ihor Radchenko]

> I think the more useful question is: are all those overlays and text
> properties necessary?  If they are, they take the memory they are
> supposed to take.

Some of the overlays and text properties are not strictly necessary.
For example, text properties are sometimes used as cache to avoid
parsing text multiple times. Is the resulting speedup worth extra memory
usage? It is not clear since we do not have an easy way to determine the
extra memory usage. 

>> Thanks! I did not know that. Would it be possible to have a "real"
>> memory profiler showing how much the memory usage changed after-before
>> running separate functions?

> If you mean memory used by Lisp objects, then GC reports that.
>
> If you mean memory used by non-Lisp objects, I don't see how we could
> produce that without having infrastructure for tracking memory
> allocation, something that debugging malloc libraries already do.

Hmm. I am looking again at the profiler-report output. It seems to
report the memory allocation for individual function calls (that's what
I meant by "real" memory profiler). Do I miss something?

Function                                                  Bytes        %

- command-execute                                         516,227,475  77%
 - call-interactively                                     516,222,195  77%
  - funcall-interactively                                 516,222,195  77%
   - helm-M-x                                             203,107,508  30%
    - helm-M-x-read-extended-command                      203,107,508  30%
     - helm                                               202,313,652  30%
      - apply                                             202,310,580  30%
       - helm                                             202,310,580  30%
        - apply                                           202,310,580  30%
         - helm-internal                                  202,310,580  30%
          - helm-display-buffer                           112,541,910  16%
           - helm-default-display-buffer                  112,540,854  16%
            - display-buffer                              112,540,854  16%
             - display-buffer--maybe-pop-up-frame-or-window   112,540,854  16%
              - display-buffer--maybe-pop-up-window       112,540,854  16%
               - display-buffer-pop-up-window             112,540,854  16%
                + window--try-to-split-window              56,401,971   8%
                  window--display-buffer                   56,130,435   8%
                  window--maybe-raise-frame                     5,280   0%

Best,
Ihor


Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@gmail.com>
>> Cc: larsi@gnus.org, emacs-devel@gnu.org
>> Date: Sat, 19 Sep 2020 22:34:28 +0800
>> 
>> >> top shows more than 200Mb memory increase.
>> >
>> > Which are explained by the statistics produced by GC.  IOW, you have
>> > many more live Lisp objects, which take up those megabytes.
>> 
>> I meant that 200Mb is Lisp objects, but top shows more then 200Mb
>> (around 300Mb). So, there is extra 100Mb coming from somewhere else.
>
> I got a different (smaller) value, but it isn't worth to continue this
> argument.
>
>> > Since that's related to Org buffers, the best place to discuss this is
>> > on Org mailing lists.  Perhaps there are ways to make Org use less
>> > memory, but the expertise for that is there.
>> 
>> The problem is how to identify where the memory usage is coming from.
>> Indeed, org is using overlays and text properties extensively. But how
>> much do those influence the memory usage?
>
> I think the more useful question is: are all those overlays and text
> properties necessary?  If they are, they take the memory they are
> supposed to take.
>
>> > The "memory" profiler doesn't measure the usage of memory, it measures
>> > CPU usage triggered by memory allocation calls (instead of the
>> > periodic profiling signal).  So this profile is not supposed to be
>> > useful for profiling memory usage.
>> 
>> Thanks! I did not know that. Would it be possible to have a "real"
>> memory profiler showing how much the memory usage changed after-before
>> running separate functions?
>
> If you mean memory used by Lisp objects, then GC reports that.
>
> If you mean memory used by non-Lisp objects, I don't see how we could
> produce that without having infrastructure for tracking memory
> allocation, something that debugging malloc libraries already do.