[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: How to avoid large unwanted cell-heap-segments?
From: |
Roland Orre |
Subject: |
Re: How to avoid large unwanted cell-heap-segments? |
Date: |
Mon, 11 Oct 2004 23:55:56 +0200 |
I solved my gc-problem with a quick and dirty solution so far.
With the procedure below "gc-heap-size" I can set the heap size
to a low value when my data storage is allocated.
By doing:
(define gc-heap1 (gc-heap-size 1))
(define gc-heap2 (gc-heap-size 2))
and then doing (gc-heap-size 1 gc-heap1))
at some critical points my process now runs smoothly without
allocating incredible much cell heap.
(the reason for doing this is that I was stubborn enough to
stick to running this project on my laptop which only has 1Gb,
but I saw that as no problem as the data base only used 300 Mb)
It is now 6 years since the generational gc discussion started
on this list. Some time in the future I may be interested in
implementing it, but.. I have a lot of other stuff that need
implementation first (as I think for most of us) but I guess
it is easier now with the garbage bits separated. Has anyone
had any recent consideration about generational gc?
Here is a link I found:
http://www.ipsj.or.jp/members/Trans/Eng/03/2000/4109/article008.html
and here are some comments from 2000 from the guile list
http://sources.redhat.com/ml/guile/2000-03/msg00561.html
Best regards
Roland Orre
extern struct scm_t_cell_type_statistics scm_i_master_freelist;
extern struct scm_t_cell_type_statistics scm_i_master_freelist2;
SCM_PROC(s_gc_heap_size, "gc-heap-size", 1, 1, 0, scm_gc_heap_size);
SCM scm_gc_heap_size( SCM which, SCM hsize )
{
long newsize;
SCM_ASSERT( SCM_INUMP(which), which, SCM_ARG1, s_gc_heap_size );
if (! SCM_UNBNDP(hsize)) {
SCM_ASSERT( SCM_INUMP(hsize), hsize, SCM_ARG2 ,s_gc_heap_size);
newsize=SCM_INUM(hsize);
}
switch (SCM_INUM(which)) {
case 1:
if (!SCM_UNBNDP(hsize))
scm_i_master_freelist.heap_size = newsize;
return SCM_MAKINUM(scm_i_master_freelist.heap_size);
break;
case 2:
if (!SCM_UNBNDP(hsize))
scm_i_master_freelist2.heap_size = newsize;
return SCM_MAKINUM( scm_i_master_freelist2.heap_size );
break;
default:
scm_wta( hsize, (char *)SCM_ARG2, s_gc_heap_size );
}
} /* scm_gc_heap_size */
On Mon, 2004-10-11 at 14:33, Roland Orre wrote:
> Does anyone have any hints about how to avoid large unwanted
> cell-heap-segments?
>
> Between these two statistics below, the memory usage has increased
> from 460 MB to 640 MB, the only thing being done is performing an
> iteration over a vector (3109805 items) and performing a few
> array-map! and some float calculations for each vector item.
>
> The only noticable difference are these cell-heap-segments:
> (1394960384 . 1575913472) (1578743808 . 1715167232)
>
> After this each garbage collection run takes a lot more time...
> If generational gc was implemented then one solution would possibly
> be to dedicate a "generation" for static memory.
>
> Ideally I would now want to be able to allocate my data structures
> (cells, vectors and uniform vectors) in non gc visible memory, to
> not let the garbage collector bother at all about this large static
> memory (a data base read from disc). (I could of course do this by
> using private structures and smobs, but this implies more work...)
>
> Any hint how I could (easy...) avoid this kind of problem?
> About 8 years ago I had a similar problem, then I patched gc.c
> and changed the cell heap allocation memory growth.
>
> Best regards
> Roland Orre
>
> ;before
> ((gc-time-taken . 48862) (cells-allocated . 33781163)
> (cell-heap-size . 33994558) (bytes-malloced . 172137454)
> (gc-malloc-threshold . 179237085) (gc-times . 318)
> (gc-mark-time-taken . 48541) (cells-marked . 1690839800)
> (cells-swept . 3342457125) (malloc-yield . 2) (cell-yield . 42)
> (cell-heap-segments
> (134795264 . 134830080) (1077979136 . 1078243328)
> <snip>
> (1241004032 . 1301637120) (1302587392 . 1393537024)))
>
> ;after
> ((gc-time-taken . 52196) (cells-allocated . 38980360)
> (cell-heap-size . 73511652) (bytes-malloced . 177976821)
> (gc-malloc-threshold . 179237085) (gc-times . 322)
> (gc-mark-time-taken . 51871) (cells-marked . 1837245337)
> (cells-swept . 3545722215) (malloc-yield . 2) (cell-yield . 29)
> (cell-heap-segments
> (134795264 . 134830080) (1077979136 . 1078243328)
> <snip>
> (1241004032 . 1301637120) (1302587392 . 1393537024)
> (1394960384 . 1575913472) (1578743808 . 1715167232)))
>
>
>
>
> _______________________________________________
> Guile-user mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/guile-user