Re: [GNUnet-developers] Insertion speed

[Christian Grothoff]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sure, the directory storage could be made better by adding multiple
levels (see also: squid!). Since the directory approach often takes
a lot more space than gdbm, I did not bother with this yet. But yes, this 
would be useful. I also wonder how fast tdb is. Did you do the insertion 
with the "-n" option or without? 

Christian

On Tuesday 06 August 2002 07:01 am, Niklas Höglund wrote:
> I've experimented a bit with the insertion speed.  I did a benchmark
> using different storage methods and file systems to see if I could get
> gnunet to insert files faster.
>
> First I used a 37M file, that I inserted into the same GDBM database on
> different filesystems.  What filesystem was used didn't affect
> performance a lot.  It took between 633 and 722 seconds for all my
> tests.
>
> Then I created a new gnunetd with support for the directory backend, and
> inserted the same file into it.  It took only 124 seconds.  Quite fast.
>
> I realized, though, that it may be faster to insert into an empty store,
> so I filled it with about the same amount I had had earlier.  That made
> the speed go down to about the same as with the GDBM store.
>
> As I see it, there are two things that could cause this: the size of
> that directory or filesystem fragmentation.
>
> It might be a good idea to insert one or two levels of directories into
> the directory store.
>
>
> By the way, noone should call me a leecher:
>
> Server Statistics:
> Shared files       :         3
> Size of shared data:  4294967295k
> Connected hosts    :         8
> Uptime             :    479620s
>
> I share quite a bit of data :)

Well, this is not gnutella, so sharing will only help you if somebody else 
wants this data (thus /dev/random will not work :-). What is astonishing is 
that you have only 3 shared files -- so either your files are gigantic or you 
used the "-n" option a lot. You may also want to check the comment in 
config.h -- by default gnunet is compiled with a set of options that limits 
the amount of shared data to 8 GB (which you seem to be over) -- which makes 
me wonder how accurate that number can be (or if there's a bug in the code 
and we're printing something wrong). 

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE9UHHJ9tNtMeXQLkIRAndnAJ9LbQPQtlIkNX8InvdkSIPZ4vA2pgCgo/Tu
+P4d8vHqJNLdAOOTnc5MeAc=
=Apsq
-----END PGP SIGNATURE-----