Re: [Help-gnunet] Error uploading file

[Christian Grothoff]

On Friday 16 December 2005 12:00 pm, David Kuehling wrote:
> >>>>> "Christian" == Christian Grothoff <address@hidden> writes:
> >
> > Hmm.  The #bytes in datastore look very close to 2 GB.  Makes me
> > wonder if you might be using a mysql/sqlite table where the 2 GB limit
> > is present and reached.
>
> What does determine, whether the mysql/sqllite table is limited to 2^31?

Well, AFAIK mostly the underlying Linux filesystem (and maybe the way the 
mysql/sqlite binary was compiled). On modern systems, both should not be an 
isuse.

> This is certainly not a filesystem issue, since the size of the gnunet
> database-file is already beyond the 2GB limit:
>
>   ll /var/lib/GNUnet/data/fs/content/
>   -rw-r--r--  1 gnunet gnunet 2282303488 2005-12-16 20:38 gnunet.dat

See, not the issue.  Still I would think performance will likely be 
not-so-great with such a huge DB (indexing is good for performance!).

> > I think the best cause of action is to run gnunetd with -L DEBUG and
> > look at what it says when gnunet-insert dies.
>
> I reran the whole process with gnunetd freshly restarted and the logfile
> deleted prior to invocation.  Looking at the logfile I already see one
> problem, but this shouldn't have an effect on gnunet-insert:
>
>   Dec 16 20:18:03 FAILURE: `open' failed on file
>   `/var/lib/GNUnet/data/fs//bloomfilter' at bloomfilter.c:393 with error:
>   Permission denied
>
> And gnunet is right, the bloomfilter-file is owned by root and not by
> the "gnunet" user.  Certainly my fault...
>
> However I cannot find any messages related to the failed gnunet-insert.
>
> > Also, did you insert or index the 2 GB of content?  Either way,
> > if that does not solve the mystery, please send me the (relevant) log
> > output so that I can have a look.
>
> I always insert files fully "gnunet-insert -n", since this way it is
> easier to manage the memory used by gnunet.  Only one 10Gb database at
> one place...

I'm not sure I understand this.  The *memory* used by gnunet?  You'll just use 
much more disk space (making a copy of the files in the DB) instead of 
linking to existing content.  This will cost you DB access performance...

> I will send you the full log output in private mail, since if
> insufficiently cleaned, it might compromise the anonymity of my node.
> Yes, being paranoid is the primary reason behind using gnunet :)

Ok, will look.

Christian