Re: [Help-gnunet] indexing -> size taken in database

[Christian Schulte]

On Fri, 16 Apr 2004 15:43:34 -0700
Steven <address@hidden> wrote:

> On Friday 16 April 2004 12:45 pm, Brent Miller wrote:
> > If you mean inserted content, that's just what happens. I've found that
> > I've been unable to download inserted content only 12 hours after
> > inserting it.
> 
> This sounds like a serious issue!  Perhaps the distributed file storage in 
> Gnunet isn't working.  I wonder what percentage of un-indexed files are 
> actually retrievable in Gnunet.  Perhaps this kind of distributed file 
> storage system is not viable without FEC redundancy.  (perhaps the block size 
> is a problem as well?)

FEC might be of great help but ...
As is see the situation of my node the problem is not caused by malfunction of 
GNUnet it is more a lack of control over what happens in my database.
If i have a 1Gb sized DB and i insert a few 100MB and go on indexing some other 
100MB
i dont know when i will reach the point where a new indexed block will start to 
throw
out my own content instead of migrated one.
But i have to mention that the statement in the config file sais: 

# Currently, activating active migration can cause some problems when
# the database is getting full (gdbm reorganization can take very,
# very long and make GNUnet look like it hangs for that time). Thus if
# you turn it on, you may want to disable it after you hit the
# quota. A better content managment system should solve this problem
# in the near future...


> If you were to insert a 100MB file that would create 100,000 (?) 1k blocks 
> that scatter randomly across gnunet.  If even one of those blocks falls off 
> the network, that file is completely irretrievable.  Maybe some statistics 
> guru would be able to come up with a figure that would show the probability 
> of retrieving this kind of data?  After all, if this aspect of gnunet isn't 
> working, it would seem to me that the security of Gnunet is at stake as well. 
>  
> It seems that security wise, gnunet would be reduced to that of MUTE but with 
> the possibility of a direct connection between sender and receiver.

This direct connection between sender and receiver is exactly what makes MUTE 
more vulnerable. GNUnet uses far more advanced techniques than just encryption
and small blocks and the theory which GNUnet is based on considers a potential
attacker to be much more powerfull than any MUTE theory does. 

Anyway as you and Benjamin said we need to have facts (by testing) before 
discussing the results.
For that reason we started collecting legal indexed/inserted content in the 
forum.

I hope that many people will participate and post their content in the forum.
Then if people download other nodes content and post the result with at least 
date, time
and speed back to the forum we could start to analyze the behaviour a little.

> I know that it would be no trivial task, but I think that it would be 
> interesting to do experiments with a network where indexing was disabled.  
> 

If i am not mistaken it might be easier to experiment without indexing because 
the data
amount would be less but the effect of data loss is same fatal with indexed 
content as
with inserted one. If one of the structural blocks of an indexed file gets lost 
it will
become impossible to get the whole indexed file also.

By the way i dont know why Crysslies 3DMark is so slow (for me also) but i had 
much faster 
downloads recently with a little smaller files. Like 2-10MB. See the forum for 
some results
and see how it works for you.



  Chris