Re: Fwd: Re: [Help-gnunet] finding files & database management

[Krista Bennett]

Before I start my answer, I misspoke in my last email; you actually 
don't overwrite the top keyword block by reinserting another with the 
same name (unless it has the same description, etc.). You simply create 
a second block with the same keyword referring to the same file. It's 
been a long time since I thought about this stuff, and Christian has 
kindly corrected me :)

This is why I never say anything on the list! *grin*

Ok, so on to the reply...

Benjamin Kay wrote:
> I don't really care too much about plausible deniability. I encrypt my
> harddrive. With that out of the way, records of what I've indexed and how
> would be really useful to me. I don't really care about records of what I've
> inserted since, if I inserted it instead of indexing it, I may actually be
> going for that plausible deniability thingy.

Well, it's certainly something that could be done, and I'll see about 
putting something like that in this weekend if I get to it as an option 
for anyone who'd want to use it.

> Of course. But modifying the description could be beneficial to users who see
> the description off of my node or off of a node that migrated the content
> after the description change.

No, I agree, it certainly has a useful purpose. I was just explaining 
why it was designed to be such a pain in the butt.

>  Ugh! I can't even see the keywords/descriptions on indexed files? 

I'll give some more information here, but see 
http://www.ovmj.org/GNUnet/faq.php3?xlang=English#tell for an explanation.

For the keywords and descriptions, as far as what actually happens under 
the hood, both indexing and inserting a file still encode the file the 
same way; the intermediate blocks necessary for forming queries (and 
this includes the keyword block) are still created and inserted for 
both; indexing simply trades encrypting and inserting the entire file 
for telling the database where to find those blocks to encrypt them on 
the fly if they're requested. The keywords and descriptions aren't 
stored anywhere separately on their own for later retrieval; they're 
encoded just like they would be if you'd inserted the file with them.

Since the block containing keyword data and descriptions is the same for 
both, there's no easy way to retrieve anything other than file names. 
Thus, the only way to really do this would to be to keep an external 
record, which is how I'll try to give you a way to do this.

>  It did occur to me that if I keep a list of all files I've indexed (which 
> GNUnet
>  does for me) and if I put the filename as one of the keywords (which
>  libextractor does for me) then I can see the description by searching GNUnet
>  for the filename. But I don't think that will show me the keyword. 

You're right, it won't. See above.

>  Is there a way to see the keywords if you, say, have the URI of the file you're
>  looking for (as this would solve the problem)?

Nope; this is, in a sense, asking the same question in a different way. 
The URI doesn't "contain" the query keyword either.

GNUnet is designed so that those queries are not reversible to see what 
was asked for unless you know the query (in this case, keyword) in the 
first place (that's why we use one-way hashes instead of some other 
scheme). So the only real way around this is to keep track of those 
keywords somewhere and reindex, at the very least, the top block, which 
is what we'll do.

>  OK. I'll clarify. I download a file. I have content migration enabled. So
>  when I downloaded that file, there's an awfully good chance it got inserted
>  into my node as migrated content. If that's the case, it would be really
>  nice to be able to tell GNUnet to reindex the file (so it doesn't fall off
>  my node and so I have a record of it) without wasting all that time indexing
>  it manually and making up new keywords.

Well, a user chooses to keep an external record of keywords, 
descriptions, and filenames, I don't see why this too couldn't be 
automated if a user wanted it to be.

> And the million dollar question: how to I get GNUnet to overwrite that first
> index block in the database (as this seems to be how descriptions and
> keywords may be modifyed)?

As I said before, you can't overwrite it, but you can at least add a 
corrected version.

Lazy answer: I'll see what I can do about making that available over the 
weekend.

> Thanks for your help,

No problem. I'll send a mail to the list once I've hacked something up.

Hopefully I haven't screwed something up THIS time...

- Krista