[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Gzz] Raw pools?
From: |
Benja Fallenstein |
Subject: |
Re: [Gzz] Raw pools? |
Date: |
Mon, 18 Nov 2002 19:14:47 +0100 |
User-agent: |
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020913 Debian/1.1-1 |
Hi Tuomas,
Tuomas Lukka wrote:
On Sun, Nov 17, 2002 at 10:46:17PM +0100, Benja Fallenstein wrote:
Tuomas Lukka wrote:
Between pools??
Yes, so? (Whenever a block moves, it moves between pools? Except if a
pool moves physically, such as moving a disk to a different place, which
is not interesting to consider.)
Oh, my definition of pool was "all mediaserver repositories anywhere with
the same pool ID" -- more virtual.
Hmm, right, that was the original definition... I'm getting all confused
thinking about this, right now it's not clear to me at all what the
boundary of a pool in that sense should be and how to rightly decide in
which pool to put a block etc. -- I considered pool to be one storage
location, that could be addressed through a StormPool object. I guess I
then thought that pools'd have names (ids) and names could be shared
between pools... I.e., for me, mediaserver repository = storm pool.
Can someone make use cases for multiple pools on the same computer (when
we want to use them, and in which pools the user would expect specific
blocks to go)? I'm confuzzled right now.
So you mean moving between physical locations?
Yes, where 'physical location' means Storm pool ;-P
No, really, it includes moving data from a CD to the hard disk or to a
DHT as well as moving data from the 'cache' to the 'permanent' pool
where it won't be deleted automatically.
Quite a suite. Would greatly help people trying to understand what we're
up to, though.
How about starting with
High-level
1. Storm concept and rationale
- eternal blocks, moving easily
Should also have the commitment PEG at the same time.
Well, that *could* also be later, as making this more exact.
Ok.
2. Storm and crypto: signatures, identities
What's this?
The overall intent on how Storm is supposed to work with crypto.
Hm, ok.
Implementation
1. Storm block format
2. Storm pointers layer 1: no crypto, nothing except branching versioning
Should also have the pointer concept PEG then. Describing the formats
without the rationale doesn't make much sense.
I didn't say formats here. This *would* be the concept. ;)
Then it's not implementation. :-)
Note that pointers are to Storm like TCP is to IP-- TCP is *one*
application of IP, though a very important one. So it's not like
pointers are a way of implementing a part of the Storm core; they
implement a service on top of the Storm core which may be useful for may
applications (but not all: xanalogical content blocks are specifically
*not* versioned).
Also, 'implementation' is a misnomer: 'Formats' would be better.
Well, then the above pointers thing shouldn't be here.
*gg*
Ok, it's just words ;-)
Ok. Canonical blocks are bad, because
1) they don't solve the problem of getting the unadulterated data's
SHA-1
except through new features which just make things more complicated
(everyone has to support those mappings etc!)
Yes, but not *in the core*. It is an application, so latter implementors
of the Storm formats won't have to support it. It might be nice if
*someone* still implemented that application, but you could implement
the Storm core without implementing these applications, too. (Which
*may* be unnecessary because it may be either possible to retrieve the
block itself, or alternately impossible to retrieve the article alone,
because it dropped off the web.)
2) they have a part of the normal header but not all; everyone needs to
support
that as well.
You are saying that Storm blocks are required to have a Date: header
etc.? That's *really bad*, we need canonicality for many applications
(for example the email client, which takes care that equal body parts of
the same email received through different paths get the same xanalogical
id).
Also, I think this is very true for pdf/ps also: If I insert a PDF and
make links to it, and someone else inserts the *same* PDF and makes
links to it, the copies should have the same xanalogical id (thus this
needs canonical headers even if header and body are separated). There's
no problem with this here: any article PDF will contain enough data that
it is impossible that someone else entered the same data by chance.
I think they get us *more* complexity than the header-block separation would,
so it might be good to ditch them *now*.
I agree that 1) above is additional complexity; but I don't think that
we actually need to support it eternally, as the header-block separation
(because e.g. an external mapping would be *external to Storm*; if at
some point we have xu links between blocks, or are able to distribute
the Storm blocks or whatever, we can ditch it *without breaking any
Storm blocks*, just one specific method to create these blocks by
downloading them by SHA-1). I don't agree that 2) is additional
complexity; IMO we need this anyway.
- Benja
- Re: [Gzz] Raw pools?, (continued)
- Re: [Gzz] Raw pools?, Benja Fallenstein, 2002/11/10
- Re: [Gzz] Raw pools?, Tuomas Lukka, 2002/11/10
- Re: [Gzz] Raw pools?, Benja Fallenstein, 2002/11/10
- Re: [Gzz] Raw pools?, Tuomas Lukka, 2002/11/11
- Re: [Gzz] Raw pools?, Benja Fallenstein, 2002/11/11
- Re: [Gzz] Raw pools?, Tuomas Lukka, 2002/11/16
- Re: [Gzz] Raw pools?, Benja Fallenstein, 2002/11/16
- Re: [Gzz] Raw pools?, Tuomas Lukka, 2002/11/16
- Re: [Gzz] Raw pools?, Benja Fallenstein, 2002/11/17
- Re: [Gzz] Raw pools?, Tuomas Lukka, 2002/11/18
- Re: [Gzz] Raw pools?,
Benja Fallenstein <=
- Re: [Gzz] Raw pools?, Benja Fallenstein, 2002/11/16
- Re: [Gzz] Raw pools?, Tuomas Lukka, 2002/11/17