Re: [Vrs-development] Re: Overview

[Chris Smith]

On Wednesday 13 February 2002 19:05, Bill Lance wrote:

> Yup.  Your right there.  Some initialization
> configuration files are probably going to be needed.
>
> I was hoping, however, that we could keep all running
> cluster configuration data off any disk at any time.
> It should stay mirrored in running trusted nodes.

You're still going to have private LDS configuration files that don't fit 
into this scheme.  Though they don't have to contain anything sensitive.  
Things like what ports to bind to, where the Discovery server is blaa blaa.

> All LDS's have three startup modes when they execute
> on their host computers;
>
> 1) Cluster Bootstrap
> 2) Subscribe to Cluster
> 3) Defalt Login
>
> The Bootstrap mode is closely associated with the
> Cluster Administrator role. In fact, when you run the
> 'bootstarp' mode, you are by definition the
> Administrator for the Cluster you are about to create.
> When it first starts, new VRS has no datasets
> registered and only itself on the subscription list.
> It then listens to the net for a subscription request.
>
> Next, the second LDS starts in the 'Subscribe to
> Cluster' mode.  It asks for the IP of the first LDS,
> calls in and signs up (there is an interactive session
> with the new user here to collect id data as needed.)
>
> Now we have two machines in the young CLuster.  Now
> Dataset can be registered, stored and services can
> begin.  Presumably, the Cluster grows from here.

So you need two machines for a cluster to be usable.  Thats fine.
But..... you've discovered the problem..... later....

> The second issue is running a total VRS on a single
> machine, just one LDS.  Any one LDS contains all the
> functionality to do this.  The question may be 'Why do
> this?'

Because......

> There is one time where it becomes a serious question.
>  That's the reverse of the bootstrap process at the
> end of a Cluster's life cycle.  What happens if a
> Cluster shrink to only two LDS's, and one of them
> drops.  The Cluster could die right then and there, or
> it could try to hold everything alive on the one
> surviving machine. Technically, either should be
> possible.

Which was what I could see happening.  If a cluster can be reduced down to a 
single LDS through failures/loss of interest by other LDS's, then does the 
'uni machine cluster' stop serving requests because there is only one LDS 
left?

If the answer is NO, then you must allow the cluster to start up with a 
single LDS.

It's an interesting issue.  However, if you mandate that a single machine in 
a cluster may only store n-1 chunks of data, where n is the number of 
machines in the cluster, then when you're down to 1 machine, you cannot 
satisfy any requests.

Ah. Problem.
Cluster starts with 2 machines.  Resources are registered with the cluster 
and shared across the machines.  2 more machines join the cluster.
[Q: is the data re-partitaioned across the 4 machines]
More resources are registered with the cluster.
2 machines now leave (or rather DIE HORRIBLY, taking their data share with 
them.
Is the data set complete for all resources registered?

The trick will be to have a fixed minimum cluster size (x) and have each 
machine hold x-y chunks of data (where 1 <= y <= x/2) which means that a 
single machine will never have the complete set of data chunks, and never 
have less than half the set of chunks.  Each machine gets different chunks 
for the set.
This allows for a finite number of machines leaving the cluster without 
warning and the full dataset being available.

[ Someone PLEASE check that and point a hole in my logic / math! It came to 
me in a flash and seems far to easy on paper! ]

> Putting everything on one machine violates some of the
> data distribution privacy and security rules.  But it
> does allow for a final emergancy backup mechanism.  It
> becomes an administrative configuration question then.

Yeah.  I'd not have the entire dataset available on a single LDS UNLESS it is 
a Private_LDS ( I think we need a proper term for this class of LDS to help 
with discussions - how about Private Data Server PDS or LDS/P)

> indeed I am.
>
> I'm still thinking about that perl prototyping and
> conversion to C as a overall project framwork too.

Yeah.  As long as the interface to the Goldwater Services you construct do 
not pass language data structures (like serialised Perl Hashes or C Linked 
Lists) then you can write the services in Perl or C.
For a secure system I'd use C though as Perl is hackable at any time..... 
unless we run Perl's byte code when it's available - or even Rhys's C# !!!!


Chris


-- 
Chris Smith
  Technical Architect - netFluid Technology Limited.
  "Internet Technologies, Distributed Systems and Tuxedo Consultancy"
  E: address@hidden  W: http://www.nfluid.co.uk