Re: [geni-dev] CF Requirements: 3) Slice Controller

I guess this is the area where most of my energy has gone over the  
last few years :)

On 20/03/2009, at 5:24 AM, Jeannie Albrecht wrote:
> One point that I'd like to emphasize is that I think the functionality
> of the SC should be largely independent of the underlying control
> framework.

This won't be possible as the SC needs to interface with it. It needs  
to obtain the resources the experimenter wants and needs to  
(optionally) tracking the state of the assigned resources.

Now, given that we are trying to come up with a common architecture  
and common data/resource models it shouldn't be that difficult to  
target a specific SC to a different control framework.

You mentioned that you have done that already for more than one and it  
also didn't take us long to get our version of the SC running on top  
of Planetlab. We haven't tried any other ones.

> but I believe that a single SC could actually work
> across Clearinghouses.

Yes, that is very important and it's one of our deliverables (not in  
the GENI context) to support experiments across federated testbeds  
which will have separate CHs.

> Or maybe there needs to be an Experiment
> Controller that is built on top of a Slice Controller, so the slice
> controller handles only CF-specific actions. (This is basically how
> we've structured Gush to work with both PlanetLab and Orca using the
> same front-end.)  Then multiple control frameworks could essentially
> co-exist in GENI, and researchers would only have to know one
> interface/front-end for managing experiments.

That's more a pragmatic approach. You either wait for all the control  
frameworks to agree on common interfaces or you write your own glue to  
the common abstraction you think is most useful. Should be a pretty  
easy choice :)

> As we flesh out the requirements of the SC, at some point we should
> start thinking about what defines a GENI experiment.  (Although
> perhaps the experiment workflow working group is a better place for
> that discussion.)  We have had numerous discussions on RSpecs, for
> example, but as far as I know we have yet to talk about an ExpSpec
> (experiment specification language).  How should a GENI researcher
> define a GENI experiment?  I personally believe that one component of
> an ExpSpec is an Rspec, but there are many other factors to consider
> here as well.

I fully agree that we need ways to describe experiments. However,  
there is no pressing need to standardize that or agree on one as this  
will initially be just between the researcher and their tool. Having  
said that, I have been preaching for years that a fundamental  
requirement for supporting repeatable experiments is the ability to  
describe it and fully capture all externalities. What I want to see at  
some stage is that any graph in a paper showing experimental results  
is referencing the experiment description it used - enabling others to  
repeat the experiment.

Now, I'm not sure if ExpSpec has the right connotation. First of all,  
RSpec won't be sufficient as we need to capture relationships as well,  
but I'm repeating myself.

I believe it needs something more like a language. In Orbit we have  
OIDL, which is effectively a domains specific language (DSL) built on  
top of Ruby, but most experimenters need to know very little about  
that. What is more important is the programming model. In our case the  
experimenter first describes all the resources they need (here is the  
link to resource descriptions and what's need to be requested from the  
various AMs and their respective CHs) and then they describe the  
"orchestration" of the experiment. n most cases this is along a time  
line but given that measurement collection is tightly integrated we  
can also easily describe "steerable" experiments (e.g. increase load  
by x% every y seconds until the error rate is above z%)

The orchestration is modeled as a state machine which makes it easier  
to deal with errors, or more generally with events coming from the  
resources inside the slice (e.g. an application crashing), or from the  
control framework (e.g. a sliver crashed). This way we can, for  
instance, maintain a certain level of resources by requesting new ones  
when we fall below a certain threshold. A state machine allows me to  
add all that outside my "main" experiment description. Right now, most  
users in Orbit don't even know that their experiment has quite a bit  
of failure recovery built in this way.

>  I'm assuming that the SC is responsible
> for controlling experiments.  Are Experiment Controllers and Slice
> Controllers the same thing, or should they actually be viewed as
> separate components in the GENI architecture?

That's an excellent point. Especially for short experiments (or trying  
to get something to work) we want the slice to be more persistent to  
keep the overhead down.

One idea we have been throwing around for a while is that of a  
"virtual" testbed. In all of our installations an AM represents an  
entire testbed and it's the AM the experimenter is directly talking  
to. Now if the AM manages a set of resources, and slivers are  
essentially resources as well (virtualization is supposed to be  
transparent) is the interface for an experimenter to manage all it's  
resources fundamentally different?

I assume one of the reasons for Jeff to bring the SC into the  
discussion is that the control framework needs a way to send  
information to the experimenter when things change, or whatever. A  
classical web service abstraction doesn't provide a simple to send  
asynchronous messages. We could do polling as we do with many web  
services, but given that we want a decentralized architecture there  
would be a lot of that, so why not have something for ET to call.

Now it's way past my bedtime to argue against web services as well and  
I'm not sure if these last few paragraphs make much sense anymore.

Cheers,

-max
>

_______________________________________________
control-wg mailing list
control-wg@...
http://lists.geni.net/mailman/listinfo/control-wg