Re: [cwg] questions on RSpecs from GEC2 meeting

[Jeff Chase]

Aaron Falk wrote:

> I took a few notes on the questions that came up during Ted & Rob's  
> talks:
>
> Where is time expressed?
>
> How are duty-cycles expressed?
>
> How are probabilistic constraints expressed?
>
> (something about elemental actuators....?)
>
> c.f. sensorML, ontology for services, network markup languages from  
> OGF, resource descriptor framework
>
> How is isolation expressed?
>
> _______________________________________________
> control-wg mailing list
> control-wg@...
> http://lists.geni.net/mailman/listinfo/control-wg
>  

It seems to me that there's a larger core question here that came up
about the role of rspec as a language (format?) for representing
information about resources and slivers in the GENI architecture.   How
do we balance the tension between standardization and extensibility?

There are a couple of options that bracket a continuum:

1. Exactly one: Rspec is the standard specification.  It can be extended
through various consensus processes defined by GENI governance.

2. At least one: Rspec is an initial standard for representing some
information, but there may be other alternatives for representing other
information.    If the various actors (component manager, clearinghouse,
researcher tools) understand some other language, then they can
negotiate for resources using that language instead of or in addition to
Rspec.

Strategy #1 has several risks.  It means we have to invest a lot in
trying to agree on an specification for what GENI actors say to other
actors about resources and slivers, and their topology and
capabilities.  What does it mean for Rspec to be "extensible"?  Can it
incorporate information represented in NDL, or DCML, or sensorML, or
various other languages that other related communities might use?  Or
will we build "our" own language, borrowing from these "other"
communities as we see fit?  Can the Rspec standard evolve quickly enough
to avoid being a barrier to innovation?

The risk of strategy #2 is that we later degenerate into a jello-nailing
exercise of some sort.  But I hope we can keep GENI open enough to
encompass multiple ways of talking about resources, just as the Web has
multiple formats for representing content.  If we can do that, then we
will have a more evolvable result and better inclusion of other groups
who have thought hard about these problems for a long time (e.g.,
http://www.science.uva.nl/research/sne/ndl).  We should at least ask the
question of whether we can get where we want to go in some other way,
e.g., by extending NDL.

Jeff


_______________________________________________
services-wg mailing list
services-wg@...
http://lists.geni.net/mailman/listinfo/services-wg

[Peter Steenkiste]

Hello Jeff,
Your two extremes do look scary.  I have always assumed that the Rspec
would pick an intermediate point in your spectrum: if two parties need
to exchange some information other than the "standard" information in
the Rspec, they would just be able to do that.   "Standardizatin" or at
least rough consensus would be needed if information field became widely
used.

I think that the issues of time, duty cycle, scheduling unit, isolation,
etc. are harder.  When creating a slice, you will need to make sure that
these properties of your slivers "line up" so you get the right behavior
at the slice level.  That looks like a hard problem.  Clearly it will
require consensus over how to represent this information and what
various values mean for different substrates.  This issue exists
independent of whether you include the information in the Rspec or not.

Peter

Jeffrey Chase wrote:

> Aaron Falk wrote:
>  
>> I took a few notes on the questions that came up during Ted & Rob's  
>> talks:
>>
>> Where is time expressed?
>>
>> How are duty-cycles expressed?
>>
>> How are probabilistic constraints expressed?
>>
>> (something about elemental actuators....?)
>>
>> c.f. sensorML, ontology for services, network markup languages from  
>> OGF, resource descriptor framework
>>
>> How is isolation expressed?
>>
>> _______________________________________________
>> control-wg mailing list
>> control-wg@...
>> http://lists.geni.net/mailman/listinfo/control-wg
>>  
>>    
>
> It seems to me that there's a larger core question here that came up
> about the role of rspec as a language (format?) for representing
> information about resources and slivers in the GENI architecture.   How
> do we balance the tension between standardization and extensibility?
>
> There are a couple of options that bracket a continuum:
>
> 1. Exactly one: Rspec is the standard specification.  It can be extended
> through various consensus processes defined by GENI governance.
>
> 2. At least one: Rspec is an initial standard for representing some
> information, but there may be other alternatives for representing other
> information.    If the various actors (component manager, clearinghouse,
> researcher tools) understand some other language, then they can
> negotiate for resources using that language instead of or in addition to
> Rspec.
>
> Strategy #1 has several risks.  It means we have to invest a lot in
> trying to agree on an specification for what GENI actors say to other
> actors about resources and slivers, and their topology and
> capabilities.  What does it mean for Rspec to be "extensible"?  Can it
> incorporate information represented in NDL, or DCML, or sensorML, or
> various other languages that other related communities might use?  Or
> will we build "our" own language, borrowing from these "other"
> communities as we see fit?  Can the Rspec standard evolve quickly enough
> to avoid being a barrier to innovation?
>
> The risk of strategy #2 is that we later degenerate into a jello-nailing
> exercise of some sort.  But I hope we can keep GENI open enough to
> encompass multiple ways of talking about resources, just as the Web has
> multiple formats for representing content.  If we can do that, then we
> will have a more evolvable result and better inclusion of other groups
> who have thought hard about these problems for a long time (e.g.,
> http://www.science.uva.nl/research/sne/ndl).  We should at least ask the
> question of whether we can get where we want to go in some other way,
> e.g., by extending NDL.
>
> Jeff
>
>
> _______________________________________________
> services-wg mailing list
> services-wg@...
> http://lists.geni.net/mailman/listinfo/services-wg
>
>  

_______________________________________________
services-wg mailing list
services-wg@...
http://lists.geni.net/mailman/listinfo/services-wg

[John Wroclawski]

[Resending due to pilot error. Please direct any replies to both
<services-wg@...> and <control-wg@...>]

Hi Jeff,

At 5:54 PM -0500 3/3/08, Jeffrey Chase wrote:
>
>It seems to me that there's a larger core question here that came up
>about the role of rspec as a language (format?) for representing
>information about resources and slivers in the GENI architecture.   How
>do we balance the tension between standardization and extensibility?

It seems worth it to take a minute to frame the context here.

This data structure, whatever you call it, is the key boundary
interface between manufacturers of GENI components, who have to
create them, and the rest of the system, which has to use them. NSF
is eventually proposing to do competitive procurement of $300 million
or so of such components, from formal requests for proposals (which
means well formed specifications), from any number of different
vendors. NSF is also trying to be open to donations of useful
resources (eg, satellites, what have you) from other folks who might
be willing to make them available if they could figure out how.

It is, in my view, a very basic goal of the design that as many of
these components work with as much of the rest of the system as
possible. To make this happen, two things would seem to be true:

1) The design should encourage as much common understanding of what
an RSpec *means* as possible, across different communities,
components, component manufacturers, service creators, tool builders,
and researcher/users - so that we get as much interoperability as
possible, and so that it's as easy (and useful) as possible for a new
component vendor/creator/whatever to add themselves in.

2) The RSpec specification itself has to be as *clear* as possible,
so that random folks can figure out what the heck is going on.

To my mind this argues for as much standardization - or maybe I
should say "agreement" - as we can. If I thought there was any way to
do it, I'd argue that GENI would be best served by a standard
*non-extensible* RSpec, which would at least create some chance that
everything would work with everything else.

But of course that won't work.

So, as you say, we're looking for an extensibility model that
balances these things. I would say it as "an extensibility model that
encourages as much commonality as possible, but still allows for
evolution and change".

Now, in fact, Ted & Rob's talk yesterday outlined a "two-step"
extensibility model - although I agree it went by very quickly. The
main properties of this model are

a) An "object/subclass" flavor, so that, for example, if you have a
new sort of router with unique parameters, you can describe it as
such, and people who understand your parameters will get them, while
others will still understand it as a "router".

b) Within the XML container there are two (at least) levels of
extensibility - call them "private" and "standardized". You're free
to do anything you want with the private namespace - if it made sense
to you to put in a private attribute called "SensorML-Desc", noone
would stop you, although it might have to work as a request format as
well as a description format.

c) We imagine some process - to be clear, *not* the CFWG, since it
has to survive into the construction and operational phase of GENI as
well as the design phase - that would be responsible for elevating
new description attributes from the private to the standardized
namespace. You could also imagine intermediate "sub-community"
processes that would elevate attributes from the private namespace to
a level of shared agreement across some community of interest, such
as sensors. But other than its existence, nothing about this process
has even been proposed yet, let alone defined.


>http://www.science.uva.nl/research/sne/ndl).  We should at least ask the
>question of whether we can get where we want to go in some other way,
>e.g., by extending NDL.

IMO this is sort of an orthogonal issue - if there's an existing (and
fairly community-neutral) format that we can build on to meet the
objectives above, we certainly aren't precluding it. We've already
swiped a lot of existing stuff for interface descriptions, etc. etc.

Cheers,
John

_______________________________________________
services-wg mailing list
services-wg@...
http://lists.geni.net/mailman/listinfo/services-wg

[McGeer, Rick]

There is very little in a data formats discussion that I, at least, find scary.

Look, Rspecs will be wrong.  There will be stuff they omit, stuff they overspecify, and  I guarantee  in five years we'll look at some chunks of it and say what were we thinking?  I also guarantee that Rspec 2.0 will be different from 1.0, since we'll learn things from 1.0.  (In general, 2.0's are more complex and therefore worse than 1.0's, but that is life; if we are very wise and lucky we'll zap stuff between 1.0 and 2.0, but more likely we'll be stupid and unfortunate and add stuff).  That said, this isn't really all that complicated.

At the end of the day, as an experimenter, what I want to specify is a virtual or physical network, and then ask the GENI facility to go instantiate that network for me.  Emulab and PlanetLab both do wonderful jobs at  this, so a good way to start is to take an XMLified version of the Emulab ns commands and the PlanetLab XML-RPC interface and ask why doesn't this work?  I can think of several enhancements to the unified PlanetLab/Emulab interface:

1. Emulab is almost ideal, since I allocate physical nodes and specify node type, which automatically gives me memory, disk, cpu speed and type.    If GENI slices nodes -- and one  lesson of PlanetLab is that we should  use a hypervisor which offers performance isolation -- then I just want to specify memory, disk, cpu speed and type.  Four parameters, and add wildcards.  The emulab system of OSIDs and image IDs work just fine.
2. I need to be able  to pick specific nodes (a la PlanetLab) or at least network node locations ("give me any node at U Chicago meeting these specs", or "give me any node with an interface matching this netmask and these specs").  For Emulab of course this is irrelevant; PlanetLab lets me pick individual nodes but not resource guarantees on them.
3. This doesn't affect me personally, since like any sensible person  I instantiate small networks to run my experiments.  Unfortunately, many others (Kubi and the DHT guys) do not share my good taste :-) and instantiate monsters.  To make life easier on them (why?) we need hierarchical descriptions.  For example, OceanStore really needed clusters at each site to do the signature computations, and so we'll need the ability to specify clusters and instantiate them.  That means hierarchical descriptions, and modules.
4. Truly hoggish experiments (like the ones my buddy Joe Mambretti and I kick around) require gobs-o-bandwidth and optical links, so I need to be able to specify links of truly hideous capacity -- 10 Gb/s and above.
5. A call needs to be made -- are we fooling around at layer 1?  I haven't the faintest idea about what link characteristics need to be specified here, but Ben Yoo and Keren Bergman probably do.
6. Need to specify layer 2 link characteristics.  (Ethernet and parameters, including jumbo and 1522-byte framing, sonet, wifi, fiber channel, etc...

> -----Original Message-----
> From: services-wg-bounces@...
> [mailto:services-wg-bounces@...] On Behalf Of Peter Steenkiste
> Sent: Tuesday, March 04, 2008 6:33 AM
> To: chase@...
> Cc: services-wg@...; control-wg@...
> Subject: Re: [services-wg] [cwg] questions on RSpecs from GEC2 meeting
>
>
> Hello Jeff,
> Your two extremes do look scary.  I have always assumed that
> the Rspec would pick an intermediate point in your spectrum:
> if two parties need to exchange some information other than
> the "standard" information in
> the Rspec, they would just be able to do that.
> "Standardizatin" or at
> least rough consensus would be needed if information field
> became widely used.
>
> I think that the issues of time, duty cycle, scheduling unit,
> isolation, etc. are harder.  When creating a slice, you will
> need to make sure that these properties of your slivers "line
> up" so you get the right behavior at the slice level.  That
> looks like a hard problem.  Clearly it will require consensus
> over how to represent this information and what various
> values mean for different substrates.  This issue exists
> independent of whether you include the information in the
> Rspec or not.
>
> Peter
>
> Jeffrey Chase wrote:
> > Aaron Falk wrote:
> >
> >> I took a few notes on the questions that came up during Ted & Rob's
> >> talks:
> >>
> >> Where is time expressed?
> >>
> >> How are duty-cycles expressed?
> >>
> >> How are probabilistic constraints expressed?
> >>
> >> (something about elemental actuators....?)
> >>
> >> c.f. sensorML, ontology for services, network markup
> languages from
> >> OGF, resource descriptor framework
> >>
> >> How is isolation expressed?
> >>
> >> _______________________________________________
> >> control-wg mailing list
> >> control-wg@...
> >> http://lists.geni.net/mailman/listinfo/control-wg
> >>
> >>
> >
> > It seems to me that there's a larger core question here
> that came up
> > about the role of rspec as a language (format?) for representing
> > information about resources and slivers in the GENI
> architecture.   How
> > do we balance the tension between standardization and extensibility?
> >
> > There are a couple of options that bracket a continuum:
> >
> > 1. Exactly one: Rspec is the standard specification.  It can be
> > extended through various consensus processes defined by
> GENI governance.
> >
> > 2. At least one: Rspec is an initial standard for representing some
> > information, but there may be other alternatives for
> representing other
> > information.    If the various actors (component manager,
> clearinghouse,
> > researcher tools) understand some other language, then they can
> > negotiate for resources using that language instead of or
> in addition
> > to Rspec.
> >
> > Strategy #1 has several risks.  It means we have to invest a lot in
> > trying to agree on an specification for what GENI actors
> say to other
> > actors about resources and slivers, and their topology and
> > capabilities.  What does it mean for Rspec to be
> "extensible"?  Can it
> > incorporate information represented in NDL, or DCML, or
> sensorML, or
> > various other languages that other related communities
> might use?  Or
> > will we build "our" own language, borrowing from these "other"
> > communities as we see fit?  Can the Rspec standard evolve quickly
> > enough to avoid being a barrier to innovation?
> >
> > The risk of strategy #2 is that we later degenerate into a
> > jello-nailing exercise of some sort.  But I hope we can
> keep GENI open
> > enough to encompass multiple ways of talking about
> resources, just as
> > the Web has multiple formats for representing content.  If
> we can do
> > that, then we will have a more evolvable result and better
> inclusion
> > of other groups who have thought hard about these problems
> for a long
> > time (e.g., http://www.science.uva.nl/research/sne/ndl).
> We should at
> > least ask the question of whether we can get where we want to go in
> > some other way, e.g., by extending NDL.
> >
> > Jeff
> >
> >
> > _______________________________________________
> > services-wg mailing list
> > services-wg@...
> > http://lists.geni.net/mailman/listinfo/services-wg
> >
> >
>
> _______________________________________________
> services-wg mailing list
> services-wg@...
> http://lists.geni.net/mailman/listinfo/services-wg
>

_______________________________________________
services-wg mailing list
services-wg@...
http://lists.geni.net/mailman/listinfo/services-wg

[Robert P Ricci]

I've tried to capture some of the thoughts that have come out of this
discussion in a new draft of the RSpec document (0.5), which I've
attached to the wiki here:
http://groups.geni.net/geni/wiki/GeniRspec

Comments (especially in the form of proposed edits to the document!) are
invited.

Here's the relevant new text:

----------
3.1     Low-level Component Description

The purpose of the RSpec is to describe substrate resources. It
describes resources that are manipulated at the "GENI layer"
(ie. by the GMC API). It does not describe the configuration of
resources above that layer. Expressed in another way, RSpec is intended
to describe the set of resources that a GENI user must invoke privileged
operations (via GMC calls) to obtain or configure. Once given a sliver,
the user has the ability to configure and/or program that sliver within
the bounds of the component's capabilities and the slicing
technology used on the component. The core RSpec is concerned with
describing these bounds, which are outside of a user's direct
control, not the contents of the sliver, which are under the control of
the user.

As a simple, concrete example, consider a component that is a sliced PC.
The RSpec for this PC would include the details of the PC's
processor, memory, disk, etc. It would also give the level of
abstraction provided by the slicing mechanism on this component; for
example, this might be a hypervisor-based virtual machine, giving the
user a virtual x86 PC on which the user may install an operating system,
or it may be kernel-based virtualization giving the user a virtual Linux
machine. The core RSpec for this component would not attempt to describe
the set of operating systems or applications that may be run on the
component; it is up to the user (possibly, with the help of some service
providing a repository of "standard GENI software components")
to decide what they choose to do with a virtual x86 machine or a virtual
Linux PC. If the IP (or other protocol) stack on the component is not
fixed, but user-replaceable, it is not in the scope of the core RSpec.
----------
3.2.1   Standardization of RSpec Extensions

Entities contributing components and other resources to GENI will have
the flexibility to define their own extensions to RSpec, and to use
these extensions in a limited fashion without formal standardization.
Such non-standard extensions will be confined to a special namespace.
If an extension proves useful, and of interest to at least some subset
of the GENI community, it should go through a standardization process,
to be defined by the Control Framework Working Group, so that its
semantics and meaning are well-defined and well-understood, and it may
be used in an interoperable fashion. The result of this process with be
a "standard extension".

It is expected that, as components with new technologies are introduced
into GENI, the RSpec extensions used to describe them will begin as
"private" extensions used by the component developers and
operators, and become standardized as those components become suitable
and available for wider use.
----------
3.5     Limits to the Scope of RSpec

While RSpec is designed to be extensible, its purpose remains the
description, at a low level, of GENI substrate resources. This does not
preclude services that choose to operate at a higher level of
abstraction; we limit the RSpec to its original scope by allowing such
services to use their own resource descriptions.
The fundamental goal of the GMC API is interoperability between the
various parts of GENI. Thus, parameters to standard GMC API calls that
describe resources will all be given in RSpec. APIs, however, that
individual services choose to implement, but that are not part of the
GMC, may use RSpec, but are not required to do so. Slice embedding
services, for example, that wish to allow users to describe topologies
in ways that are higher-level and more flexible than those permitted by
RSpec may chose to do so using domain-specific languages. When they
interact with component managers or other GMC entities, however, the
will do so using RSpec.
----------

--
/-----------------------------------------------------------
| Robert P Ricci <ricci@...> | <ricci@...>
| Research Associate, University of Utah Flux Group
| www.flux.utah.edu | www.emulab.net
\-----------------------------------------------------------

_______________________________________________
services-wg mailing list
services-wg@...
http://lists.geni.net/mailman/listinfo/services-wg