For the record, I also agree with Ilia's comments.
Thus spake Aaron Falk on Fri, Feb 05, 2010 at 10:43:04AM -0500:
> > 8. The AMs communicate in pairwise fashion to agree on things like
> > VLAN#s for the interconnection point. They can do this because the
> > request RSpec that each AM sees includes 'external references' to the
> > other AMs in the topology.
> As I said to Max, I'm a little concerned about the complexity in the
> 'pairwise fashion' negotiation. See below.
> > 9. The AMs, using the negotiated VLAN#s, etc., individually create their
> > slivers and inform the user when they're ready.
> > We've implemented up to step 7. While 'pairwise fashion' in step 8
> > sounds like it would scale poorly, consider the fact that the number of
> > places where there are physical interconnects between various aggregates
> > are likely to be limited in number and fairly static, bounding the
> > number of pairs that have to agree. And, in fact, in the medium term,
> > what's likely to happen is that there are a couple of backbone AMs, with
> > aggregates hanging off of them. This results in a very simple, easy to
> > manage structure: backbones act as 'masters' for selecting VLAN#s for
> > their attached aggregates, the only place where real negotiation has to
> > occur is at connection points between backbones.
> How do you handle the situation when it's not obvious who the 'master'
> is? E.g., when you have multiple backbones in a slice (we already have
> 2)? What about when the two ends try to assign the same VLAN to
> different slices? Maybe I'm being unreasonably conservative. Can you
> describe the 'pairwise' protocol? Is there code?
For VLAN numbers, the 'negotiation' is basically no more complicated
than set intersection - each side indicates the set of VLANs it has
available for use, you intersect the sets and pick N VLAN #s from the
result. I don't think it matters which side 'drives' this process, since
it's trivial to verify that it was done correctly. Even for more
complicated setups where there are are more than two parties that have
to agree (such as several aggregates that might share some wireless
spectrum), this is still very straightforward.
Now, as you say, longer term, we would ideally like to be in a world
where both sides don't have to *agree* on the VLAN numbers (or more
generally, labels) to use, which of course basically gets rid of the
negotiation process completely - each side simply informs the other of
the label it is using, and one or both translates between the label it
receives 'on the wire' (or over the air) and the one it uses within the
aggregate. For the wired world, technologies such as VLAN translation,
q-in-q, and MPLS will all help with this.
> Can you comment on my point to Max about keeping one sliver from sending
> data out of the aggregate until the end to end slice is stitched. Do
> you think this is a concern?
I think that AMs should have the policy that if it receives a packet at
a 'touch point' with an unknown label (eg. VLAN tag #), it should
discard that packet. This is a good idea for a number of reasons,
especially security. I would be extremely surprised if an AM *didn't*
implement this policy; I would be unwilling to implement any other
policy. I think this addresses your concern: while a link is in a 'half
set up' state, one side will not be configured to accept the new label,
and traffic it receives will just be dropped, not disruptive.
> Finally, I suggest that the VLAN information gets logged in the
> clearinghouse. This seems like important operational information that
> will be needed to help diagnose faulty slices and to assign
> accountability for packets leaving an aggregate.
In our design, all logging is local to the aggregate. What we're
planning (but have not yet implemented), is that each AM *report* to the
clearninghouse to inform it about slices and slivers that exist on the
aggregate - this makes it straightforward to go around collecting the
relevant logs when you want them.