Gianny Damour wrote:
> Hi,
>
> I think that now that WADI can support catastrophic node failure and
> rehydrates a session state from a backing storage, we should try to
> define a default contextualiser stack and backend implementation. The
> goal being to review that this default configruation behaves as
> expected under various scenarios.
>
> Jules, could you please advise such a default stack? From what I
> understand, this is what I have in mind:
>
> Contextualiser stack:
> * MemoryContextualiser -> SerialContextualiser ->
> StatelessContextualiser -> ExclusiveStoreContextualiser ->
> ClusterContextualiser -> ReplicaAwareContextualiser ->
> DummyContextualiser
>
> This is basically the stack defined by wadi-web.xml with
> SharedStoreContextualiser replaced by ReplicaAwareContextualiser. I am
> not very keen to have this SharedStoreContextualiser because it relies
> on an external database and I think that users, who really want to
> store critical state in the session, should do it themself, i.e. in
> their application code.
>
So, lets discuss this stack in more detail - I'll add more detail than
we need, so that others who don't know all about the stack can follow
the thread...
Basically, WADI comprises two major areas :
the horizontal - all the clustering stuff that connects peers together,
allowing them the share and exchange state and responsibilities.
the vertical - a stack of Contextualisers, through which an incoming
Invocation descends, until it can be processed or relocated to another
node...
The Contextualiser stack is basically an extendable interceptor stack.
We can plug any Contextualiser we like into the stack... At the moment,
the usual WADI stack looks like this :
Memory - checks an in-memory cache, to see if the Session is located on
the local peer
Serial - takes a [per session] lock, ensuring that only one
thread-per-session may descend any further at any one time
Stateless - performs optimisations for known stateless requests
ExclusiveStore - looks in a local (i.e. not shared - exclusively owned)
'paging' store for the relevant Session
Cluster - checks the rest of the Cluster for the relevant Session
(relocation occur here) - the horizontal...
SharedStore - a remote (i.e. shared) 'persistant' store, in which
Sessions may live between Cluster incarnations.
Dummy - the end of the stack
As the Invocation descends from Memory, looking for its Session a
'Promoter' object is passed with it. If the Session is found, the
Promoter is used to promote the Session into Memory and render the
Invocation there.
With hindsight, I want to revisit the Exclusive and SharedStore
Contextualisers... Currently, an Evicter in the MemoryContextualiser is
run periodically and may evict Sessions down the stack into the
ExclusiveStore (generally a cache on local disc). Sessions may be
reloaded from this cache back up into Memory (promotion), if an
Invocation for them descends the stack. The SharedStoreContextualiser
should only (bug here at the moment) be used as the first peer joins a
Cluster, or the last peer leaves a Cluster, to load Sessions from the
last Cluster incarntion, or persist them until the next...
Issues:
The Exclusive store is a Single Point of Failure (SPoF). If the peer
that owns it should die, all Sessions held by this Contextualiser become
unavailable. We could replicate the Sessions held exclusively to avoid
this, but this just makes our architecture even more complicated.
The Shared store may contain a very large number of Sessions which
suddenly become the responsibility of the first Cluster member. It must
promote all of them up its stack (generally into exclusive store, since
there may be too many to hold in memory), before starting. Whilst they
sit in this exclusively owned store, all these Sessions are vulnerable
to the failure of the first peer (see above).
Solution :
I have given this some though and come up with a solution which I think
resolves all issues and fits nicely into WADI... The idea is to collapse
the Exclusive and Shared stores into a single partitioned Shared store.
By sharing the exclusive store, we ensure that it is no longer an SPoF.
By partitioning it, we overcome the issues associated with a number of
peers all reading/writing sessions to a common resource. Store
partitions would be negotiated along with Location partitions, so that
if a node that owned e.g. a couple of directories worth of the shared
disc paging store should die, other peers could take over responsibility
for these dirs and ensure the continued availability of their contents.
By having paging and persistant stores in the same place, we can
completely avoid having the SharedStoreContextualiser at the bottom of
the stack. Sessions persisted (paged out) at the end of the last Cluster
are already sitting in the paging store when the next Cluster starts
up.... etc....
The only downside that I can see with this approach is that to page-out
a Session, it may have to actually be migrated from the peer holding it,
to the peer that owns the relevant partition of the shared store - as
this peer is necessarily the only one allowed to r/w this section of the
store. Similarly, paging-in a Session may involve remote interaction
with the same peer. I think that the gain (removal of an SPoF and
simplification of the stack) outweighs the cost (increased migration)
and since Session paging frequency may be controlled via tuning of the
eviction strategy, the deployment owner does have some control over
this... Furthermore, as we become more sophisticated, Sessions will
usually be born, live and die on the node which manages their partition,
largely removing this associated cost.
Does anyone else have any thoughts about this ? Does it sound like a
sensible way forward ?
Jules
> From a backend perspective, my order of preference is Tribes and then
> AC. I tend to prefer Tribes as we have Tribes people contributing to
> WADI.
>
> Thanks,
> Gianny
>
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
