DiskPageStore improvements for 1.3.1

[Matej Knopp]

Hi all,

I've committed couple of changes to DiskPageStore to address some
issues. Mind you the commit is rather fresh and needs to be tested
properly before 1.3.1 gets out.

First of all it solves the problem of context reloads (see
https://issues.apache.org/jira/browse/WICKET-1161). When context is
reloaded, the DiskPageStore index is no longer lost and all serialized
page files can be used as if no reload has happened.

The other improvement is better clustering support
(https://issues.apache.org/jira/browse/WICKET-1272). The back button
support is improved in clustered environment and also the memory
consumption is reduced significantly.

What's new? Imagine a simplified environment, cluster with two nodes
(NodeA and NodeB). All sessions from a NodeA are replicated to NodeB
and vice versa.

When a page instance is being replicated from NodeA to NodeB, it is
immediately stored to DiskPageStore on nodeB, rather than kept in
session. This means that the instance is later accessible on NodeB
even after another page from the same pagemap has been replicated to
NodeB, because it's already stored in DiskPageStore. Also it doesn't
have to be kept in memory, significantly reducing the session state.


There is one exception though, where the backbutton won't work across
cluster. Imagine following scenario:

Session is created to NodeA
NodeB goes down
User visits page1, page2, page3 on NodeA  (these pages were not
replicated to NodeB, because the node is not available)
NodeB goes up

only page3 (the last accessed page in it's pagemap) will be available
on NodeB. page1 and page2 are not available, because when they were
visited NodeB was not running. page3 is available, because it was
active in nodeA session, thus got replicated to NodeB when the context
on NodeB was initialized.

-Matej

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[igor.vaynberg]

On Jan 9, 2008 5:09 AM, Matej Knopp <matej.knopp@...> wrote:

>
> What's new? Imagine a simplified environment, cluster with two nodes
> (NodeA and NodeB). All sessions from a NodeA are replicated to NodeB
> and vice versa.
>
> When a page instance is being replicated from NodeA to NodeB, it is
> immediately stored to DiskPageStore on nodeB, rather than kept in
> session. This means that the instance is later accessible on NodeB
> even after another page from the same pagemap has been replicated to
> NodeB, because it's already stored in DiskPageStore. Also it doesn't
> have to be kept in memory, significantly reducing the session state.

sweet

> There is one exception though, where the backbutton won't work across
> cluster. Imagine following scenario:

this is ok. thats how all clusters work if you only use a single node
as a backup buddy.

-igor

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Doug Leeper]

Besides get the latest source from SVN...is there anything I need to do to utilize this?  

BTW...I am using the default session store impl from WebApplication.

[Jonathan Locke]

Ideally, NodeB ought to "catch up" when it joins the cluster by
syncing the content in the disk page stores at that time.  I think
it's a flaw we should track and fix at some point.

Nice job, Matej!

Matej Knopp-2 wrote:

Hi all,

I've committed couple of changes to DiskPageStore to address some
issues. Mind you the commit is rather fresh and needs to be tested
properly before 1.3.1 gets out.

First of all it solves the problem of context reloads (see
https://issues.apache.org/jira/browse/WICKET-1161). When context is
reloaded, the DiskPageStore index is no longer lost and all serialized
page files can be used as if no reload has happened.

The other improvement is better clustering support
(https://issues.apache.org/jira/browse/WICKET-1272). The back button
support is improved in clustered environment and also the memory
consumption is reduced significantly.

What's new? Imagine a simplified environment, cluster with two nodes
(NodeA and NodeB). All sessions from a NodeA are replicated to NodeB
and vice versa.

When a page instance is being replicated from NodeA to NodeB, it is
immediately stored to DiskPageStore on nodeB, rather than kept in
session. This means that the instance is later accessible on NodeB
even after another page from the same pagemap has been replicated to
NodeB, because it's already stored in DiskPageStore. Also it doesn't
have to be kept in memory, significantly reducing the session state.


There is one exception though, where the backbutton won't work across
cluster. Imagine following scenario:

Session is created to NodeA
NodeB goes down
User visits page1, page2, page3 on NodeA  (these pages were not
replicated to NodeB, because the node is not available)
NodeB goes up

only page3 (the last accessed page in it's pagemap) will be available
on NodeB. page1 and page2 are not available, because when they were
visited NodeB was not running. page3 is available, because it was
active in nodeA session, thus got replicated to NodeB when the context
on NodeB was initialized.

-Matej

---------------------------------------------------------------------
To unsubscribe, e-mail: [e-mail]
For additional commands, e-mail: [e-mail]

[igor.vaynberg]

this cant be accomplished transparently. we would need our own network
connection, node discovery, blah blah blah. loses a bit of appeal.

-igor


On Jan 9, 2008 10:24 PM, Jonathan Locke <jonathan.locke@...> wrote:

>
>
> Ideally, NodeB ought to "catch up" when it joins the cluster by
> syncing the content in the disk page stores at that time.  I think
> it's a flaw we should track and fix at some point.
>
> Nice job, Matej!
>
>
>
> Matej Knopp-2 wrote:
> >
> > Hi all,
> >
> > I've committed couple of changes to DiskPageStore to address some
> > issues. Mind you the commit is rather fresh and needs to be tested
> > properly before 1.3.1 gets out.
> >
> > First of all it solves the problem of context reloads (see
> > https://issues.apache.org/jira/browse/WICKET-1161). When context is
> > reloaded, the DiskPageStore index is no longer lost and all serialized
> > page files can be used as if no reload has happened.
> >
> > The other improvement is better clustering support
> > (https://issues.apache.org/jira/browse/WICKET-1272). The back button
> > support is improved in clustered environment and also the memory
> > consumption is reduced significantly.
> >
> > What's new? Imagine a simplified environment, cluster with two nodes
> > (NodeA and NodeB). All sessions from a NodeA are replicated to NodeB
> > and vice versa.
> >
> > When a page instance is being replicated from NodeA to NodeB, it is
> > immediately stored to DiskPageStore on nodeB, rather than kept in
> > session. This means that the instance is later accessible on NodeB
> > even after another page from the same pagemap has been replicated to
> > NodeB, because it's already stored in DiskPageStore. Also it doesn't
> > have to be kept in memory, significantly reducing the session state.
> >
> >
> > There is one exception though, where the backbutton won't work across
> > cluster. Imagine following scenario:
> >
> > Session is created to NodeA
> > NodeB goes down
> > User visits page1, page2, page3 on NodeA  (these pages were not
> > replicated to NodeB, because the node is not available)
> > NodeB goes up
> >
> > only page3 (the last accessed page in it's pagemap) will be available
> > on NodeB. page1 and page2 are not available, because when they were
> > visited NodeB was not running. page3 is available, because it was
> > active in nodeA session, thus got replicated to NodeB when the context
> > on NodeB was initialized.
> >
> > -Matej
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@...
> > For additional commands, e-mail: users-help@...
> >
> >
> >
>
> --
> View this message in context: http://www.nabble.com/DiskPageStore-improvements-for-1.3.1-tp14711582p14728308.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

["C. Bergström"]

> > Matej Knopp-2 wrote:
> > >
> > > Hi all,
> > >
> > > I've committed couple of changes to DiskPageStore to address some
> > > issues. Mind you the commit is rather fresh and needs to be tested
> > > properly before 1.3.1 gets out.
> > >
> > > First of all it solves the problem of context reloads (see
> > > https://issues.apache.org/jira/browse/WICKET-1161). When context is
> > > reloaded, the DiskPageStore index is no longer lost and all serialized
> > > page files can be used as if no reload has happened.
> > >
> > > The other improvement is better clustering support
> > > (https://issues.apache.org/jira/browse/WICKET-1272). The back button
> > > support is improved in clustered environment and also the memory
> > > consumption is reduced significantly.
> > >
> > > What's new? Imagine a simplified environment, cluster with two nodes
> > > (NodeA and NodeB). All sessions from a NodeA are replicated to NodeB
> > > and vice versa.
> > >
> > > When a page instance is being replicated from NodeA to NodeB, it is
> > > immediately stored to DiskPageStore on nodeB, rather than kept in
> > > session. This means that the instance is later accessible on NodeB
> > > even after another page from the same pagemap has been replicated to
> > > NodeB, because it's already stored in DiskPageStore. Also it doesn't
> > > have to be kept in memory, significantly reducing the session state.
> > >
> > >
> > > There is one exception though, where the backbutton won't work across
> > > cluster. Imagine following scenario:
> > >
> > > Session is created to NodeA
> > > NodeB goes down
> > > User visits page1, page2, page3 on NodeA  (these pages were not
> > > replicated to NodeB, because the node is not available)
> > > NodeB goes up
> > >
> > > only page3 (the last accessed page in it's pagemap) will be available
> > > on NodeB. page1 and page2 are not available, because when they were
> > > visited NodeB was not running. page3 is available, because it was
> > > active in nodeA session, thus got replicated to NodeB when the context
> > > on NodeB was initialized.

>
> On Jan 9, 2008 10:24 PM, Jonathan Locke <jonathan.locke@...> wrote:
> >
> >
> > Ideally, NodeB ought to "catch up" when it joins the cluster by
> > syncing the content in the disk page stores at that time.  I think
> > it's a flaw we should track and fix at some point.
> >
> > Nice job, Matej!
>

On Wed, 2008-01-09 at 22:35 -0800, Igor Vaynberg wrote:
> this cant be accomplished transparently. we would need our own network
> connection, node discovery, blah blah blah. loses a bit of appeal.
>
> -igor
>
>

I'm intending to stir the pot a bit, but why are we kicking this further?  Igor is absolutely right..  For something simple, currently in the linux kernel try

drbd [1]

1) Nodes can catch up/sync
2) It'll be faster than anything you can do inside java
3) transparent to the application
4) debugged and proven to work inside a production env
5) means you can revert your changes and we don't have to worry/test anything
-----------------------------------

To be honest you may be on the right path long term and I don't currently have a better suggestion, but we are really reinventing the wheel here which leads me to be highly cautious.

Thanks

./C


[1] http://www.drbd.org/


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Matej Knopp]

> > > Ideally, NodeB ought to "catch up" when it joins the cluster by
> > > syncing the content in the disk page stores at that time.  I think
> > > it's a flaw we should track and fix at some point.
> > >
> > > Nice job, Matej!
> >
Well, I'm afraid as Igor said this is not possible to do without
having container specific code. So when currently a new node catch up
it only gets the last page for each pagemap.

> On Wed, 2008-01-09 at 22:35 -0800, Igor Vaynberg wrote:
> > this cant be accomplished transparently. we would need our own network
> > connection, node discovery, blah blah blah. loses a bit of appeal.
Yeah. This is actually not that difficult to do, there are frameworks
such as tribes that do this quite nicely. The main problem is that we
don't know the topology. We don't know which node is backed by which
node, so we would do lot of unnecearry copying. I don't see any easy
way to implement this.

> I'm intending to stir the pot a bit, but why are we kicking this further?  Igor is absolutely right..  For something simple, currently in the linux kernel try
>
> drbd [1]
>
> 1) Nodes can catch up/sync
> 2) It'll be faster than anything you can do inside java
> 3) transparent to the application
> 4) debugged and proven to work inside a production env
> 5) means you can revert your changes and we don't have to worry/test anything
> -----------------------------------
>
> To be honest you may be on the right path long term and I don't currently have a better suggestion, but we are really reinventing the wheel here which leads me to be highly cautious.

Reinventing the wheels? How does drbd handle the topology. Does it
know which nodes are backed by which node? Usually in more
sophisticated environment each node in cluster is backed by one other
node forming a ring and the container can be in charge of forming the
topology. How does drbd handle scenario like that?

-Matej

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

["C. Bergström"]

On Thu, 2008-01-10 at 13:35 +0100, Matej Knopp wrote:

> > > > Ideally, NodeB ought to "catch up" when it joins the cluster by
> > > > syncing the content in the disk page stores at that time.  I
> think
> > > > it's a flaw we should track and fix at some point.
> > > >
> > > > Nice job, Matej!
> > >
> Well, I'm afraid as Igor said this is not possible to do without
> having container specific code. So when currently a new node catch up
> it only gets the last page for each pagemap.
>
> > On Wed, 2008-01-09 at 22:35 -0800, Igor Vaynberg wrote:
> > > this cant be accomplished transparently. we would need our own
> network
> > > connection, node discovery, blah blah blah. loses a bit of appeal.
> Yeah. This is actually not that difficult to do, there are frameworks
> such as tribes that do this quite nicely. The main problem is that we
> don't know the topology. We don't know which node is backed by which
> node, so we would do lot of unnecearry copying. I don't see any easy
> way to implement this.
>
> > I'm intending to stir the pot a bit, but why are we kicking this further?  Igor is absolutely right..  For something simple, currently in the linux kernel try
> >
> > drbd [1]
> >
> > 1) Nodes can catch up/sync
> > 2) It'll be faster than anything you can do inside java
> > 3) transparent to the application
> > 4) debugged and proven to work inside a production env
> > 5) means you can revert your changes and we don't have to worry/test anything
> > -----------------------------------
> >
> > To be honest you may be on the right path long term and I don't currently have a better suggestion, but we are really reinventing the wheel here which leads me to be highly cautious.
>
> Reinventing the wheels? How does drbd handle the topology. Does it
> know which nodes are backed by which node? Usually in more
> sophisticated environment each node in cluster is backed by one other
> node forming a ring and the container can be in charge of forming the
> topology. How does drbd handle scenario like that?

I'm being lazy and uncreative in my response.. So here are a few quotes
from the site..

"DRBD is a block device which is designed to build high availability
clusters. This is done by mirroring a whole block device via (a
dedicated) network. You could see it as a network raid-1."

"Each device (DRBD provides more than one of these devices) has a state,
which can be 'primary' or 'secondary'. "

I understand your description and I'm wondering what you're using for
the fencing/quorum device or monitoring if you're handling this entirely
in java.  Lets be honest that adding transparent clustering support to
wicket will be a multi-phased roll-out.

Stable single node under duress
two node cluster
N node.. and then increase by a factor of 10 working out bottlenecks as
we go..

Focusing on just the two-node HA use case I think is a first step.  For
me this is especially important in that I primarily work with ASP/SAAS
companies where maintaining a quality SLA and a single point of failure
just isn't reasonable.  (The way wicket lets you roll out complex
applications quickly makes it a sweet spot in terms of frameworks for
them.)

All with a grain of salt..

./C


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Matej Knopp]

On Jan 12, 2008 1:04 PM, C. Bergström <cbergstrom@...> wrote:

>
>
> On Thu, 2008-01-10 at 13:35 +0100, Matej Knopp wrote:
> > > > > Ideally, NodeB ought to "catch up" when it joins the cluster by
> > > > > syncing the content in the disk page stores at that time.  I
> > think
> > > > > it's a flaw we should track and fix at some point.
> > > > >
> > > > > Nice job, Matej!
> > > >
> > Well, I'm afraid as Igor said this is not possible to do without
> > having container specific code. So when currently a new node catch up
> > it only gets the last page for each pagemap.
> >
> > > On Wed, 2008-01-09 at 22:35 -0800, Igor Vaynberg wrote:
> > > > this cant be accomplished transparently. we would need our own
> > network
> > > > connection, node discovery, blah blah blah. loses a bit of appeal.
> > Yeah. This is actually not that difficult to do, there are frameworks
> > such as tribes that do this quite nicely. The main problem is that we
> > don't know the topology. We don't know which node is backed by which
> > node, so we would do lot of unnecearry copying. I don't see any easy
> > way to implement this.
> >
> > > I'm intending to stir the pot a bit, but why are we kicking this further?  Igor is absolutely right..  For something simple, currently in the linux kernel try
> > >
> > > drbd [1]
> > >
> > > 1) Nodes can catch up/sync
> > > 2) It'll be faster than anything you can do inside java
> > > 3) transparent to the application
> > > 4) debugged and proven to work inside a production env
> > > 5) means you can revert your changes and we don't have to worry/test anything
> > > -----------------------------------
> > >
> > > To be honest you may be on the right path long term and I don't currently have a better suggestion, but we are really reinventing the wheel here which leads me to be highly cautious.
> >
> > Reinventing the wheels? How does drbd handle the topology. Does it
> > know which nodes are backed by which node? Usually in more
> > sophisticated environment each node in cluster is backed by one other
> > node forming a ring and the container can be in charge of forming the
> > topology. How does drbd handle scenario like that?
>

The thing is, we don't really care what the topology is. The only
requirement is that the session gets immediately deserialized once it
is replicated to other node, which should be either default or at
least configurable on most containers.

So we let the container manage the replication topology. All we need
is to be notified when the session is replicated (pagemap
serialization and deserialization). All other approaches would require
container specific code.

-Matej

> I'm being lazy and uncreative in my response.. So here are a few quotes
> from the site..
>
> "DRBD is a block device which is designed to build high availability
> clusters. This is done by mirroring a whole block device via (a
> dedicated) network. You could see it as a network raid-1."
>
> "Each device (DRBD provides more than one of these devices) has a state,
> which can be 'primary' or 'secondary'. "
>
> I understand your description and I'm wondering what you're using for
> the fencing/quorum device or monitoring if you're handling this entirely
> in java.  Lets be honest that adding transparent clustering support to
> wicket will be a multi-phased roll-out.
>
> Stable single node under duress
> two node cluster
> N node.. and then increase by a factor of 10 working out bottlenecks as
> we go..
>
> Focusing on just the two-node HA use case I think is a first step.  For
> me this is especially important in that I primarily work with ASP/SAAS
> companies where maintaining a quality SLA and a single point of failure
> just isn't reasonable.  (The way wicket lets you roll out complex
> applications quickly makes it a sweet spot in terms of frameworks for
> them.)
>
> All with a grain of salt..
>
> ./C
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

--
Resizable and reorderable grid components.
http://www.inmethod.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

["C. Bergström"]

On Sat, 2008-01-12 at 14:12 +0100, Matej Knopp wrote:

> On Jan 12, 2008 1:04 PM, C. Bergström <cbergstrom@...> wrote:
> >
> >
> > On Thu, 2008-01-10 at 13:35 +0100, Matej Knopp wrote:
> > > > > > Ideally, NodeB ought to "catch up" when it joins the cluster by
> > > > > > syncing the content in the disk page stores at that time.  I
> > > think
> > > > > > it's a flaw we should track and fix at some point.
> > > > > >
> > > > > > Nice job, Matej!
> > > > >
> > > Well, I'm afraid as Igor said this is not possible to do without
> > > having container specific code. So when currently a new node catch up
> > > it only gets the last page for each pagemap.
> > >
> > > > On Wed, 2008-01-09 at 22:35 -0800, Igor Vaynberg wrote:
> > > > > this cant be accomplished transparently. we would need our own
> > > network
> > > > > connection, node discovery, blah blah blah. loses a bit of appeal.
> > > Yeah. This is actually not that difficult to do, there are frameworks
> > > such as tribes that do this quite nicely. The main problem is that we
> > > don't know the topology. We don't know which node is backed by which
> > > node, so we would do lot of unnecearry copying. I don't see any easy
> > > way to implement this.
> > >
> > > > I'm intending to stir the pot a bit, but why are we kicking this further?  Igor is absolutely right..  For something simple, currently in the linux kernel try
> > > >
> > > > drbd [1]
> > > >
> > > > 1) Nodes can catch up/sync
> > > > 2) It'll be faster than anything you can do inside java
> > > > 3) transparent to the application
> > > > 4) debugged and proven to work inside a production env
> > > > 5) means you can revert your changes and we don't have to worry/test anything
> > > > -----------------------------------
> > > >
> > > > To be honest you may be on the right path long term and I don't currently have a better suggestion, but we are really reinventing the wheel here which leads me to be highly cautious.
> > >
> > > Reinventing the wheels? How does drbd handle the topology. Does it
> > > know which nodes are backed by which node? Usually in more
> > > sophisticated environment each node in cluster is backed by one other
> > > node forming a ring and the container can be in charge of forming the
> > > topology. How does drbd handle scenario like that?
> >
>
> The thing is, we don't really care what the topology is. The only
> requirement is that the session gets immediately deserialized once it
> is replicated to other node, which should be either default or at
> least configurable on most containers.
>
> So we let the container manage the replication topology. All we need
> is to be notified when the session is replicated (pagemap
> serialization and deserialization). All other approaches would require
> container specific code.

Yeah.. well. when node A fails over to node B and user clicks the back
button is the end goal to handle that gracefully.  I was just suggesting
a trivial way to accomplish the replication of the DiskPageStore's
actual files.  This works around app container specifics for this point.

Also you realize you responded to your own question asking about
topology..

./C


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Matej Knopp]

> >
> > The thing is, we don't really care what the topology is. The only
> > requirement is that the session gets immediately deserialized once it
> > is replicated to other node, which should be either default or at
> > least configurable on most containers.
> >
> > So we let the container manage the replication topology. All we need
> > is to be notified when the session is replicated (pagemap
> > serialization and deserialization). All other approaches would require
> > container specific code.
>
> Yeah.. well. when node A fails over to node B and user clicks the back
> button is the end goal to handle that gracefully.  I was just suggesting
> a trivial way to accomplish the replication of the DiskPageStore's
> actual files.  This works around app container specifics for this point.
>
> Also you realize you responded to your own question asking about
> topology..

I do :), but you don't seem to quite realize the importance of knowing
the topology.

First of all, one DiskPageStore location was never supposed to be
shared across nodes and it will never work correctly that way. Each
node must have separate location where it's DiskPageStore stores the
files. Otherwise the result is pretty much unpredicable. This
basically rules out usage of replicated file system. Unless you
implement your own store probably with one file per page. We had such
implementation, the problem was that the performance didn't scale well
under load (lot of small writes).

As for the topology. In order to scale well with growing number of
nodes each node should be ideally backed by one other node.
Applications servers do this. The advantage of current DiskPageStore
implementation is that the DPS files are replicated to the exact node
that backs the session. So they are not just replicated all over the
cluster, which certainly wouldn't scale well.

-Matej

>
>
> ./C
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

--
Resizable and reorderable grid components.
http://www.inmethod.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...