Food Tasting, Memory and Rolling Back Transactions.

[Ross Rick]

As I understand it, food tasting keeps a complete duplicate model set  
in memory and runs the transaction against the duplicate first to  
ensure that it is safe.

1.   Is that correct?

2.   If it is correct then is it correct to presume that I can  
effectively double the amount of memory available to storage by  
turning it off.

3.   If I turn it off, what are the ramifications?    It seems  
possible that rolling back a transaction would then be my  
responibility.   Is that correct?



Finally,  closing the Prevayler, this should release all memory, right?



Thanks,


Rick






------------------------------------------------------------------------------
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[William Pietri]

Hi, Rick.

Ross Rick wrote:
> As I understand it, food tasting keeps a complete duplicate model set  
> in memory and runs the transaction against the duplicate first to  
> ensure that it is safe.
>
> 1.   Is that correct?
>  

That's my understanding.

> 2.   If it is correct then is it correct to presume that I can  
> effectively double the amount of memory available to storage by  
> turning it off.
>  

That is correct.
> 3.   If I turn it off, what are the ramifications?    It seems  
> possible that rolling back a transaction would then be my  
> responibility.   Is that correct?
>  

The risk is that a transaction blows up in the middle, leaving your
model in an inconsistent state, one that might not be what you'd get
from replaying the log. I think of that less as being responsible for
rolling back, and more for being responsible that your transactions
never, ever break. But perhaps it amounts to the same thing.

> Finally,  closing the Prevayler, this should release all memory, right?

Subject to all the joys of the JVM, that's my understanding. I haven't
verified that myself, though, as my Prevayler instances tend to be
long-lived.

William

------------------------------------------------------------------------------
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[Ross Rick]

William,

Thanks.  I appreciate the response.   I have inserted a comment below.

R

On Jul 25, 2009, at 5:02 PM, William Pietri wrote:

> Hi, Rick.
>
> Ross Rick wrote:
>> As I understand it, food tasting keeps a complete duplicate model set
>> in memory and runs the transaction against the duplicate first to
>> ensure that it is safe.
>>
>> 1.   Is that correct?
>>
>
> That's my understanding.
>
>> 2.   If it is correct then is it correct to presume that I can
>> effectively double the amount of memory available to storage by
>> turning it off.
>>
>
> That is correct.
>> 3.   If I turn it off, what are the ramifications?    It seems
>> possible that rolling back a transaction would then be my
>> responibility.   Is that correct?
>>
>
> The risk is that a transaction blows up in the middle, leaving your
> model in an inconsistent state, one that might not be what you'd get
> from replaying the log. I think of that less as being responsible for
> rolling back, and more for being responsible that your transactions
> never, ever break. But perhaps it amounts to the same thing.

I am perfectly fine with testing everything I can before executing the  
transaction.  My issue would be with the things that are somewhat out  
of my control, like say, memory allocation.   I suppose in that  
situation I would have to force the user to quit anyway, and only be  
losing the last transaction, but I'm not so sure that's gonna work.

In my case, a single delete can cause a number of cascading deletes,  
with additional reassignments, for example a delete of a tree node  
could cause leaves to be removed as well plus branches  to be moved to  
the parent of the deleted node

So I doubt my ability to accurately to correctly roll back.   In that  
case, as rare as one hopes they are, a transaction roll back is really  
just an app restart or damn near, if I turn off food tasting.    I'm  
not complaining, I am just trying to be sure that I understand.

>
>> Finally,  closing the Prevayler, this should release all memory,  
>> right?
>
> Subject to all the joys of the JVM, that's my understanding. I haven't
> verified that myself, though, as my Prevayler instances tend to be
> long-lived.
>
> William
>
> ------------------------------------------------------------------------------
> _______________________________________________
> To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org

------------------------------------------------------------------------------
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[Justin T. Sampson-2]

On Sat, Jul 25, 2009 at 10:30 PM, Ross Rick <rick@...> wrote:

> On Jul 25, 2009, at 5:02 PM, William Pietri wrote:
>
> > > 3.   If I turn it off, what are the ramifications?    It seems
> > > possible that rolling back a transaction would then be my
> > > responibility.   Is that correct?
> >
> > The risk is that a transaction blows up in the middle, leaving your
> > model in an inconsistent state, one that might not be what you'd get
> > from replaying the log. I think of that less as being responsible for
> > rolling back, and more for being responsible that your transactions
> > never, ever break. But perhaps it amounts to the same thing.
>
> I am perfectly fine with testing everything I can before executing the
> transaction. My issue would be with the things that are somewhat out
> of my control, like say, memory allocation.   I suppose in that
> situation I would have to force the user to quit anyway, and only be
> losing the last transaction, but I'm not so sure that's gonna work.

You guys have actually described three very different issues, only one
of which the food taster helps with. (By the way, you may notice that
Prevayler purists like me are fairly anti-food-taster!)

1. Remember that all of Prevayler, including the food taster, assumes
that your code is deterministic. Even if a RuntimeException is thrown,
Prevayler assumes that it was thrown deterministically. The exception
is propagated on up to the client code, and Prevayler goes right on
executing further transactions.

If you have a transaction that fails nondeterministically, it could
just as well succeed when run against the food taster and then fail
against the king, or fail later when replayed from the journal. It's
the nondeterminism that causes the inconsistency, not the exception.

Conversely, if a transaction fails deterministically, it will always
fail, whether against the food taster, against the king, or later when
replayed from the journal. Therefore, eliminating the food taster does
not introduce any particular inconsistency: Whatever state the system
is in after the exception is thrown, that state will be faithfully
reproduced when replaying the journal, because the same exception will
be thrown (and ignored, at that point).

2. Note that Prevayler does treat Errors specially. Basically, Errors
(such as OutOfMemoryError) are assumed to be nondeterministic. If an
Error is thrown while executing a transaction, Prevayler will refuse
to process any further transactions and queries, to prevent an
inconsistent state from being observed. (Any memory held by Prevayler
will be released at that point.)

The only way to recover is to restart, or at least close that
Prevayler instance and create a new one. The failed transaction will
be replayed along with the rest of the journal.

Of course, the food taster doesn't prevent that situation, and
actually exacerbates it by using twice as much memory. :)

3. I concur with William's point that the goal is really correctness,
and rollback is just one tool to help achieve correctness -- as long
as you understand its limits.

Ideally, each object should maintain its own internal consistency even
when a RuntimeException is thrown. In fact, I think a major reason to
throw a RuntimeException is precisely to maintain consistency: If you
try to add a DOM Document as a child of a DOM Element, it's not going
to add the Document as a child and *then* throw a DOMException to
*indicate* that the Element is now in an invalid state, it's going to
throw a DOMException right away to *prevent* putting the Element into
an invalid state.

A rollback mechanism such as the food taster only helps you defend
against a very narrow class of bugs: *Deterministic* bugs that put the
system in an *inconsistent state* AND cause a RuntimeException to be
thrown from the *same* Transaction that introduced the inconsistent
state. Any other bug will slip by unnoticed. (This is not unique to
Prevayler!)

Cheers,
Justin

------------------------------------------------------------------------------
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[Ross Rick]

Ok.   I am following, and I am on board.  I will probably kill my  
taster too.

I will do everything possible to ensure that no changes to the object  
model are made until I have assured myself that all preconditions are  
met.  So my code should never throw an exception out of the  
transaction *after* changes have begun.

So inside my executes, it would look something like this :

         // check everything
         .....

         // Do changes
         .....

So yes, I am ok with keeping the model safe from deterministic  
errors.  My concern is an error beyond my control after the model has  
been modified in the transaction.   As I understand it Prevayler will  
not set the model back to the state it was in before the transaction  
started.  I can't imagine how it could.    And now reading this I  
understand that this is independent of food tasting.  Of course, the  
transaction will not be serialized to disk so I can reload the model  
to the correct state.  Since we are in an exception state I would be  
unlikely to be able to reverse the changes made to the model anyway,  
so a reload is really required in all cases.

Unless I've made a mistake above, I think I am good to go.   Thanks  
for taking the time to drive the points home.

R




On Jul 27, 2009, at 12:41 PM, Justin T. Sampson wrote:

> On Sat, Jul 25, 2009 at 10:30 PM, Ross Rick <rick@...> wrote:
>
>> On Jul 25, 2009, at 5:02 PM, William Pietri wrote:
>>
>>>> 3.   If I turn it off, what are the ramifications?    It seems
>>>> possible that rolling back a transaction would then be my
>>>> responibility.   Is that correct?
>>>
>>> The risk is that a transaction blows up in the middle, leaving your
>>> model in an inconsistent state, one that might not be what you'd get
>>> from replaying the log. I think of that less as being responsible  
>>> for
>>> rolling back, and more for being responsible that your transactions
>>> never, ever break. But perhaps it amounts to the same thing.
>>
>> I am perfectly fine with testing everything I can before executing  
>> the
>> transaction. My issue would be with the things that are somewhat out
>> of my control, like say, memory allocation.   I suppose in that
>> situation I would have to force the user to quit anyway, and only be
>> losing the last transaction, but I'm not so sure that's gonna work.
>
> You guys have actually described three very different issues, only one
> of which the food taster helps with. (By the way, you may notice that
> Prevayler purists like me are fairly anti-food-taster!)
>
> 1. Remember that all of Prevayler, including the food taster, assumes
> that your code is deterministic. Even if a RuntimeException is thrown,
> Prevayler assumes that it was thrown deterministically. The exception
> is propagated on up to the client code, and Prevayler goes right on
> executing further transactions.
>
> If you have a transaction that fails nondeterministically, it could
> just as well succeed when run against the food taster and then fail
> against the king, or fail later when replayed from the journal. It's
> the nondeterminism that causes the inconsistency, not the exception.
>
> Conversely, if a transaction fails deterministically, it will always
> fail, whether against the food taster, against the king, or later when
> replayed from the journal. Therefore, eliminating the food taster does
> not introduce any particular inconsistency: Whatever state the system
> is in after the exception is thrown, that state will be faithfully
> reproduced when replaying the journal, because the same exception will
> be thrown (and ignored, at that point).
>
> 2. Note that Prevayler does treat Errors specially. Basically, Errors
> (such as OutOfMemoryError) are assumed to be nondeterministic. If an
> Error is thrown while executing a transaction, Prevayler will refuse
> to process any further transactions and queries, to prevent an
> inconsistent state from being observed. (Any memory held by Prevayler
> will be released at that point.)
>
> The only way to recover is to restart, or at least close that
> Prevayler instance and create a new one. The failed transaction will
> be replayed along with the rest of the journal.
>
> Of course, the food taster doesn't prevent that situation, and
> actually exacerbates it by using twice as much memory. :)
>
> 3. I concur with William's point that the goal is really correctness,
> and rollback is just one tool to help achieve correctness -- as long
> as you understand its limits.
>
> Ideally, each object should maintain its own internal consistency even
> when a RuntimeException is thrown. In fact, I think a major reason to
> throw a RuntimeException is precisely to maintain consistency: If you
> try to add a DOM Document as a child of a DOM Element, it's not going
> to add the Document as a child and *then* throw a DOMException to
> *indicate* that the Element is now in an invalid state, it's going to
> throw a DOMException right away to *prevent* putting the Element into
> an invalid state.
>
> A rollback mechanism such as the food taster only helps you defend
> against a very narrow class of bugs: *Deterministic* bugs that put the
> system in an *inconsistent state* AND cause a RuntimeException to be
> thrown from the *same* Transaction that introduced the inconsistent
> state. Any other bug will slip by unnoticed. (This is not unique to
> Prevayler!)
>
> Cheers,
> Justin
>
> ------------------------------------------------------------------------------
> _______________________________________________
> To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>

------------------------------------------------------------------------------
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[Klaus Wuestefeld]

Hi Justin,

These two things you wrote below seem sort of contradictory:

> Conversely, if a transaction fails deterministically, it will always
> fail, whether against the food taster, against the king, or later when
> replayed from the journal. Therefore, eliminating the food taster does
> not introduce any particular inconsistency
...
> A rollback mechanism such as the food taster only helps you defend
> against a very narrow class of bugs: *Deterministic* bugs that put the
> system in an *inconsistent state* AND cause a RuntimeException to be
> thrown from the *same* Transaction that introduced the inconsistent
> state.

See you, Klaus.

------------------------------------------------------------------------------
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[Justin T. Sampson-2]

On Mon, Jul 27, 2009 at 1:26 PM, Ross Rick<rick@...> wrote:

> So yes, I am ok with keeping the model safe from deterministic
> errors.  My concern is an error beyond my control after the model has
> been modified in the transaction.   As I understand it Prevayler will
> not set the model back to the state it was in before the transaction
> started.  I can't imagine how it could.    And now reading this I
> understand that this is independent of food tasting.  Of course, the
> transaction will not be serialized to disk so I can reload the model
> to the correct state.  Since we are in an exception state I would be
> unlikely to be able to reverse the changes made to the model anyway,
> so a reload is really required in all cases.

The transaction *is* serialized to disk. Transactions are written to
the journal before being executed.

Cheers,
Justin

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[Justin T. Sampson-2]

On Mon, Jul 27, 2009 at 2:13 PM, Klaus
Wuestefeld<klauswuestefeld@...> wrote:

> These two things you wrote below seem sort of contradictory:
>
> > Conversely, if a transaction fails deterministically, it will always
> > fail, whether against the food taster, against the king, or later when
> > replayed from the journal. Therefore, eliminating the food taster does
> > not introduce any particular inconsistency
> ...
> > A rollback mechanism such as the food taster only helps you defend
> > against a very narrow class of bugs: *Deterministic* bugs that put the
> > system in an *inconsistent state* AND cause a RuntimeException to be
> > thrown from the *same* Transaction that introduced the inconsistent
> > state.

How so?

Note that I'm talking about slightly different concepts in these two
paragraphs: The first uses the term "fail" while the second uses the
term "bug". A transaction can fail (throw a RuntimeException) without
it being a bug.

If the issue is my emphasis of "*same* Transaction", what I meant was
that there are plenty of bugs in the wild where an inconsistency is
introduced which only leads to visible errors at some later time. The
food taster can only help if an error becomes visible (causes a
RuntimeException) during the same Transaction that caused it.

Cheers,
Justin

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

[Ross Rick]

Ok..  let me see if I understand.    Transaction starts, is written to  
disk, then the code blows up.

What happens next?    The transaction is attempted again on load.  If  
it continues to fail (deterministic or not) then I have to manually  
remove the transaction?


On Jul 27, 2009, at 3:17 PM, Justin T. Sampson wrote:

> On Mon, Jul 27, 2009 at 1:26 PM, Ross Rick<rick@...> wrote:
>
>> So yes, I am ok with keeping the model safe from deterministic
>> errors.  My concern is an error beyond my control after the model has
>> been modified in the transaction.   As I understand it Prevayler will
>> not set the model back to the state it was in before the transaction
>> started.  I can't imagine how it could.    And now reading this I
>> understand that this is independent of food tasting.  Of course, the
>> transaction will not be serialized to disk so I can reload the model
>> to the correct state.  Since we are in an exception state I would be
>> unlikely to be able to reverse the changes made to the model anyway,
>> so a reload is really required in all cases.
>
> The transaction *is* serialized to disk. Transactions are written to
> the journal before being executed.
>
> Cheers,
> Justin
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008  
> 30-Day
> trial. Simplify your report design, integration and deployment - and  
> focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org