RE: How to make Environment, XmlManager and XmlContainer objects visible

[David murphy-8]

Elias

>>We have listeners attached to our web application that create
Environment,
>>XmlManager and XmlContainer as required and then deletes them on
>>application shutdown.

> Expensive, but I guess you've already realized this.

Is there another way to avoid this expense? The java application is
actually middleware software (Cold Fusion) that we use to build our CMS
application, which in turn, relies on BDBXML.

Thanks, David


------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[George Feinberg]

David,

> Elias
>
>>> We have listeners attached to our web application that create
> Environment,
>>> XmlManager and XmlContainer as required and then deletes them on
>>> application shutdown.
>
>> Expensive, but I guess you've already realized this.
>
> Is there another way to avoid this expense? The java application is
> actually middleware software (Cold Fusion) that we use to build our  
> CMS
> application, which in turn, relies on BDBXML.

It's not entirely clear what you are asking.
If you are asking about the expense of creating, then closing an
entire environment, xmlmanager, and container on each
request, then the way to avoid that is to create them at application
startup, share them, then close them when you shut down.
This means you need to keep them around between requests.

If you do not, then much of the benefit of using a database is
lost.

Regards,

George



------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[David murphy-8]

George

No, I was interested in what Elias said about the fact that closing down
these objects on application shutdown was 'expensive'.

We do not create/delete on each request; we simply create single shared
instances of Environment, XmlManager and XmlContainer upon application
start-up and have these persist for the lifetime of the app (could be
weeks or months) via a hashmap.

So, I guess my question is whether there is a more efficient way of
interfacing with BDBXML other than creating/deleting these 3 key objects
on app startup / shutdown. I thought this was the best way to do this
but Elias' comments seemed to suggest that there might be a less
expensive way.

Thanks, David

> -----Original Message-----
> From: George Feinberg [mailto:gmf@...]
> Sent: 21 July 2006 18:17
> To: David Murphy
> Cc: Elias Sinderson; xml@...
> Subject: Re: [bdbxml] How to make Environment, XmlManager and
XmlContainer

> objects visible
>
> David,
>
> > Elias
> >
> >>> We have listeners attached to our web application that create
> > Environment,
> >>> XmlManager and XmlContainer as required and then deletes them on
> >>> application shutdown.
> >
> >> Expensive, but I guess you've already realized this.
> >
> > Is there another way to avoid this expense? The java application is
> > actually middleware software (Cold Fusion) that we use to build our
> > CMS
> > application, which in turn, relies on BDBXML.
>
> It's not entirely clear what you are asking.
> If you are asking about the expense of creating, then closing an
> entire environment, xmlmanager, and container on each
> request, then the way to avoid that is to create them at application
> startup, share them, then close them when you shut down.
> This means you need to keep them around between requests.
>
> If you do not, then much of the benefit of using a database is
> lost.
>
> Regards,
>
> George

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[George Feinberg]

David,

> George
>
> No, I was interested in what Elias said about the fact that closing  
> down
> these objects on application shutdown was 'expensive'.

The expense associated with shutting these down is the
need to flush the cache and log entries to the physical database.
The cost depends on (1) whether you are using transactions,
and (2) whether you've been using database checkpoints
to flush data during the life of your application.

For long-running applications using transactions, it's nearly
always necessary to use checkpoints to control flushing of
log entries to databases.  See this documentation for more details:
        http://www.sleepycat.com/xmldocs/ref/transapp/checkpoint.html

Using checkpoints also makes recovery faster in the event of
an unexpected application exit/crash.  Recovery will also
flush log entries to databases.

>
> We do not create/delete on each request; we simply create single  
> shared
> instances of Environment, XmlManager and XmlContainer upon application
> start-up and have these persist for the lifetime of the app (could be
> weeks or months) via a hashmap.
>
> So, I guess my question is whether there is a more efficient way of
> interfacing with BDBXML other than creating/deleting these 3 key  
> objects
> on app startup / shutdown. I thought this was the best way to do this
> but Elias' comments seemed to suggest that there might be a less
> expensive way.

These objects are required in order to use BDB XML, so there's
no way to avoid creating them.  Keeping them around as shared
objects, as you are doing, is the right thing.

You do need to do some checkpoint operations, though, to
keep a reasonable bound on shutdown and/or recovery time.

Regards,

George




------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[Antony Grinyer]

Hi,

We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use Tomcat
for our web application and during start-up we open our containers using
a listener; we also close our containers using a listener when Tomcat
shuts down, we do not use the container 'auto-open' environment
parameter. During a recent demo, Tomcat crashed unexpectedly which
appeared to corrupt all of our containers when we started Tomcat again.
Therefore, my question is what is the solution to ensure containers are
kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
crash' type parameters in BDB XML? Has anyone implemented a fail safe
scenario using Tomcat and BDB XML given a Tomcat crash?

Thanks in advance for any help,

Ant
---
Senior Technical Consultant
CSW Group Ltd.



------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[Madhu Sadagopan]

Hi:

Enable EnvironmentConfig.setRunRecovery()
(Getting started guide on Java - Chapter 3)

or

Recover the XMLDB from the command prompt using 'db_recover' after any ungraceful shutdown.

Recovery is not possible if the logging subsytem is disabled as it generates the needed log files for recovery. Hope this helps.

bye,
Madhu



Antony Grinyer <antony.grinyer@...> wrote:

Hi,

We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use Tomcat
for our web application and during start-up we open our containers using
a listener; we also close our containers using a listener when Tomcat
shuts down, we do not use the container 'auto-open' environment
parameter. During a recent demo, Tomcat crashed unexpectedly which
appeared to corrupt all of our containers when we started Tomcat again.
Therefore, my question is what is the solution to ensure containers are
kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
crash' type parameters in BDB XML? Has anyone implemented a fail safe
scenario using Tomcat and BDB XML given a Tomcat crash?

Thanks in advance for any help,

Ant
---
Senior Technical Consultant
CSW Group Ltd.



------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[John Snelson-4]

Hi Antony,

So long as you are using transactions, this situation is not a problem.
Since DB XML does not run as a separate server, the containers can often
be corrupted due to an application crash. In these situations, you will
need to run the recovery process, which will replay the transaction
logs. This webpage should help you out:

http://www.sleepycat.com/docs/ref/transapp/recovery.html

Many people run recovery every time their applications start, since
there is no penalty to running recovery when it is not needed.

You should also be checkpointing, if you aren't already. This will
reduce the amount of transaction logging that will need to be replayed:

http://www.sleepycat.com/docs/ref/transapp/checkpoint.html

Hope that helps,

John

Antony Grinyer wrote:

> Hi,
>
> We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use Tomcat
> for our web application and during start-up we open our containers using
> a listener; we also close our containers using a listener when Tomcat
> shuts down, we do not use the container 'auto-open' environment
> parameter. During a recent demo, Tomcat crashed unexpectedly which
> appeared to corrupt all of our containers when we started Tomcat again.
> Therefore, my question is what is the solution to ensure containers are
> kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
> crash' type parameters in BDB XML? Has anyone implemented a fail safe
> scenario using Tomcat and BDB XML given a Tomcat crash?
>
> Thanks in advance for any help,
>
> Ant
> ---
> Senior Technical Consultant
> CSW Group Ltd.
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an
> email to xml-unsubscribe@...
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[Antony Grinyer]

Thanks for the help John; how do you run a recovery on a programmatic
level in DB XML? From your advice, it would appear that running a
recovery each time our application starts is most sensible, but how is
this done in the DB XML API? We set the setRunRecovery=true environment
parameter but is there anything else we must do to ensure our containers
are recovered?

On a slightly different note, does BD XML produce any of its own error
logs we can read?

Thanks again for your excellent support and advise.

Ant
---
Senior Technical Consultant
CSW Group Ltd

-----Original Message-----
From: John Snelson [mailto:john.snelson@...]
Sent: 01 August 2006 18:08
To: Antony Grinyer
Cc: xml@...
Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?

Hi Antony,

So long as you are using transactions, this situation is not a problem.
Since DB XML does not run as a separate server, the containers can often

be corrupted due to an application crash. In these situations, you will
need to run the recovery process, which will replay the transaction
logs. This webpage should help you out:

http://www.sleepycat.com/docs/ref/transapp/recovery.html

Many people run recovery every time their applications start, since
there is no penalty to running recovery when it is not needed.

You should also be checkpointing, if you aren't already. This will
reduce the amount of transaction logging that will need to be replayed:

http://www.sleepycat.com/docs/ref/transapp/checkpoint.html

Hope that helps,

John

Antony Grinyer wrote:
> Hi,
>
> We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use Tomcat
> for our web application and during start-up we open our containers
using
> a listener; we also close our containers using a listener when Tomcat
> shuts down, we do not use the container 'auto-open' environment
> parameter. During a recent demo, Tomcat crashed unexpectedly which
> appeared to corrupt all of our containers when we started Tomcat
again.
> Therefore, my question is what is the solution to ensure containers
are

> kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
> crash' type parameters in BDB XML? Has anyone implemented a fail safe
> scenario using Tomcat and BDB XML given a Tomcat crash?
>
> Thanks in advance for any help,
>
> Ant
> ---
> Senior Technical Consultant
> CSW Group Ltd.
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an
> email to xml-unsubscribe@...
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[John Snelson-4]

To run recovery using the java API, you simply need to to use
EnvironmentConfig.setRunRecovery(true) when you create the Environment
object:

http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig.html#setRunRecovery(boolean)

Make sure that you follow the guidelines in the documentation above, and
run recovery in a single-threaded manner.

DB XML will log to the Berkeley DB error stream or error handler, which
is set using one of the following methods:

http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig.html#setErrorHandler(com.sleepycat.db.ErrorHandler)
http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig.html#setErrorStream(java.io.OutputStream)

To enable DB XML logging, you have to set the log categories and levels
that you want to receive logging from:

http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/XmlManager.html#setLogCategory(int,%20boolean)
http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/XmlManager.html#setLogLevel(int,%20boolean)

Hope that helps,

John

Antony Grinyer wrote:

> Thanks for the help John; how do you run a recovery on a programmatic
> level in DB XML? From your advice, it would appear that running a
> recovery each time our application starts is most sensible, but how is
> this done in the DB XML API? We set the setRunRecovery=true environment
> parameter but is there anything else we must do to ensure our containers
> are recovered?
>
> On a slightly different note, does BD XML produce any of its own error
> logs we can read?
>
> Thanks again for your excellent support and advise.
>
> Ant
> ---
> Senior Technical Consultant
> CSW Group Ltd
>
> -----Original Message-----
> From: John Snelson [mailto:john.snelson@...]
> Sent: 01 August 2006 18:08
> To: Antony Grinyer
> Cc: xml@...
> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>
> Hi Antony,
>
> So long as you are using transactions, this situation is not a problem.
> Since DB XML does not run as a separate server, the containers can often
>
> be corrupted due to an application crash. In these situations, you will
> need to run the recovery process, which will replay the transaction
> logs. This webpage should help you out:
>
> http://www.sleepycat.com/docs/ref/transapp/recovery.html
>
> Many people run recovery every time their applications start, since
> there is no penalty to running recovery when it is not needed.
>
> You should also be checkpointing, if you aren't already. This will
> reduce the amount of transaction logging that will need to be replayed:
>
> http://www.sleepycat.com/docs/ref/transapp/checkpoint.html
>
> Hope that helps,
>
> John
>
> Antony Grinyer wrote:
>> Hi,
>>
>> We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use Tomcat
>> for our web application and during start-up we open our containers
> using
>> a listener; we also close our containers using a listener when Tomcat
>> shuts down, we do not use the container 'auto-open' environment
>> parameter. During a recent demo, Tomcat crashed unexpectedly which
>> appeared to corrupt all of our containers when we started Tomcat
> again.
>> Therefore, my question is what is the solution to ensure containers
> are
>> kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
>> crash' type parameters in BDB XML? Has anyone implemented a fail safe
>> scenario using Tomcat and BDB XML given a Tomcat crash?
>>
>> Thanks in advance for any help,
>>
>> Ant
>> ---
>> Senior Technical Consultant
>> CSW Group Ltd.
>>
>>
>>
>> ------------------------------------------
>> To remove yourself from this list, send an
>> email to xml-unsubscribe@...
>>
>
>
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an
> email to xml-unsubscribe@...
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[Antony Grinyer]

Hi,

Apologies to keep going on, but I am still unsure of a programmatic
solution to Tomcat crashing with containers open - we already set the
setRunRecovery(true) flag on the environment config which makes no
difference if Tomcat crashes and our containers have not been closed -
we're still not able to use our containers after we start Tomcat again
after a crash. Realistically can we only use the recovery shell utility
to fix containers left open after a Tomcat crash?

Thanks for your continued help,
Ant

-----Original Message-----
From: John Snelson [mailto:john.snelson@...]
Sent: 03 August 2006 11:55
To: Antony Grinyer
Cc: xml@...
Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?

To run recovery using the java API, you simply need to to use
EnvironmentConfig.setRunRecovery(true) when you create the Environment
object:

http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig
.html#setRunRecovery(boolean)

Make sure that you follow the guidelines in the documentation above, and

run recovery in a single-threaded manner.

DB XML will log to the Berkeley DB error stream or error handler, which
is set using one of the following methods:

http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig
.html#setErrorHandler(com.sleepycat.db.ErrorHandler)
http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig
.html#setErrorStream(java.io.OutputStream)

To enable DB XML logging, you have to set the log categories and levels
that you want to receive logging from:

http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/XmlManager.htm
l#setLogCategory(int,%20boolean)
http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/XmlManager.htm
l#setLogLevel(int,%20boolean)

Hope that helps,

John

Antony Grinyer wrote:
> Thanks for the help John; how do you run a recovery on a programmatic
> level in DB XML? From your advice, it would appear that running a
> recovery each time our application starts is most sensible, but how is
> this done in the DB XML API? We set the setRunRecovery=true
environment
> parameter but is there anything else we must do to ensure our
containers

> are recovered?
>
> On a slightly different note, does BD XML produce any of its own error
> logs we can read?
>
> Thanks again for your excellent support and advise.
>
> Ant
> ---
> Senior Technical Consultant
> CSW Group Ltd
>
> -----Original Message-----
> From: John Snelson [mailto:john.snelson@...]
> Sent: 01 August 2006 18:08
> To: Antony Grinyer
> Cc: xml@...
> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>
> Hi Antony,
>
> So long as you are using transactions, this situation is not a

problem.
> Since DB XML does not run as a separate server, the containers can
often
>
> be corrupted due to an application crash. In these situations, you
will

> need to run the recovery process, which will replay the transaction
> logs. This webpage should help you out:
>
> http://www.sleepycat.com/docs/ref/transapp/recovery.html
>
> Many people run recovery every time their applications start, since
> there is no penalty to running recovery when it is not needed.
>
> You should also be checkpointing, if you aren't already. This will
> reduce the amount of transaction logging that will need to be

replayed:

>
> http://www.sleepycat.com/docs/ref/transapp/checkpoint.html
>
> Hope that helps,
>
> John
>
> Antony Grinyer wrote:
>> Hi,
>>
>> We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use

Tomcat

>> for our web application and during start-up we open our containers
> using
>> a listener; we also close our containers using a listener when Tomcat
>> shuts down, we do not use the container 'auto-open' environment
>> parameter. During a recent demo, Tomcat crashed unexpectedly which
>> appeared to corrupt all of our containers when we started Tomcat
> again.
>> Therefore, my question is what is the solution to ensure containers
> are
>> kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
>> crash' type parameters in BDB XML? Has anyone implemented a fail safe
>> scenario using Tomcat and BDB XML given a Tomcat crash?
>>
>> Thanks in advance for any help,
>>
>> Ant
>> ---
>> Senior Technical Consultant
>> CSW Group Ltd.
>>
>>
>>
>> ------------------------------------------
>> To remove yourself from this list, send an
>> email to xml-unsubscribe@...
>>
>
>
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an
> email to xml-unsubscribe@...
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[George Feinberg-3]

Antony,

Are you also setting EnvironmentConfig.setAllowCreate(true) ?
That's also necessary for recovery to be automatic.
It's also necessary that recovery be run single-threaded, while
no other threads or processes are using the environment.

For debugging, another thing you can set is:
EnvironmentConfig.setVerboseRecovery(true);

and you'll see recovery-related messages during Environment
creation.

Regards,

George

> Hi,
>
> Apologies to keep going on, but I am still unsure of a programmatic
> solution to Tomcat crashing with containers open - we already set the
> setRunRecovery(true) flag on the environment config which makes no
> difference if Tomcat crashes and our containers have not been closed -
> we're still not able to use our containers after we start Tomcat again
> after a crash. Realistically can we only use the recovery shell  
> utility
> to fix containers left open after a Tomcat crash?
>
> Thanks for your continued help,
> Ant
>
> -----Original Message-----
> From: John Snelson [mailto:john.snelson@...]
> Sent: 03 August 2006 11:55
> To: Antony Grinyer
> Cc: xml@...
> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>
> To run recovery using the java API, you simply need to to use
> EnvironmentConfig.setRunRecovery(true) when you create the Environment
> object:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/ 
> EnvironmentConfig
> .html#setRunRecovery(boolean)
>
> Make sure that you follow the guidelines in the documentation  
> above, and
>
> run recovery in a single-threaded manner.
>
> DB XML will log to the Berkeley DB error stream or error handler,  
> which
> is set using one of the following methods:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/ 
> EnvironmentConfig
> .html#setErrorHandler(com.sleepycat.db.ErrorHandler)
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/ 
> EnvironmentConfig
> .html#setErrorStream(java.io.OutputStream)
>
> To enable DB XML logging, you have to set the log categories and  
> levels
> that you want to receive logging from:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/ 
> XmlManager.htm
> l#setLogCategory(int,%20boolean)
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/ 
> XmlManager.htm
> l#setLogLevel(int,%20boolean)
>
> Hope that helps,
>
> John
>
> Antony Grinyer wrote:
>> Thanks for the help John; how do you run a recovery on a programmatic
>> level in DB XML? From your advice, it would appear that running a
>> recovery each time our application starts is most sensible, but  
>> how is
>> this done in the DB XML API? We set the setRunRecovery=true
> environment
>> parameter but is there anything else we must do to ensure our
> containers
>> are recovered?
>>
>> On a slightly different note, does BD XML produce any of its own  
>> error
>> logs we can read?
>>
>> Thanks again for your excellent support and advise.
>>
>> Ant
>> ---
>> Senior Technical Consultant
>> CSW Group Ltd
>>
>> -----Original Message-----
>> From: John Snelson [mailto:john.snelson@...]
>> Sent: 01 August 2006 18:08
>> To: Antony Grinyer
>> Cc: xml@...
>> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>>
>> Hi Antony,
>>
>> So long as you are using transactions, this situation is not a
> problem.
>> Since DB XML does not run as a separate server, the containers can
> often
>>
>> be corrupted due to an application crash. In these situations, you
> will
>> need to run the recovery process, which will replay the transaction
>> logs. This webpage should help you out:
>>
>> http://www.sleepycat.com/docs/ref/transapp/recovery.html
>>
>> Many people run recovery every time their applications start, since
>> there is no penalty to running recovery when it is not needed.
>>
>> You should also be checkpointing, if you aren't already. This will
>> reduce the amount of transaction logging that will need to be
> replayed:
>>
>> http://www.sleepycat.com/docs/ref/transapp/checkpoint.html
>>
>> Hope that helps,
>>
>> John
>>
>> Antony Grinyer wrote:
>>> Hi,
>>>
>>> We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use
> Tomcat
>>> for our web application and during start-up we open our containers
>> using
>>> a listener; we also close our containers using a listener when  
>>> Tomcat
>>> shuts down, we do not use the container 'auto-open' environment
>>> parameter. During a recent demo, Tomcat crashed unexpectedly which
>>> appeared to corrupt all of our containers when we started Tomcat
>> again.
>>> Therefore, my question is what is the solution to ensure containers
>> are
>>> kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
>>> crash' type parameters in BDB XML? Has anyone implemented a fail  
>>> safe
>>> scenario using Tomcat and BDB XML given a Tomcat crash?
>>>
>>> Thanks in advance for any help,
>>>
>>> Ant
>>> ---
>>> Senior Technical Consultant
>>> CSW Group Ltd.
>>>
>>>
>>>
>>> ------------------------------------------
>>> To remove yourself from this list, send an
>>> email to xml-unsubscribe@...
>>>
>>
>>
>>
>>
>>
>> ------------------------------------------
>> To remove yourself from this list, send an
>> email to xml-unsubscribe@...
>>
>
>
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an
> email to xml-unsubscribe@...
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[John Snelson-4]

The recovery shell program, db_recover, will do exactly the same as
specifying setRunRecovery(true) when you open your Environment object.

If you need more help, there is getting started with transactions guide
that is available here:

http://www.sleepycat.com/docs/gsg_txn/JAVA/index.html

This should answer most of your questions.

John

Antony Grinyer wrote:

> Hi,
>
> Apologies to keep going on, but I am still unsure of a programmatic
> solution to Tomcat crashing with containers open - we already set the
> setRunRecovery(true) flag on the environment config which makes no
> difference if Tomcat crashes and our containers have not been closed -
> we're still not able to use our containers after we start Tomcat again
> after a crash. Realistically can we only use the recovery shell utility
> to fix containers left open after a Tomcat crash?
>
> Thanks for your continued help,
> Ant
>
> -----Original Message-----
> From: John Snelson [mailto:john.snelson@...]
> Sent: 03 August 2006 11:55
> To: Antony Grinyer
> Cc: xml@...
> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>
> To run recovery using the java API, you simply need to to use
> EnvironmentConfig.setRunRecovery(true) when you create the Environment
> object:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig
> .html#setRunRecovery(boolean)
>
> Make sure that you follow the guidelines in the documentation above, and
>
> run recovery in a single-threaded manner.
>
> DB XML will log to the Berkeley DB error stream or error handler, which
> is set using one of the following methods:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig
> .html#setErrorHandler(com.sleepycat.db.ErrorHandler)
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/EnvironmentConfig
> .html#setErrorStream(java.io.OutputStream)
>
> To enable DB XML logging, you have to set the log categories and levels
> that you want to receive logging from:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/XmlManager.htm
> l#setLogCategory(int,%20boolean)
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/XmlManager.htm
> l#setLogLevel(int,%20boolean)
>
> Hope that helps,
>
> John
>
> Antony Grinyer wrote:
>> Thanks for the help John; how do you run a recovery on a programmatic
>> level in DB XML? From your advice, it would appear that running a
>> recovery each time our application starts is most sensible, but how is
>> this done in the DB XML API? We set the setRunRecovery=true
> environment
>> parameter but is there anything else we must do to ensure our
> containers
>> are recovered?
>>
>> On a slightly different note, does BD XML produce any of its own error
>> logs we can read?
>>
>> Thanks again for your excellent support and advise.
>>
>> Ant
>> ---
>> Senior Technical Consultant
>> CSW Group Ltd
>>
>> -----Original Message-----
>> From: John Snelson [mailto:john.snelson@...]
>> Sent: 01 August 2006 18:08
>> To: Antony Grinyer
>> Cc: xml@...
>> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>>
>> Hi Antony,
>>
>> So long as you are using transactions, this situation is not a
> problem.
>> Since DB XML does not run as a separate server, the containers can
> often
>> be corrupted due to an application crash. In these situations, you
> will
>> need to run the recovery process, which will replay the transaction
>> logs. This webpage should help you out:
>>
>> http://www.sleepycat.com/docs/ref/transapp/recovery.html
>>
>> Many people run recovery every time their applications start, since
>> there is no penalty to running recovery when it is not needed.
>>
>> You should also be checkpointing, if you aren't already. This will
>> reduce the amount of transaction logging that will need to be
> replayed:
>> http://www.sleepycat.com/docs/ref/transapp/checkpoint.html
>>
>> Hope that helps,
>>
>> John
>>
>> Antony Grinyer wrote:
>>> Hi,
>>>
>>> We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use
> Tomcat
>>> for our web application and during start-up we open our containers
>> using
>>> a listener; we also close our containers using a listener when Tomcat
>>> shuts down, we do not use the container 'auto-open' environment
>>> parameter. During a recent demo, Tomcat crashed unexpectedly which
>>> appeared to corrupt all of our containers when we started Tomcat
>> again.
>>> Therefore, my question is what is the solution to ensure containers
>> are
>>> kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
>>> crash' type parameters in BDB XML? Has anyone implemented a fail safe
>>> scenario using Tomcat and BDB XML given a Tomcat crash?
>>>
>>> Thanks in advance for any help,
>>>
>>> Ant
>>> ---
>>> Senior Technical Consultant
>>> CSW Group Ltd.
>>>
>>>
>>>
>>> ------------------------------------------
>>> To remove yourself from this list, send an
>>> email to xml-unsubscribe@...
>>>
>>
>>
>>
>>
>> ------------------------------------------
>> To remove yourself from this list, send an
>> email to xml-unsubscribe@...
>>
>
>
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an
> email to xml-unsubscribe@...
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[Antony Grinyer]

George - yes we have EnvironmentConfig.setAllowCreate(true) set.

Thanks very much for your help again John/George, I'll have a look at
the documentation and try the setVerboseRecovery flag to see what
messages are produced.

Thanks,
Ant

-----Original Message-----
From: George Feinberg [mailto:george.feinberg@...]
Sent: 03 August 2006 14:39
To: Antony Grinyer
Cc: xml@...
Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?

Antony,

Are you also setting EnvironmentConfig.setAllowCreate(true) ?
That's also necessary for recovery to be automatic.
It's also necessary that recovery be run single-threaded, while
no other threads or processes are using the environment.

For debugging, another thing you can set is:
EnvironmentConfig.setVerboseRecovery(true);

and you'll see recovery-related messages during Environment
creation.

Regards,

George

> Hi,
>
> Apologies to keep going on, but I am still unsure of a programmatic
> solution to Tomcat crashing with containers open - we already set the
> setRunRecovery(true) flag on the environment config which makes no
> difference if Tomcat crashes and our containers have not been closed -
> we're still not able to use our containers after we start Tomcat again
> after a crash. Realistically can we only use the recovery shell  
> utility
> to fix containers left open after a Tomcat crash?
>
> Thanks for your continued help,
> Ant
>
> -----Original Message-----
> From: John Snelson [mailto:john.snelson@...]
> Sent: 03 August 2006 11:55
> To: Antony Grinyer
> Cc: xml@...
> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>
> To run recovery using the java API, you simply need to to use
> EnvironmentConfig.setRunRecovery(true) when you create the Environment
> object:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/ 
> EnvironmentConfig
> .html#setRunRecovery(boolean)
>
> Make sure that you follow the guidelines in the documentation  
> above, and
>
> run recovery in a single-threaded manner.
>
> DB XML will log to the Berkeley DB error stream or error handler,  
> which
> is set using one of the following methods:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/ 
> EnvironmentConfig
> .html#setErrorHandler(com.sleepycat.db.ErrorHandler)
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/db/ 
> EnvironmentConfig
> .html#setErrorStream(java.io.OutputStream)
>
> To enable DB XML logging, you have to set the log categories and  
> levels
> that you want to receive logging from:
>
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/ 
> XmlManager.htm
> l#setLogCategory(int,%20boolean)
> http://www.sleepycat.com/xmldocs/java/com/sleepycat/dbxml/ 
> XmlManager.htm
> l#setLogLevel(int,%20boolean)
>
> Hope that helps,
>
> John
>
> Antony Grinyer wrote:
>> Thanks for the help John; how do you run a recovery on a programmatic
>> level in DB XML? From your advice, it would appear that running a
>> recovery each time our application starts is most sensible, but  
>> how is
>> this done in the DB XML API? We set the setRunRecovery=true
> environment
>> parameter but is there anything else we must do to ensure our
> containers
>> are recovered?
>>
>> On a slightly different note, does BD XML produce any of its own  
>> error
>> logs we can read?
>>
>> Thanks again for your excellent support and advise.
>>
>> Ant
>> ---
>> Senior Technical Consultant
>> CSW Group Ltd
>>
>> -----Original Message-----
>> From: John Snelson [mailto:john.snelson@...]
>> Sent: 01 August 2006 18:08
>> To: Antony Grinyer
>> Cc: xml@...
>> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>>
>> Hi Antony,
>>
>> So long as you are using transactions, this situation is not a
> problem.
>> Since DB XML does not run as a separate server, the containers can
> often
>>
>> be corrupted due to an application crash. In these situations, you
> will
>> need to run the recovery process, which will replay the transaction
>> logs. This webpage should help you out:
>>
>> http://www.sleepycat.com/docs/ref/transapp/recovery.html
>>
>> Many people run recovery every time their applications start, since
>> there is no penalty to running recovery when it is not needed.
>>
>> You should also be checkpointing, if you aren't already. This will
>> reduce the amount of transaction logging that will need to be
> replayed:
>>
>> http://www.sleepycat.com/docs/ref/transapp/checkpoint.html
>>
>> Hope that helps,
>>
>> John
>>
>> Antony Grinyer wrote:
>>> Hi,
>>>
>>> We are using BDB XML 2.2.13 on Windows XP with Java 1.5. We use
> Tomcat
>>> for our web application and during start-up we open our containers
>> using
>>> a listener; we also close our containers using a listener when  
>>> Tomcat
>>> shuts down, we do not use the container 'auto-open' environment
>>> parameter. During a recent demo, Tomcat crashed unexpectedly which
>>> appeared to corrupt all of our containers when we started Tomcat
>> again.
>>> Therefore, my question is what is the solution to ensure containers
>> are
>>> kept uncorrupted given a Tomcat crash? Is there any 'auto-close on
>>> crash' type parameters in BDB XML? Has anyone implemented a fail  
>>> safe
>>> scenario using Tomcat and BDB XML given a Tomcat crash?
>>>
>>> Thanks in advance for any help,
>>>
>>> Ant
>>> ---
>>> Senior Technical Consultant
>>> CSW Group Ltd.
>>>
>>>
>>>
>>> ------------------------------------------
>>> To remove yourself from this list, send an
>>> email to xml-unsubscribe@...
>>>
>>
>>
>>
>>
>>
>> ------------------------------------------
>> To remove yourself from this list, send an
>> email to xml-unsubscribe@...
>>
>
>
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an
> email to xml-unsubscribe@...
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[Antony Grinyer]

Having followed all the advice to recover the state of our containers
our container still won't open after a Tomcat crash despite using the
run recovery parameters; therefore I guess as a last effort, it's best
to describe the setup and scenario we have.

We use DBXML 2.2.13 and set the following environment parameters when
our application starts:

EnvironmentConfig config = new EnvironmentConfig();
config.setAllowCreate(true);
config.setInitializeCache(true);
config.setInitializeLocking(true);
config.setInitializeLogging(true);
config.setTransactional(true);
config.setRunRecovery(true);
config.setVerboseRecovery(true);
config.setCacheSize(512000000);
config.setMaxLocks(2000);
config.setMaxLockers(2000);
config.setMaxLockObjects(2000);

We use Tomcat 5.5.17 as our application server. To simulate the problem
I run Tomcat which in turn starts our application; I login to our
application and create a container which stays open for the lifetime of
the application (while Tomcat is running); then, if we close the
application altogether and close Tomcat, we have a listener hooked into
Tomcat which closes all the open containers and the XmlManager we use in
the application. This seems to work very nicely if we close Tomcat
sensibly. After simulating a Tomcat crash (killing the java.exe process
in Task Manager, which does not close the containers or XmlManager
object gracefully) we log into our application and try to select the
container we created - this results in an error message reported to the
console saying the container does not exist; interestingly, we also get
a reported message that a recovery was complete. Here are the messages
written to the console:

Finding last valid log LSN: file: 1 offset 16856
Recovery complete at Wed Aug 09 11:06:56 2006
Maximum transaction ID 0 Recovery checkpoint [0][0]
com.sleepycat.dbxml.XmlException: Error: No such file or directory,
errcode = DATABASE_ERROR

The 'No such file or directory' error occurs even though the containers
exist.

Note we are not using transactions in our code; however we're not sure
how this would prevent the same problem as the container would still be
open during the crash and subsequently would become unstable/corrupted
without a sensible closure.

Because of this, we're thinking of implementing some sort of real-time
back up plan (in addition to our robust incremental backup plan) to copy
the containers to another directly every so often to ensure we can at
least pull back the containers quickly.

Any last ditch advice would be great.

Thanks,
Ant
CSW Group Ltd.



------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[George Feinberg-3]

Antony,

What configuration do you use to create the container itself?

Reminder -- this list could start bouncing email any day now.
Traffic is moving to the OTN forum:
        http://forums.oracle.com/forums/forum.jspa?forumID=274

George

> Having followed all the advice to recover the state of our containers
> our container still won't open after a Tomcat crash despite using the
> run recovery parameters; therefore I guess as a last effort, it's best
> to describe the setup and scenario we have.
>
> We use DBXML 2.2.13 and set the following environment parameters when
> our application starts:
>
> EnvironmentConfig config = new EnvironmentConfig();
> config.setAllowCreate(true);
> config.setInitializeCache(true);
> config.setInitializeLocking(true);
> config.setInitializeLogging(true);
> config.setTransactional(true);
> config.setRunRecovery(true);
> config.setVerboseRecovery(true);
> config.setCacheSize(512000000);
> config.setMaxLocks(2000);
> config.setMaxLockers(2000);
> config.setMaxLockObjects(2000);
>
> We use Tomcat 5.5.17 as our application server. To simulate the  
> problem
> I run Tomcat which in turn starts our application; I login to our
> application and create a container which stays open for the  
> lifetime of
> the application (while Tomcat is running); then, if we close the
> application altogether and close Tomcat, we have a listener hooked  
> into
> Tomcat which closes all the open containers and the XmlManager we  
> use in
> the application. This seems to work very nicely if we close Tomcat
> sensibly. After simulating a Tomcat crash (killing the java.exe  
> process
> in Task Manager, which does not close the containers or XmlManager
> object gracefully) we log into our application and try to select the
> container we created - this results in an error message reported to  
> the
> console saying the container does not exist; interestingly, we also  
> get
> a reported message that a recovery was complete. Here are the messages
> written to the console:
>
> Finding last valid log LSN: file: 1 offset 16856
> Recovery complete at Wed Aug 09 11:06:56 2006
> Maximum transaction ID 0 Recovery checkpoint [0][0]
> com.sleepycat.dbxml.XmlException: Error: No such file or directory,
> errcode = DATABASE_ERROR
>
> The 'No such file or directory' error occurs even though the  
> containers
> exist.
>
> Note we are not using transactions in our code; however we're not sure
> how this would prevent the same problem as the container would  
> still be
> open during the crash and subsequently would become unstable/corrupted
> without a sensible closure.
>
> Because of this, we're thinking of implementing some sort of real-time
> back up plan (in addition to our robust incremental backup plan) to  
> copy
> the containers to another directly every so often to ensure we can at
> least pull back the containers quickly.
>
> Any last ditch advice would be great.
>
> Thanks,
> Ant
> CSW Group Ltd.

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...

[Antony Grinyer]

George,

I'll move this issue to the new forum - thanks.

Ant

> -----Original Message-----
> From: George Feinberg [mailto:george.feinberg@...]
> Sent: 10 August 2006 19:42
> To: Antony Grinyer
> Cc: xml@...; John Snelson
> Subject: Re: [bdbxml] Unexpected Tomcat closure breaks containers?
>
>
> Antony,
>
> What configuration do you use to create the container itself?
>
> Reminder -- this list could start bouncing email any day now.
> Traffic is moving to the OTN forum:
> http://forums.oracle.com/forums/forum.jspa?forumID=274
>
> George
>
> > Having followed all the advice to recover the state of our
> containers
> > our container still won't open after a Tomcat crash despite
> using the
> > run recovery parameters; therefore I guess as a last
> effort, it's best
> > to describe the setup and scenario we have.
> >
> > We use DBXML 2.2.13 and set the following environment
> parameters when
> > our application starts:
> >
> > EnvironmentConfig config = new EnvironmentConfig();
> > config.setAllowCreate(true); config.setInitializeCache(true);
> > config.setInitializeLocking(true);
> > config.setInitializeLogging(true);
> > config.setTransactional(true);
> > config.setRunRecovery(true);
> > config.setVerboseRecovery(true);
> > config.setCacheSize(512000000);
> > config.setMaxLocks(2000);
> > config.setMaxLockers(2000);
> > config.setMaxLockObjects(2000);
> >
> > We use Tomcat 5.5.17 as our application server. To simulate the
> > problem I run Tomcat which in turn starts our application;
> I login to
> > our application and create a container which stays open for the
> > lifetime of the application (while Tomcat is running); then, if we
> > close the application altogether and close Tomcat, we have
> a listener
> > hooked into Tomcat which closes all the open containers and the
> > XmlManager we use in the application. This seems to work
> very nicely
> > if we close Tomcat sensibly. After simulating a Tomcat
> crash (killing
> > the java.exe process in Task Manager, which does not close the
> > containers or XmlManager object gracefully) we log into our
> > application and try to select the container we created -
> this results
> > in an error message reported to the console saying the
> container does
> > not exist; interestingly, we also get a reported message that a
> > recovery was complete. Here are the messages written to the console:
> >
> > Finding last valid log LSN: file: 1 offset 16856 Recovery
> complete at
> > Wed Aug 09 11:06:56 2006 Maximum transaction ID 0 Recovery
> checkpoint
> > [0][0]
> > com.sleepycat.dbxml.XmlException: Error: No such file or directory,
> > errcode = DATABASE_ERROR
> >
> > The 'No such file or directory' error occurs even though the
> > containers exist.
> >
> > Note we are not using transactions in our code; however
> we're not sure
> > how this would prevent the same problem as the container
> would still
> > be open during the crash and subsequently would become
> > unstable/corrupted without a sensible closure.
> >
> > Because of this, we're thinking of implementing some sort
> of real-time
> > back up plan (in addition to our robust incremental backup plan) to
> > copy the containers to another directly every so often to ensure we
> > can at least pull back the containers quickly.
> >
> > Any last ditch advice would be great.
> >
> > Thanks,
> > Ant
> > CSW Group Ltd.
>
>
>
> ------------------------------------------
> To remove yourself from this list, send an email to
> xml-unsubscribe@...
>
>
>

------------------------------------------
To remove yourself from this list, send an
email to xml-unsubscribe@...