Creating objects in the database

View: New views
14 Messages — Rating Filter:   Alert me  

Creating objects in the database

by Khayundi, Peter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.

I am doing some timings with the Prevayler database. I inserted 20000 items into the database and timed this operation. The experiment was repeated 10 times. The resulting times differ widely and have a high standard deviation. Does anybody have an explanation as to why the same experiment should have such widely different timings?

 

Peter

The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further: http://www.ufh.ac.za/disclaimer/
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-
http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by William Pietri :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.
Khayundi, Peter wrote:

I am doing some timings with the Prevayler database. I inserted 20000 items into the database and timed this operation. The experiment was repeated 10 times. The resulting times differ widely and have a high standard deviation. Does anybody have an explanation as to why the same experiment should have such widely different timings?

 


Hi, Peter. My first two guesses: other IO on the same disk and Java garbage collection. You can reduce problems with the first by using a separate disk, and with the latter by giving Java lots more RAM than necessary. There are also assorted tunings you can do, but those depend on OS and JDK version.


William

--
William Pietri - william@... - +1-415-643-1024
Agile consulting, coaching, and development: http://www.scissor.com/
We'd love feedback on our new blog: http://agilefocus.com/





------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Klaus Wuestefeld :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Are you doing SCADA or OLTP?


On Fri, Feb 6, 2009 at 3:24 PM, William Pietri <william@...> wrote:

> Khayundi, Peter wrote:
>
> I am doing some timings with the Prevayler database. I inserted 20000 items
> into the database and timed this operation. The experiment was repeated 10
> times. The resulting times differ widely and have a high standard deviation.
> Does anybody have an explanation as to why the same experiment should have
> such widely different timings?
>
>
>
> Hi, Peter. My first two guesses: other IO on the same disk and Java garbage
> collection. You can reduce problems with the first by using a separate disk,
> and with the latter by giving Java lots more RAM than necessary. There are
> also assorted tunings you can do, but those depend on OS and JDK version.
>
>
> William
>
> --
> William Pietri - william@... - +1-415-643-1024
> Agile consulting, coaching, and development: http://www.scissor.com/
> We'd love feedback on our new blog: http://agilefocus.com/
>
>
>
>
>
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with
> Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code
> to
> build responsive, highly engaging applications that combine the power of
> local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Klaus Wuestefeld :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Also: Are you creating 20000 objects in a single transaction or in
20000 separate transactions?


On Fri, Feb 6, 2009 at 6:49 PM, Klaus Wuestefeld
<klauswuestefeld@...> wrote:

> Are you doing SCADA or OLTP?
>
>
> On Fri, Feb 6, 2009 at 3:24 PM, William Pietri <william@...> wrote:
>> Khayundi, Peter wrote:
>>
>> I am doing some timings with the Prevayler database. I inserted 20000 items
>> into the database and timed this operation. The experiment was repeated 10
>> times. The resulting times differ widely and have a high standard deviation.
>> Does anybody have an explanation as to why the same experiment should have
>> such widely different timings?
>>
>>
>>
>> Hi, Peter. My first two guesses: other IO on the same disk and Java garbage
>> collection. You can reduce problems with the first by using a separate disk,
>> and with the latter by giving Java lots more RAM than necessary. There are
>> also assorted tunings you can do, but those depend on OS and JDK version.
>>
>>
>> William
>>
>> --
>> William Pietri - william@... - +1-415-643-1024
>> Agile consulting, coaching, and development: http://www.scissor.com/
>> We'd love feedback on our new blog: http://agilefocus.com/
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Create and Deploy Rich Internet Apps outside the browser with
>> Adobe(R)AIR(TM)
>> software. With Adobe AIR, Ajax developers can use existing skills and code
>> to
>> build responsive, highly engaging applications that combine the power of
>> local
>> resources and data with the reach of the web. Download the Adobe AIR SDK and
>> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
>> _______________________________________________
>> To unsubscribe go to the end of this page:
>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> _______________________________________________
>> "Databases in Memoriam" -- http://www.prevayler.org
>>
>>
>

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Sergey Didenko :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Could you give more details, Peter? Running times, the code, etc.

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Khayundi, Peter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi William,

 

Thanks for the reply to my query. About the IO, I have ensured that there is no other unnecessary process running on the machine during the operation. And also, about the second problem, I increased the Java heap size to 512MB which should be more than adequate for this operation. Any other ideas that I may try?

 

Peter.

 

From: William Pietri [mailto:william@...]
Sent: Friday, February 06, 2009 7:24 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database

 

Khayundi, Peter wrote:

I am doing some timings with the Prevayler database. I inserted 20000 items into the database and timed this operation. The experiment was repeated 10 times. The resulting times differ widely and have a high standard deviation. Does anybody have an explanation as to why the same experiment should have such widely different timings?

 


Hi, Peter. My first two guesses: other IO on the same disk and Java garbage collection. You can reduce problems with the first by using a separate disk, and with the latter by giving Java lots more RAM than necessary. There are also assorted tunings you can do, but those depend on OS and JDK version.


William

--
William Pietri - william@... - +1-415-643-1024
Agile consulting, coaching, and development: http://www.scissor.com/
We'd love feedback on our new blog: http://agilefocus.com/



The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further: http://www.ufh.ac.za/disclaimer/
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-
http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Khayundi, Peter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Klaus,

Thanks for the reply. I am doing a comparison of Open Source object
databases and I have so far tried to add 20000 objects to various other
databases. These other databases do not give me the problems that I
mentioned when I use Prevayler. I would therefore like to know if there
is an error that I may be overlooking. I have attached the code with
this message.

Peter.

-----Original Message-----
From: Klaus Wuestefeld [mailto:klauswuestefeld@...]
Sent: Friday, February 06, 2009 10:49 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database

Are you doing SCADA or OLTP?


On Fri, Feb 6, 2009 at 3:24 PM, William Pietri <william@...>
wrote:
> Khayundi, Peter wrote:
>
> I am doing some timings with the Prevayler database. I inserted 20000
items
> into the database and timed this operation. The experiment was
repeated 10
> times. The resulting times differ widely and have a high standard
deviation.
> Does anybody have an explanation as to why the same experiment should
have
> such widely different timings?
>
>
>
> Hi, Peter. My first two guesses: other IO on the same disk and Java
garbage
> collection. You can reduce problems with the first by using a separate
disk,
> and with the latter by giving Java lots more RAM than necessary. There
are
> also assorted tunings you can do, but those depend on OS and JDK
version.

>
>
> William
>
> --
> William Pietri - william@... - +1-415-643-1024
> Agile consulting, coaching, and development: http://www.scissor.com/
> We'd love feedback on our new blog: http://agilefocus.com/
>
>
>
>
>
>
------------------------------------------------------------------------
------
> Create and Deploy Rich Internet Apps outside the browser with
> Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and
code
> to
> build responsive, highly engaging applications that combine the power
of
> local
> resources and data with the reach of the web. Download the Adobe AIR
SDK and
> Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>

------------------------------------------------------------------------
------
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
code to
build responsive, highly engaging applications that combine the power of
local
resources and data with the reach of the web. Download the Adobe AIR SDK
and
Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org



The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further:
http://www.ufh.ac.za/disclaimer/




------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Peter.rar (3K) Download Attachment

Re: Creating objects in the database

by Khayundi, Peter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

The objects are created in a single transaction.

-----Original Message-----
From: Klaus Wuestefeld [mailto:klauswuestefeld@...]
Sent: Friday, February 06, 2009 11:04 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database

Also: Are you creating 20000 objects in a single transaction or in
20000 separate transactions?


On Fri, Feb 6, 2009 at 6:49 PM, Klaus Wuestefeld
<klauswuestefeld@...> wrote:
> Are you doing SCADA or OLTP?
>
>
> On Fri, Feb 6, 2009 at 3:24 PM, William Pietri <william@...>
wrote:
>> Khayundi, Peter wrote:
>>
>> I am doing some timings with the Prevayler database. I inserted 20000
items
>> into the database and timed this operation. The experiment was
repeated 10
>> times. The resulting times differ widely and have a high standard
deviation.
>> Does anybody have an explanation as to why the same experiment should
have
>> such widely different timings?
>>
>>
>>
>> Hi, Peter. My first two guesses: other IO on the same disk and Java
garbage
>> collection. You can reduce problems with the first by using a
separate disk,
>> and with the latter by giving Java lots more RAM than necessary.
There are
>> also assorted tunings you can do, but those depend on OS and JDK
version.

>>
>>
>> William
>>
>> --
>> William Pietri - william@... - +1-415-643-1024
>> Agile consulting, coaching, and development: http://www.scissor.com/
>> We'd love feedback on our new blog: http://agilefocus.com/
>>
>>
>>
>>
>>
>>
------------------------------------------------------------------------
------
>> Create and Deploy Rich Internet Apps outside the browser with
>> Adobe(R)AIR(TM)
>> software. With Adobe AIR, Ajax developers can use existing skills and
code
>> to
>> build responsive, highly engaging applications that combine the power
of
>> local
>> resources and data with the reach of the web. Download the Adobe AIR
SDK and
>> Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
>> _______________________________________________
>> To unsubscribe go to the end of this page:
>> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
>> _______________________________________________
>> "Databases in Memoriam" -- http://www.prevayler.org
>>
>>
>

------------------------------------------------------------------------
------
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
code to
build responsive, highly engaging applications that combine the power of
local
resources and data with the reach of the web. Download the Adobe AIR SDK
and
Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org



The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further:
http://www.ufh.ac.za/disclaimer/



------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Khayundi, Peter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I have attached the code above.

-----Original Message-----
From: Sergey Didenko [mailto:sergey.didenko@...]
Sent: Saturday, February 07, 2009 12:44 AM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database

Could you give more details, Peter? Running times, the code, etc.

------------------------------------------------------------------------
------
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
code to
build responsive, highly engaging applications that combine the power of
local
resources and data with the reach of the web. Download the Adobe AIR SDK
and
Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org



The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further:
http://www.ufh.ac.za/disclaimer/




------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Peter.rar (3K) Download Attachment

Re: Creating objects in the database

by William Pietri :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.
Khayundi, Peter wrote:

Hi William,

 

Thanks for the reply to my query. About the IO, I have ensured that there is no other unnecessary process running on the machine during the operation. And also, about the second problem, I increased the Java heap size to 512MB which should be more than adequate for this operation. Any other ideas that I may try?

 


Could necessary processes also write to or read from the same disk? Normally if one is doing tests like this, one uses an entirely separate disk. That's because people who care about performance use separate spindles in production.

Beyond that, correct server optimization is really an OS-dependent thing. Looking at your code, I see you're doing each write as a transaction. That's fine, but that means performance will vary widely depending on how things are tuned. With the default filesystem and tuning on my laptop, I get about 8.4ms per transaction, and I can hear the drive seeking like crazy. If I eliminate the seeks, the time per transaction drops to 0.14ms per transaction, nearly a 60x difference.

Also, have you tried using GC logging to rule out GC activity during your runs? Although I agree that 512MB should be more than adequate, what I think the JVM should do in the abstract often diverges from what it actually gets up to.

William

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by William Pietri :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Just to be clear, each object is saved in its own transaction, so that's 20000 separate transactions.

William

Khayundi, Peter wrote:
The objects are created in a single transaction.

-----Original Message-----
From: Klaus Wuestefeld [klauswuestefeld@...] 
Sent: Friday, February 06, 2009 11:04 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database

Also: Are you creating 20000 objects in a single transaction or in
20000 separate transactions?


On Fri, Feb 6, 2009 at 6:49 PM, Klaus Wuestefeld
klauswuestefeld@... wrote:
  
Are you doing SCADA or OLTP?


On Fri, Feb 6, 2009 at 3:24 PM, William Pietri william@...
    
wrote:
  
Khayundi, Peter wrote:

I am doing some timings with the Prevayler database. I inserted 20000
      
items
  
into the database and timed this operation. The experiment was
      
repeated 10
  
times. The resulting times differ widely and have a high standard
      
deviation.
  
Does anybody have an explanation as to why the same experiment should
      
have
  
such widely different timings?



Hi, Peter. My first two guesses: other IO on the same disk and Java
      
garbage
  
collection. You can reduce problems with the first by using a
      
separate disk,
  
and with the latter by giving Java lots more RAM than necessary.
      
There are
  
also assorted tunings you can do, but those depend on OS and JDK
      
version.
  
William

--
William Pietri - william@... - +1-415-643-1024
Agile consulting, coaching, and development: http://www.scissor.com/
We'd love feedback on our new blog: http://agilefocus.com/






      
------------------------------------------------------------------------
------
  
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
      
code
  
to
build responsive, highly engaging applications that combine the power
      
of
  
local
resources and data with the reach of the web. Download the Adobe AIR
      
SDK and
  
Ajax docs to start building applications
      
today-http://p.sf.net/sfu/adobe-com
  
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org


      

------------------------------------------------------------------------
------
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
code to
build responsive, highly engaging applications that combine the power of
local
resources and data with the reach of the web. Download the Adobe AIR SDK
and
Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org



The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further:
http://www.ufh.ac.za/disclaimer/



------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
  


------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Klaus Wuestefeld :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

If you're doing SCADA it does not make sense to insert each object in
its own transaction. Create a few hundred objects per transaction and
observe a massive increase in speed.

If you're doing OLTP and you need massive scalability, you want your
benchmarks to run on hundreds of threads, with hundreds of
simultaneous transactions being submitted to Prevayler. There is no
realistic singlethreaded massively-intensive OLTP scenario (it becomes
a SCADA scenario).

Prevayler will easily handle 1500 simultaneous requests on 1500
different threads and gives you aprox 1000 transactions per second
throughput for medium-sized object inserts.

See you, Klaus.


On Mon, Feb 9, 2009 at 3:41 PM, William Pietri <william@...> wrote:

> Just to be clear, each object is saved in its own transaction, so that's
> 20000 separate transactions.
>
> William
>
> Khayundi, Peter wrote:
>
> The objects are created in a single transaction.
> -----Original Message-----
> From: Klaus Wuestefeld [mailto:klauswuestefeld@...]
> Sent: Friday, February 06, 2009 11:04 PM
> To: Open discussion about the Prevayler project.
> Subject: Re: [Prevayler-discussion] Creating objects in the database
> Also: Are you creating 20000 objects in a single transaction or in
> 20000 separate transactions?
> On Fri, Feb 6, 2009 at 6:49 PM, Klaus Wuestefeld
> <klauswuestefeld@...> wrote:
>
>
> Are you doing SCADA or OLTP?
> On Fri, Feb 6, 2009 at 3:24 PM, William Pietri <william@...>
>
>
> wrote:
>
>
> Khayundi, Peter wrote:
> I am doing some timings with the Prevayler database. I inserted 20000
>
>
> items
>
>
> into the database and timed this operation. The experiment was
>
>
> repeated 10
>
>
> times. The resulting times differ widely and have a high standard
>
>
> deviation.
>
>
> Does anybody have an explanation as to why the same experiment should
>
>
> have
>
>
> such widely different timings?
> Hi, Peter. My first two guesses: other IO on the same disk and Java
>
>
> garbage
>
>
> collection. You can reduce problems with the first by using a
>
>
> separate disk,
>
>
> and with the latter by giving Java lots more RAM than necessary.
>
>
> There are
>
>
> also assorted tunings you can do, but those depend on OS and JDK
>
>
> version.
>
>
> William
> --
> William Pietri - william@... - +1-415-643-1024
> Agile consulting, coaching, and development: http://www.scissor.com/
> We'd love feedback on our new blog: http://agilefocus.com/
>
>
> ------------------------------------------------------------------------
> ------
>
>
> Create and Deploy Rich Internet Apps outside the browser with
> Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and
>
>
> code
>
>
> to
> build responsive, highly engaging applications that combine the power
>
>
> of
>
>
> local
> resources and data with the reach of the web. Download the Adobe AIR
>
>
> SDK and
>
>
> Ajax docs to start building applications
>
>
> today-http://p.sf.net/sfu/adobe-com
>
>
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>
> ------------------------------------------------------------------------
> ------
> Create and Deploy Rich Internet Apps outside the browser with
> Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and
> code to
> build responsive, highly engaging applications that combine the power of
> local
> resources and data with the reach of the web. Download the Adobe AIR SDK
> and
> Ajax docs to start building applications
> today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
> The University of Fort Hare subscribes to an e-mail disclaimer. Please click
> on the following URL to read further:
> http://www.ufh.ac.za/disclaimer/
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with
> Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code
> to
> build responsive, highly engaging applications that combine the power of
> local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with
> Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code
> to
> build responsive, highly engaging applications that combine the power of
> local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> To unsubscribe go to the end of this page:
> http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
> _______________________________________________
> "Databases in Memoriam" -- http://www.prevayler.org
>
>

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by Khayundi, Peter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.

I was not aware that the database was creating 20000 separate transactions. If anyone has an idea of how I can modify my code to ensure that I do not create 20000 seperate transactions, please assist me on that.

 

From: William Pietri [mailto:william@...]
Sent: Monday, February 09, 2009 7:42 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database

 

Just to be clear, each object is saved in its own transaction, so that's 20000 separate transactions.

William

Khayundi, Peter wrote:

The objects are created in a single transaction.
 
-----Original Message-----
From: Klaus Wuestefeld [klauswuestefeld@...] 
Sent: Friday, February 06, 2009 11:04 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database
 
Also: Are you creating 20000 objects in a single transaction or in
20000 separate transactions?
 
 
On Fri, Feb 6, 2009 at 6:49 PM, Klaus Wuestefeld
klauswuestefeld@... wrote:
  
Are you doing SCADA or OLTP?
 
 
On Fri, Feb 6, 2009 at 3:24 PM, William Pietri william@...
    
wrote:
  
Khayundi, Peter wrote:
 
I am doing some timings with the Prevayler database. I inserted 20000
      
items
  
into the database and timed this operation. The experiment was
      
repeated 10
  
times. The resulting times differ widely and have a high standard
      
deviation.
  
Does anybody have an explanation as to why the same experiment should
      
have
  
such widely different timings?
 
 
 
Hi, Peter. My first two guesses: other IO on the same disk and Java
      
garbage
  
collection. You can reduce problems with the first by using a
      
separate disk,
  
and with the latter by giving Java lots more RAM than necessary.
      
There are
  
also assorted tunings you can do, but those depend on OS and JDK
      
version.
  
 
William
 
--
William Pietri - william@... - +1-415-643-1024
Agile consulting, coaching, and development: http://www.scissor.com/
We'd love feedback on our new blog: http://agilefocus.com/
 
 
 
 
 
 
      
------------------------------------------------------------------------
------
  
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
      
code
  
to
build responsive, highly engaging applications that combine the power
      
of
  
local
resources and data with the reach of the web. Download the Adobe AIR
      
SDK and
  
Ajax docs to start building applications
      
today-http://p.sf.net/sfu/adobe-com
  
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
 
 
      
 
------------------------------------------------------------------------
------
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
code to
build responsive, highly engaging applications that combine the power of
local
resources and data with the reach of the web. Download the Adobe AIR SDK
and
Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
 
 
 
The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further:
http://www.ufh.ac.za/disclaimer/
 
 
 
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
  

 

The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further: http://www.ufh.ac.za/disclaimer/
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-
http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Creating objects in the database

by William Pietri :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.
If you take a look at your InsertItem object, you'll note that it implements Transaction. When you call "prevayler.execute()" on InsertItem, that's treated as one transaction.

The simple way to make this work is to make InsertItem take an array of items rather than just one, and then you can pass in your whole array of items rather than inserting them one at a time.

William

Khayundi, Peter wrote:

I was not aware that the database was creating 20000 separate transactions. If anyone has an idea of how I can modify my code to ensure that I do not create 20000 seperate transactions, please assist me on that.

 

From: William Pietri [william@...]
Sent: Monday, February 09, 2009 7:42 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database

 

Just to be clear, each object is saved in its own transaction, so that's 20000 separate transactions.

William

Khayundi, Peter wrote:

The objects are created in a single transaction.
 
-----Original Message-----
From: Klaus Wuestefeld [klauswuestefeld@...] 
Sent: Friday, February 06, 2009 11:04 PM
To: Open discussion about the Prevayler project.
Subject: Re: [Prevayler-discussion] Creating objects in the database
 
Also: Are you creating 20000 objects in a single transaction or in
20000 separate transactions?
 
 
On Fri, Feb 6, 2009 at 6:49 PM, Klaus Wuestefeld
klauswuestefeld@... wrote:
  
Are you doing SCADA or OLTP?
 
 
On Fri, Feb 6, 2009 at 3:24 PM, William Pietri william@...
    
wrote:
  
Khayundi, Peter wrote:
 
I am doing some timings with the Prevayler database. I inserted 20000
      
items
  
into the database and timed this operation. The experiment was
      
repeated 10
  
times. The resulting times differ widely and have a high standard
      
deviation.
  
Does anybody have an explanation as to why the same experiment should
      
have
  
such widely different timings?
 
 
 
Hi, Peter. My first two guesses: other IO on the same disk and Java
      
garbage
  
collection. You can reduce problems with the first by using a
      
separate disk,
  
and with the latter by giving Java lots more RAM than necessary.
      
There are
  
also assorted tunings you can do, but those depend on OS and JDK
      
version.
  
 
William
 
--
William Pietri - william@... - +1-415-643-1024
Agile consulting, coaching, and development: http://www.scissor.com/
We'd love feedback on our new blog: http://agilefocus.com/
 
 
 
 
 
 
      
------------------------------------------------------------------------
------
  
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
      
code
  
to
build responsive, highly engaging applications that combine the power
      
of
  
local
resources and data with the reach of the web. Download the Adobe AIR
      
SDK and
  
Ajax docs to start building applications
      
today-http://p.sf.net/sfu/adobe-com
  
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
 
 
      
 
------------------------------------------------------------------------
------
Create and Deploy Rich Internet Apps outside the browser with
Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and
code to
build responsive, highly engaging applications that combine the power of
local
resources and data with the reach of the web. Download the Adobe AIR SDK
and
Ajax docs to start building applications
today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page:
http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
 
 
 
The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further:
http://www.ufh.ac.za/disclaimer/
 
 
 
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org
  

 

The University of Fort Hare subscribes to an e-mail disclaimer. Please click on the following URL to read further: http://www.ufh.ac.za/disclaimer/


------------------------------------------------------------------------------ Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM) software. With Adobe AIR, Ajax developers can use existing skills and code to build responsive, highly engaging applications that combine the power of local resources and data with the reach of the web. Download the Adobe AIR SDK and Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com


_______________________________________________ To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion _______________________________________________ "Databases in Memoriam" -- http://www.prevayler.org


------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org