Speed problems with 1.1.3

[KARASZI Istvan]

dear list!

we are developing a web application which is deployed to a GlassFish  
AS and communicating with a PostgreSQL database through EclipseLink.  
two days ago we were in a hurry of a release, on the night of the  
release we have found out that the production server is really slow  
comparing to the test server with the same code on the same HW with  
the same OS (kernel + libraries).

the last two days I was to find out the cause of the problem. I  
checked the HW, ran a lot of benchmarks and profiling, compared the  
versions of the libraries, checked the kernels, etc. in the morning  
I've found out that the problem was with the EclipseLink. on the test  
server the version was 1.1.2 which was running smoothly and fast. and  
on the production server was version 1.1.3 which was running slow  
(considerably - about 20x slower).

i haven't thought that the newer version of EclipseLink could cause  
the speed problems. have you ever experienced problems like this with  
this version? or could you give me ideas why it is so slow with the  
new version?

if you need any more information about the environment, let me know!

thanks,
--
KARASZI Istvan
_______________________________________________
eclipselink-users mailing list
eclipselink-users@...
https://dev.eclipse.org/mailman/listinfo/eclipselink-users

[James Sutherland]

I compared the source code between the two releases, and could not see anything that changed that could cause this.

Can you identify what is 20x slower, i.e. reading, inserts, updates, deletes.  Or narrow down what exactly is slower.

Are you using the same JDK on both machines?  Are you weaving the same on both machines?  Check that weaving is functioning on the slow machine.

A specific test/use case, or profile would be useful.

KARASZI Istvan wrote:

dear list!

we are developing a web application which is deployed to a GlassFish  
AS and communicating with a PostgreSQL database through EclipseLink.  
two days ago we were in a hurry of a release, on the night of the  
release we have found out that the production server is really slow  
comparing to the test server with the same code on the same HW with  
the same OS (kernel + libraries).

the last two days I was to find out the cause of the problem. I  
checked the HW, ran a lot of benchmarks and profiling, compared the  
versions of the libraries, checked the kernels, etc. in the morning  
I've found out that the problem was with the EclipseLink. on the test  
server the version was 1.1.2 which was running smoothly and fast. and  
on the production server was version 1.1.3 which was running slow  
(considerably - about 20x slower).

i haven't thought that the newer version of EclipseLink could cause  
the speed problems. have you ever experienced problems like this with  
this version? or could you give me ideas why it is so slow with the  
new version?

if you need any more information about the environment, let me know!

thanks,
--
KARASZI Istvan

[KARASZI Istvan]

hi!

On Nov 3, 2009, at 8:40 PM, James Sutherland wrote:
> I compared the source code between the two releases, and could not see
> anything that changed that could cause this.
that sad, because something between 1.1.2 and 1.1.3 caused this, I'm  
sure.

> Can you identify what is 20x slower, i.e. reading, inserts, updates,
> deletes.  Or narrow down what exactly is slower.
we've tried. I ran multiple tests including a profiler (visualvm), but  
the calls look the same only slower, so there was no methods which was  
waiting too much or using to much CPU.

I'll try to get a proper profiling data from the machine soon.

in the part we are testing the slowness there are almost(!) only  
inserts, a lot of inserts (about 90k). I'm not saying that the inserts  
are slow only, but in that part it seems like.

> Are you using the same JDK on both machines?  Are you weaving the  
> same on
> both machines?  Check that weaving is functioning on the slow machine.
first it was different, but later I changed everything to the same. so  
it's not depending on the JDK.

but to make it sure, we put back the eclipselink 1.1.3 to the fast  
machine it slowed down tha fast machine too. so the only difference is  
the eclipselink. that's why I sent this mail to the list previously.

> A specific test/use case, or profile would be useful.

yes it would be nice, but the app is too complicated. I can give you  
an account to test if if you feel like. it's absolutely reproducible.

I'll try to create a testcase but it will tak a while.

thanks,
--
KARASZI Istvan
_______________________________________________
eclipselink-users mailing list
eclipselink-users@...
https://dev.eclipse.org/mailman/listinfo/eclipselink-users

[James Sutherland]

Would it be possible to try the 1.2 release, and see if the same issue exists?

Could you also try something that is only reads, and see if it has the same issue.

KARASZI Istvan wrote:

hi!

On Nov 3, 2009, at 8:40 PM, James Sutherland wrote:
> I compared the source code between the two releases, and could not see
> anything that changed that could cause this.
that sad, because something between 1.1.2 and 1.1.3 caused this, I'm  
sure.

> Can you identify what is 20x slower, i.e. reading, inserts, updates,
> deletes.  Or narrow down what exactly is slower.
we've tried. I ran multiple tests including a profiler (visualvm), but  
the calls look the same only slower, so there was no methods which was  
waiting too much or using to much CPU.

I'll try to get a proper profiling data from the machine soon.

in the part we are testing the slowness there are almost(!) only  
inserts, a lot of inserts (about 90k). I'm not saying that the inserts  
are slow only, but in that part it seems like.

> Are you using the same JDK on both machines?  Are you weaving the  
> same on
> both machines?  Check that weaving is functioning on the slow machine.
first it was different, but later I changed everything to the same. so  
it's not depending on the JDK.

but to make it sure, we put back the eclipselink 1.1.3 to the fast  
machine it slowed down tha fast machine too. so the only difference is  
the eclipselink. that's why I sent this mail to the list previously.

> A specific test/use case, or profile would be useful.

yes it would be nice, but the app is too complicated. I can give you  
an account to test if if you feel like. it's absolutely reproducible.

I'll try to create a testcase but it will tak a while.

thanks,
--
KARASZI Istvan

[KARASZI Istvan]

On Nov 4, 2009, at 3:20 PM, James Sutherland wrote:
> Would it be possible to try the 1.2 release, and see if the same issue
> exists?
since there is a design flaw, we could not upgrade straight to 1.2.  
without modifying the source. so I could not test this right now. but  
we are on it!

> Could you also try something that is only reads, and see if it has  
> the same
> issue.
I'll.

--
KARASZI Istvan
_______________________________________________
eclipselink-users mailing list
eclipselink-users@...
https://dev.eclipse.org/mailman/listinfo/eclipselink-users

[James Sutherland]

I downloaded the 1.1.2 and 1.1.3 jars and ran our JPA mass insert performance test on both jars.
The results were the same, so it must be something specific to your environment or your usage.

KARASZI Istvan wrote:

On Nov 4, 2009, at 3:20 PM, James Sutherland wrote:
> Would it be possible to try the 1.2 release, and see if the same issue
> exists?
since there is a design flaw, we could not upgrade straight to 1.2.  
without modifying the source. so I could not test this right now. but  
we are on it!

> Could you also try something that is only reads, and see if it has  
> the same
> issue.
I'll.

--
KARASZI Istvan

[Sebastien Tardif-2]

James,

You seems to care about performance, could you please fix the two
following issues which are proven to be a bottleneck in our application?

https://bugs.eclipse.org/bugs/show_bug.cgi?id=288864 CacheKey.equals
could be faster
https://bugs.eclipse.org/bugs/show_bug.cgi?id=294259 Duplicate existence
checks in same UOW

And evaluate:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=289402 Bottleneck in
ExpressionOperator calling getClass() more often than needed




-----Original Message-----
From: eclipselink-users-bounces@...
[mailto:eclipselink-users-bounces@...] On Behalf Of James
Sutherland
Sent: Wednesday, November 04, 2009 1:43 PM
To: eclipselink-users@...
Subject: Re: [eclipselink-users] Speed problems with 1.1.3


I downloaded the 1.1.2 and 1.1.3 jars and ran our JPA mass insert
performance
test on both jars.
The results were the same, so it must be something specific to your
environment or your usage.



KARASZI Istvan wrote:
>
> On Nov 4, 2009, at 3:20 PM, James Sutherland wrote:
>> Would it be possible to try the 1.2 release, and see if the same
issue
>> exists?
> since there is a design flaw, we could not upgrade straight to 1.2.  
> without modifying the source. so I could not test this right now. but

> we are on it!
>
>> Could you also try something that is only reads, and see if it has  
>> the same
>> issue.
> I'll.
>
> --
> KARASZI Istvan
>
>

-----
http://wiki.eclipse.org/User:James.sutherland.oracle.com James
Sutherland
http://www.eclipse.org/eclipselink/
 EclipseLink ,  http://www.oracle.com/technology/products/ias/toplink/
TopLink
Wiki:  http://wiki.eclipse.org/EclipseLink EclipseLink ,
http://wiki.oracle.com/page/TopLink TopLink
Forums:  http://forums.oracle.com/forums/forum.jspa?forumID=48 TopLink ,

http://www.nabble.com/EclipseLink-f26430.html EclipseLink
Book:  http://en.wikibooks.org/wiki/Java_Persistence Java Persistence
--
View this message in context:
http://old.nabble.com/Speed-problems-with-1.1.3-tp26141846p26202278.html
Sent from the EclipseLink - Users mailing list archive at Nabble.com.

_______________________________________________
eclipselink-users mailing list
eclipselink-users@...
https://dev.eclipse.org/mailman/listinfo/eclipselink-users
_______________________________________________
eclipselink-users mailing list
eclipselink-users@...
https://dev.eclipse.org/mailman/listinfo/eclipselink-users