|
»
»
»
memory usage
|
View:
New views
12 Messages
—
Rating Filter:
Alert me
|
 |
memory usage
by DavidParker
::
Rate this Message:
I am running a long data import that uses eclipselink 1.1.2 for
persistence. I see memory increasing slowly but surely as it goes along. The YourKit profiler does not identify anything as a link, but I do seem to have an increasing "retained size" for org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork [Stack Local] I see there is a clear method on TransactionWrapperImpl, but that does not seem to be in the JPA EntityTransaction API. Is there a best practice for long running data imports? Should I be clearing something? Thanks. ============================================= David Parker dap@... _______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by tch
::
Rate this Message:
I don't have a good answer, I'm just curious as to why in the world
you'd want to use JPA for this sort of job? ./tch On Mon, Oct 19, 2009 at 5:51 PM, David Parker <dap@...> wrote: > I am running a long data import that uses eclipselink 1.1.2 for persistence. > I see memory increasing slowly but surely as it goes along. The YourKit > profiler does not identify anything as a link, but I do seem to have an > increasing "retained size" for > > org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork [Stack > Local] > > I see there is a clear method on TransactionWrapperImpl, but that does not > seem to be in the JPA EntityTransaction API. > > Is there a best practice for long running data imports? Should I be clearing > something? > > Thanks. > > > ============================================= > David Parker > dap@... > > > > > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@... > https://dev.eclipse.org/mailman/listinfo/eclipselink-users > eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
RE: memory usage
by Dave Brosius-2
::
Rate this Message:
Is the import all in one transaction? I'd be curious if you broke up the import (even for a test) into multiple transactions whether the issue would go away.
-----Original Message----- From: "David Parker" <dap@...> Sent: Monday, October 19, 2009 5:51pm To: "EclipseLink User Discussions" <eclipselink-users@...> Subject: [eclipselink-users] memory usage I am running a long data import that uses eclipselink 1.1.2 for persistence. I see memory increasing slowly but surely as it goes along. The YourKit profiler does not identify anything as a link, but I do seem to have an increasing "retained size" for org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork [Stack Local] I see there is a clear method on TransactionWrapperImpl, but that does not seem to be in the JPA EntityTransaction API. Is there a best practice for long running data imports? Should I be clearing something? Thanks. ============================================= David Parker dap@... _______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users _______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by DavidParker
::
Rate this Message:
Because our application is built in terms of JPA Objects. This is a
small part of our app, a data conversion from another version of our system. Your response is almost like asking why I didn't write it in perl...or maybe that is what you are suggesting? On Oct 19, 2009, at 5:52 PM, Tim Hollosy wrote: > I don't have a good answer, I'm just curious as to why in the world > you'd want to use JPA for this sort of job? > > ./tch > > > > On Mon, Oct 19, 2009 at 5:51 PM, David Parker <dap@... > > wrote: >> I am running a long data import that uses eclipselink 1.1.2 for >> persistence. >> I see memory increasing slowly but surely as it goes along. The >> YourKit >> profiler does not identify anything as a link, but I do seem to >> have an >> increasing "retained size" for >> >> org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork >> [Stack >> Local] >> >> I see there is a clear method on TransactionWrapperImpl, but that >> does not >> seem to be in the JPA EntityTransaction API. >> >> Is there a best practice for long running data imports? Should I be >> clearing >> something? >> >> Thanks. >> >> >> ============================================= >> David Parker >> dap@... >> >> >> >> >> _______________________________________________ >> eclipselink-users mailing list >> eclipselink-users@... >> https://dev.eclipse.org/mailman/listinfo/eclipselink-users >> > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@... > https://dev.eclipse.org/mailman/listinfo/eclipselink-users > ============================================= David Parker dap@... _______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by DavidParker
::
Rate this Message:
Yeah, we are using multiple transactions, i.e.
get an EntityTransaction do work EntityTransation.commit() set the local tx to null On Oct 19, 2009, at 5:59 PM, Dave Brosius wrote: Is the import all in one transaction? I'd be curious if you broke up the import (even for a test) into multiple transactions whether the issue would go away. _______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by tch
::
Rate this Message:
I didn't mean for you to take it that way, I just meant what benefit
do you get besides Entity re-use? Unless it's the sort of thing that happens enough that you'd want to keep the shared cache up to date I can't think of a great reason to use JPA for it with the memory/performance hit you take registering objects in the cache, constructing new instances, etc. ./tch On Mon, Oct 19, 2009 at 6:05 PM, David Parker <dap@...> wrote: > Because our application is built in terms of JPA Objects. This is a small > part of our app, a data conversion from another version of our system. Your > response is almost like asking why I didn't write it in perl...or maybe that > is what you are suggesting? > > On Oct 19, 2009, at 5:52 PM, Tim Hollosy wrote: > >> I don't have a good answer, I'm just curious as to why in the world >> you'd want to use JPA for this sort of job? >> >> ./tch >> >> >> >> On Mon, Oct 19, 2009 at 5:51 PM, David Parker <dap@...> >> wrote: >>> >>> I am running a long data import that uses eclipselink 1.1.2 for >>> persistence. >>> I see memory increasing slowly but surely as it goes along. The YourKit >>> profiler does not identify anything as a link, but I do seem to have an >>> increasing "retained size" for >>> >>> org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork >>> [Stack >>> Local] >>> >>> I see there is a clear method on TransactionWrapperImpl, but that does >>> not >>> seem to be in the JPA EntityTransaction API. >>> >>> Is there a best practice for long running data imports? Should I be >>> clearing >>> something? >>> >>> Thanks. >>> >>> >>> ============================================= >>> David Parker >>> dap@... >>> >>> >>> >>> >>> _______________________________________________ >>> eclipselink-users mailing list >>> eclipselink-users@... >>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users >>> >> _______________________________________________ >> eclipselink-users mailing list >> eclipselink-users@... >> https://dev.eclipse.org/mailman/listinfo/eclipselink-users >> > > ============================================= > David Parker > dap@... > > > > > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@... > https://dev.eclipse.org/mailman/listinfo/eclipselink-users > eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by tch
::
Rate this Message:
One other thing to look at is batch writing:
http://wiki.eclipse.org/Optimizing_the_EclipseLink_Application_%28ELUG%29#How_to_Use_Batch_Writing_for_Optimization Not sure if it will solve the memory issues, but may help. ./tch On Mon, Oct 19, 2009 at 6:07 PM, David Parker <dap@...> wrote: > Yeah, we are using multiple transactions, i.e. > get an EntityTransaction > do work > EntityTransation.commit() > set the local tx to null > > On Oct 19, 2009, at 5:59 PM, Dave Brosius wrote: > > Is the import all in one transaction? I'd be curious if you broke up the > import (even for a test) into multiple transactions whether the issue would > go away. > > -----Original Message----- > From: "David Parker" <dap@...> > Sent: Monday, October 19, 2009 5:51pm > To: "EclipseLink User Discussions" <eclipselink-users@...> > Subject: [eclipselink-users] memory usage > > I am running a long data import that uses eclipselink 1.1.2 for > persistence. I see memory increasing slowly but surely as it goes > along. The YourKit profiler does not identify anything as a link, but > I do seem to have an increasing "retained size" for > > org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork > [Stack Local] > > I see there is a clear method on TransactionWrapperImpl, but that does > not seem to be in the JPA EntityTransaction API. > > Is there a best practice for long running data imports? Should I be > clearing something? > > Thanks. > > > ============================================= > David Parker > dap@... > > > > > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@... > https://dev.eclipse.org/mailman/listinfo/eclipselink-users > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@... > https://dev.eclipse.org/mailman/listinfo/eclipselink-users > > ============================================= > David Parker > dap@... > > > > > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@... > https://dev.eclipse.org/mailman/listinfo/eclipselink-users > > eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by James Sutherland
::
Rate this Message:
Are you using JTA, or RESOURCE_LOCAL?
Are you using the same EntityManager, or do you create a new one for each transaction? What cache setting are you using? I would recommend you create a new EntityManager per transaction, otherwise the previous objects will still be part of the persistence context. You could also call clear(). You can try disabling the cache (shared=false), or using a weak cache. Do you run out of memory, or just notice a memory increase? You may wish to try a memory profiler such as JProfiler. Also ensure that your application is not holding onto any or the objects.
James Sutherland EclipseLink, TopLink Wiki: EclipseLink, TopLink Forums: TopLink, EclipseLink Book: Java Persistence |
|
|
Re: memory usage
by Gordon Yorke-2
::
Rate this Message:
Is it possible you are seeing this issue: http://bugs.eclipse.org/285599
--Gordon David Parker wrote: > I am running a long data import that uses eclipselink 1.1.2 for > persistence. I see memory increasing slowly but surely as it goes > along. The YourKit profiler does not identify anything as a link, but > I do seem to have an increasing "retained size" for > > org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork > [Stack Local] > > I see there is a clear method on TransactionWrapperImpl, but that does > not seem to be in the JPA EntityTransaction API. > > Is there a best practice for long running data imports? Should I be > clearing something? > > Thanks. > > > ============================================= > David Parker > dap@... > > > > > _______________________________________________ > eclipselink-users mailing list > eclipselink-users@... > https://dev.eclipse.org/mailman/listinfo/eclipselink-users eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by Gordon Yorke-2
::
Rate this Message:
Also make sure you are either getting an Container Managed
EntityManager, or you are either clearing the ApplicationManaged
EntityManager or creating a new Application Managed Entity Manager
after commit() otherwise the Extended Persistence Context (Entity
Manager) will continue to grow in size as more objects become managed.
--Gordon David Parker wrote: Yeah, we are using multiple transactions, i.e. _______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by DavidParker
::
Rate this Message:
Thanks, everybody, for all the responses. I appreciate your help.
This thing runs as a big loop sucking records out of one database via a simple JDBC Connection, and for each record creating the JPA objects for that record, and persisting them. As part of creating the JPA objects there is some querying against the target database via SQL queries. There is one EntityManager, and a single transaction per record.
will fix the problem. The YourKit profiler doesn't flag much it thinks is a leal - just a few k.
_______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
|
|
Re: memory usage
by DavidParker
::
Rate this Message:
I added a call to EntityManager.clear() after each transaction, but these object arrays still seem to be piling up. I am doing another test run with tracking allocations turned on, so hopefully I can pin down better where these objects are being allocated.
Does EntityManager.clear() clear the whole cache? I don't see a clear() method on EntityManagerFactory, which I understand wraps the ServerSession - is there something I can try re-setting the whole thing? Thanks. - DAP On Oct 20, 2009, at 11:57 AM, David Parker wrote:
_______________________________________________ eclipselink-users mailing list eclipselink-users@... https://dev.eclipse.org/mailman/listinfo/eclipselink-users |
| Free embeddable forum powered by Nabble | Forum Help |
