|
View:
New views
15 Messages
—
Rating Filter:
Alert me
|
|
|
Queueing jobs for processing in their own thread/transaction?I have an EJB that needs to do schedule some work for itself, like sending emails, processing certain records, etc. in their own thread and transaction. The reason I want to move them into their own thread and transaction is to avoid holding locks for too long, avoid deadlocks, and avoid the possibility that a job could throw an exception and cause all the other jobs to be aborted or rolled back. Up until now I've been using the TimerService and just scheduling each job using a timer. However, I'm concerned that if I schedule too many timers I might run into resource issues - for example, if the timers fire at the same time, TimerService might try to create a thread for each one and they would all create a transaction and then connect to the database - it's likely that I'll run out of threads, database connections, memory or sockets. I don't know how to predict or control how many threads the TimerService might create at any time. I thought I could use JMS for this but the JMS implementation with glassfish v2ur2 is too buggy and I can't upgrade to 2.1 due to some weird classpath issues in 2.1 I couldn't resolve. Is there another good way to create a work queue that in processed in a limited number of threads which can access my EJB in a transaction and do some work? Thanks! |
|
|
Re: Queueing jobs for processing in their own thread/transaction?Glassfish does allow you access to the WorkManagerFactory and thus you can create a WorkManager. WorkManager allows you to create a Work object and put it on the queue - just like an HTTP request or an EJB method. Spring has this built into its TaskExecutor framework - with the GlassFishWorkManagerTaskExecutor. Spring extends the objects that you can put on the queue - plain old Runnables and does the conversion into Work objects for you. I believe that this will be part of the Java EE 6 spec but GF lets you access the objects now. We use this for background import tasks. Works like a champ. Brian ----- Original message ----- From: "Dobes Vandermeer" <dobesv@...> To: users@... Date: Mon, 2 Feb 2009 10:18:33 -0800 (PST) Subject: Queueing jobs for processing in their own thread/transaction? I have an EJB that needs to do schedule some work for itself, like sending emails, processing certain records, etc. in their own thread and transaction. The reason I want to move them into their own thread and transaction is to avoid holding locks for too long, avoid deadlocks, and avoid the possibility that a job could throw an exception and cause all the other jobs to be aborted or rolled back. Up until now I've been using the TimerService and just scheduling each job using a timer. However, I'm concerned that if I schedule too many timers I might run into resource issues - for example, if the timers fire at the same time, TimerService might try to create a thread for each one and they would all create a transaction and then connect to the database - it's likely that I'll run out of threads, database connections, memory or sockets. I don't know how to predict or control how many threads the TimerService might create at any time. I thought I could use JMS for this but the JMS implementation with glassfish v2ur2 is too buggy and I can't upgrade to 2.1 due to some weird classpath issues in 2.1 I couldn't resolve. Is there another good way to create a work queue that in processed in a limited number of threads which can access my EJB in a transaction and do some work? Thanks! -- View this message in context: http://www.nabble.com/Queueing-jobs-for-processing-in-their-own-thread-transaction--tp21795170p21795170.html Sent from the java.net - glassfish users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?Okay, this looked really good but I'm having trouble accessing the database inside my Work object.
The errors I'm getting are: org.hibernate.LazyInitializationException: could not initialize proxy - no Session or IllegalArgumentException: entity not in the persistence context Strangely I am not re-using objects from the other thread (that I know of) I get the object using em.get() and then try to use that object. Is there anything special I need to do in the Work object to set up the persistence context and session? Does it need to construct its own copy of the stateless beans and EntityManager somehow?
|
|
|
Re: Queueing jobs for processing in their own thread/transaction?Interesting, I'm not using EJB3.0 - I'm using Spring so all I need is to get my ApplicationContext and I'm off and running. The LazyInitEx is definately related to walking the object graph after the session is closed. My guess is that putting Work on the queue doesn't get container objects injected into it - you will probably need to get them some other way. Sorry - I don't know how to do that - anyone else? Brian ----- Original message ----- From: "Dobes Vandermeer" <dobesv@...> To: users@... Date: Tue, 3 Feb 2009 11:39:37 -0800 (PST) Subject: Re: Queueing jobs for processing in their own thread/transaction? Okay, this looked really good but I'm having trouble accessing the database inside my Work object. The errors I'm getting are: org.hibernate.LazyInitializationException: could not initialize proxy - no Session or IllegalArgumentException: entity not in the persistence context Strangely I am not re-using objects from the other thread (that I know of) I get the object using em.get() and then try to use that object. Is there anything special I need to do in the Work object to set up the persistence context and session? Does it need to construct its own copy of the stateless beans and EntityManager somehow? --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?It might be impossible. I've given up on the JMS stuff since it was buggy and the WorkManager doesn't support transactions as far as I can tell. However, now I have a new issue - when my timers run (I run a bunch of them to keep operation separate) the server and database start paging due to high memory use. Apparently I *do* need a way to restrict the number of these threads that are running.
Maybe I'll go poking around in the glassfish source code and see if there's some private API I can use to workaround this and create a new transactional session in my Work instance somehow. I already poked around a bit but got scared of the code there, the code to invoke a timeout, which I thought would be a good example, seems to be pretty involved stuff. Nevertheless, having a utility class to run these jobs in a thread pool of my choosing would be very, very useful so it might be worth some brain pain to decipher the complexity of that subsystem.
|
|
|
Re: Queueing jobs for processing in their own thread/transaction?Dobes, I would really recommend that you look at using Spring. Transactional stuff works - even against JTA transaction manager. I don't know about the timeouts/timers (lots of questions/issues posted here about EJB Timers) but you might look at Quartz with the Spring TaskExecutor framework on top of that. Your work is like a servlet - nothing transactional about it but it can start one. You just don't get container injection. But again, you can use Spring for that. Not sure if that is an option for you but I have similar requirements and things just work if I don't rely on the container for it ;-) Brian ----- Original message ----- From: "Dobes Vandermeer" <dobesv@...> To: users@... Date: Mon, 9 Feb 2009 20:50:37 -0800 (PST) Subject: Re: Queueing jobs for processing in their own thread/transaction? It might be impossible. I've given up on the JMS stuff since it was buggy and the WorkManager doesn't support transactions as far as I can tell. However, now I have a new issue - when my timers run (I run a bunch of them to keep operation separate) the server and database start paging due to high memory use. Apparently I *do* need a way to restrict the number of these threads that are running. Maybe I'll go poking around in the glassfish source code and see if there's some private API I can use to workaround this and create a new transactional session in my Work instance somehow. I already poked around a bit but got scared of the code there, the code to invoke a timeout, which I thought would be a good example, seems to be pretty involved stuff. Nevertheless, having a utility class to run these jobs in a thread pool of my choosing would be very, very useful so it might be worth some brain pain to decipher the complexity of that subsystem. Brian Repko wrote: > > > Interesting, I'm not using EJB3.0 - I'm using Spring so all I need is to > get my ApplicationContext and I'm off and running. The LazyInitEx is > definately related to walking the object graph after the session is > closed. > > My guess is that putting Work on the queue doesn't get container objects > injected into it - you will probably need to get them some other way. > > Sorry - I don't know how to do that - anyone else? > > Brian > > ----- Original message ----- > From: "Dobes Vandermeer" <dobesv@...> > To: users@... > Date: Tue, 3 Feb 2009 11:39:37 -0800 (PST) > Subject: Re: Queueing jobs for processing in their own thread/transaction? > > > Okay, this looked really good but I'm having trouble accessing the > database > inside my Work object. > > The errors I'm getting are: > > org.hibernate.LazyInitializationException: could not initialize proxy - no > Session > > or > > IllegalArgumentException: entity not in the persistence context > > Strangely I am not re-using objects from the other thread (that I know of) > I > get the object using em.get() and then try to use that object. > > Is there anything special I need to do in the Work object to set up the > persistence context and session? Does it need to construct its own copy > of > the stateless beans and EntityManager somehow? > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@... > For additional commands, e-mail: users-help@... > > > -- View this message in context: http://www.nabble.com/Queueing-jobs-for-processing-in-their-own-thread-transaction--tp21795170p21927689.html Sent from the java.net - glassfish users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?After reading some more docs I realized that I can use the UserTransaction API to create a transaction, I just needed to inject it into the bean starting the work. I don't know if the bean and the UserTransaction are guaranteed to be valid for use when the Work is run, though - the bean could be passivated or something. I wonder if anyone knows whether it's okay to keep a direct reference to a Stateless EJB from a Work instance this way?
|
|
|
Re: Queueing jobs for processing in their own thread/transaction?> I thought I could use JMS for this but the JMS
> implementation with glassfish > v2ur2 is too buggy and I can't upgrade to 2.1 due to > some weird classpath > issues in 2.1 I couldn't resolve. What issues are you having with JMS? Seems to me that JMS with a constrained MDB (i.e. container can not make more than, say, 5 instances) would work well at letting you control resource uses. And this doesn't sound like a particularly exotic use of JMS. So I'm curious what problems you're encountering with it. [Message sent by forum member 'whartung' (whartung)] http://forums.java.net/jive/thread.jspa?messageID=331076 --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?I know we have to move our file import processes to WorkManager since we have a cluster and don't know which machine gets the file and thus had to send the message to a Topic. Only MDBs on a topic in a cluster will fire on only one of the servers (they effectively act like a queue). Still broken in 2.1 and JMS is not implemented yet in 3.0. Brian ----- Original message ----- From: glassfish@... To: users@... Date: Tue, 10 Feb 2009 10:28:57 PST Subject: Re: Queueing jobs for processing in their own thread/transaction? > I thought I could use JMS for this but the JMS > implementation with glassfish > v2ur2 is too buggy and I can't upgrade to 2.1 due to > some weird classpath > issues in 2.1 I couldn't resolve. What issues are you having with JMS? Seems to me that JMS with a constrained MDB (i.e. container can not make more than, say, 5 instances) would work well at letting you control resource uses. And this doesn't sound like a particularly exotic use of JMS. So I'm curious what problems you're encountering with it. [Message sent by forum member 'whartung' (whartung)] http://forums.java.net/jive/thread.jspa?messageID=331076 --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?This is the sort of thing JMS would be perfect for, if it worked. I was getting a lot of errors in the logs - for each message delivered it would print an error every once in a while (maybe 30 seconds?) and as more messages were sent the volume of errors increased. Generally, it's just a flaky unfinished product at the stage I was using it, for example when I first set up the Queue I got this: MQJMSRA_MC2001: createConnection API used w/ username, password for Container Auth Apparently I had to delete the username and password properties from the queue in the admin panel. Just something that shows a lack of usability testing. Why not just treat an empty username/password the same as a null one? Anyway, the actual error I was getting is discussed in this thread, I believe (it's been a couple weeks so I've gotten foggy on the exact error text): http://forums.java.net/jive/thread.jspa?messageID=278700 For me, the suggested workaround of changing the connection type to LOCAL did not solve the problem, things actually got worse, so I gave up on the JMS solution. |
|
|
Re: Queueing jobs for processing in their own thread/transaction?Its not the OpenMQ product. Its the resource adapters. jmsra, genericjmsra and sun-jms-adapter - none of them worked for us. I have a hard time believing that getting Sailfin working was a higher priority than a production ready JMS configuration. ----- Original message ----- From: "Dobes Vandermeer" <dobesv@...> To: users@... Date: Tue, 10 Feb 2009 13:57:54 -0800 (PST) Subject: Re: Queueing jobs for processing in their own thread/transaction? glassfish-2 wrote: > >> I thought I could use JMS for this but the JMS >> implementation with glassfish >> v2ur2 is too buggy and I can't upgrade to 2.1 due to >> some weird classpath >> issues in 2.1 I couldn't resolve. > > What issues are you having with JMS? Seems to me that JMS with a > constrained MDB (i.e. container can not make more than, say, 5 instances) > would work well at letting you control resource uses. And this doesn't > sound like a particularly exotic use of JMS. > > So I'm curious what problems you're encountering with it. > This is the sort of thing JMS would be perfect for, if it worked. I was getting a lot of errors in the logs - for each message delivered it would print an error every once in a while (maybe 30 seconds?) and as more messages were sent the volume of errors increased. Generally, it's just a flaky unfinished product at the stage I was using it, for example when I first set up the Queue I got this: MQJMSRA_MC2001: createConnection API used w/ username, password for Container Auth Apparently I had to delete the username and password properties from the queue in the admin panel. Just something that shows a lack of usability testing. Why not just treat an empty username/password the same as a null one? Anyway, the actual error I was getting is discussed in this thread, I believe (it's been a couple weeks so I've gotten foggy on the exact error text): http://forums.java.net/jive/thread.jspa?messageID=278700 For me, the suggested workaround of changing the connection type to LOCAL did not solve the problem, things actually got worse, so I gave up on the JMS solution. -- View this message in context: http://www.nabble.com/Queueing-jobs-for-processing-in-their-own-thread-transaction--tp21795170p21943493.html Sent from the java.net - glassfish users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?Yeah, it's a bit disappointing. Hopefully the situation will improve, but for now we just have to work around the issues or use another container, right? I don't think I have the ability to fix the resource adapters myself right now.
|
|
|
Re: Queueing jobs for processing in their own thread/transaction?Ok.
I know I have a queue and a topic working in JMS. We're constantly getting something like: [#|2009-02-10T13:47:58.835-0800|INFO|sun-appserver9.1|javax.resourceadapter.mqjm sra.outbound.connection|_ThreadID=20;_ThreadName=Timer-8;|MQJMSRA_MC1101: constr uctor:Created mcId=114766:xacId=4133494539453853184:Using xacf config={imqDefaul tUsername=guest, imqBrokerHostName=localhost, imqSetJMSXUserID=false, imqJMSExpi ration=0, imqQueueBrowserRetrieveTimeout=60000, imqSSLProviderClassname=com.sun. net.ssl.internal.ssl.Provider, imqConfiguredClientID=, imqSSLIsHostTrusted=false , imqSetJMSXProducerTXID=false, imqJMSDeliveryMode=PERSISTENT, imqAddressListBeh avior=RANDOM, imqEnableSharedClientID=false, imqAckOnProduce=, imqConnectionURL= http://localhost/imq/tunnel, imqConnectionFlowLimit=1000, imqSetJMSXAppID=false, imqAddressListIterations=3, imqBrokerServicePort=0, imqSetJMSXRcvTimestamp=fals e, imqReconnectInterval=5000, imqConnectionType=TCP, imqLoadMaxToServerSession=t rue, imqSetJMSXConsumerTXID=false, imqAddressList=localhost:7676, imqOverrideJMS DeliveryMode=false, imqPingInterval=30, imqDefaultPassword=guest, imqAckOnAcknow ledge=, imqJMSPriority=4, imqQueueBrowserMaxMessagesPerRetrieve=1000, imqOverrid eJMSPriority=false, imqConsumerFlowLimit=1000, imqBrokerHostPort=7676, imqReconn ectEnabled=true, imqConnectionFlowCount=100, imqReconnectAttempts=3, imqConnecti onFlowLimitEnabled=false, imqAckTimeout=0, imqBrokerServiceName=, imqOverrideJMS Expiration=false, imqConnectionHandler=com.sun.messaging.jmq.jmsclient.protocol. tcp.TCPStreamHandler, imqDisableSetClientID=false, imqOverrideJMSHeadersToTempor aryDestinations=false, imqConsumerFlowThreshold=50}|#] spammed in our logs, but my understanding is that this is harmless. I also don't drive it super hard, but it is reliable (it's mostly for event notifications). I, also, had to set it LOCAL. It flat out didn't work otherwise. Besides allowing only a single MDB on the Queue, it's pretty much generic and default all the way through. We're running: Sun Java System Application Server 9.1_02 (build b04-fcs) Don't know if that's helpful at all... I agree that the connector should "just work". [Message sent by forum member 'whartung' (whartung)] http://forums.java.net/jive/thread.jspa?messageID=331148 --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?If you can get a hold of them they should work. Both, Stateless EJB and
UserTransaction can be stored in an instance variable because they are proxies to the real objects. Regards, -marina Dobes Vandermeer wrote: > After reading some more docs I realized that I can use the UserTransaction > API to create a transaction, I just needed to inject it into the bean > starting the work. I don't know if the bean and the UserTransaction are > guaranteed to be valid for use when the Work is run, though - the bean could > be passivated or something. I wonder if anyone knows whether it's okay to > keep a direct reference to a Stateless EJB from a Work instance this way? > > > > Brian Repko wrote: > >> >>Dobes, >> >>I would really recommend that you look at using Spring. Transactional >>stuff works - even against JTA transaction manager. I don't know about >>the timeouts/timers (lots of questions/issues posted here about EJB >>Timers) but you might look at Quartz with the Spring TaskExecutor >>framework on top of that. >> >>Your work is like a servlet - nothing transactional about it but it can >>start one. You just don't get container injection. But again, you can >>use Spring for that. >> >>Not sure if that is an option for you but I have similar requirements >>and things just work if I don't rely on the container for it ;-) >> >>Brian >> >> >>----- Original message ----- >>From: "Dobes Vandermeer" <dobesv@...> >>To: users@... >>Date: Mon, 9 Feb 2009 20:50:37 -0800 (PST) >>Subject: Re: Queueing jobs for processing in their own thread/transaction? >> >> >>It might be impossible. I've given up on the JMS stuff since it was buggy >>and the WorkManager doesn't support transactions as far as I can tell. >>However, now I have a new issue - when my timers run (I run a bunch of >>them >>to keep operation separate) the server and database start paging due to >>high >>memory use. Apparently I *do* need a way to restrict the number of these >>threads that are running. >> >>Maybe I'll go poking around in the glassfish source code and see if >>there's >>some private API I can use to workaround this and create a new >>transactional >>session in my Work instance somehow. I already poked around a bit but got >>scared of the code there, the code to invoke a timeout, which I thought >>would be a good example, seems to be pretty involved stuff. Nevertheless, >>having a utility class to run these jobs in a thread pool of my choosing >>would be very, very useful so it might be worth some brain pain to >>decipher >>the complexity of that subsystem. >> >> >> >>Brian Repko wrote: >> >>> >>>Interesting, I'm not using EJB3.0 - I'm using Spring so all I need is to >>>get my ApplicationContext and I'm off and running. The LazyInitEx is >>>definately related to walking the object graph after the session is >>>closed. >>> >>>My guess is that putting Work on the queue doesn't get container objects >>>injected into it - you will probably need to get them some other way. >>> >>>Sorry - I don't know how to do that - anyone else? >>> >>>Brian >>> >>>----- Original message ----- >>>From: "Dobes Vandermeer" <dobesv@...> >>>To: users@... >>>Date: Tue, 3 Feb 2009 11:39:37 -0800 (PST) >>>Subject: Re: Queueing jobs for processing in their own >>>thread/transaction? >>> >>> >>>Okay, this looked really good but I'm having trouble accessing the >>>database >>>inside my Work object. >>> >>>The errors I'm getting are: >>> >>>org.hibernate.LazyInitializationException: could not initialize proxy - >>>no >>>Session >>> >>>or >>> >>>IllegalArgumentException: entity not in the persistence context >>> >>>Strangely I am not re-using objects from the other thread (that I know >>>of) >>>I >>>get the object using em.get() and then try to use that object. >>> >>>Is there anything special I need to do in the Work object to set up the >>>persistence context and session? Does it need to construct its own copy >>>of >>>the stateless beans and EntityManager somehow? >>> >>> >>> >>>--------------------------------------------------------------------- >>>To unsubscribe, e-mail: users-unsubscribe@... >>>For additional commands, e-mail: users-help@... >>> >>> >>> >> >>-- >>View this message in context: >>http://www.nabble.com/Queueing-jobs-for-processing-in-their-own-thread-transaction--tp21795170p21927689.html >>Sent from the java.net - glassfish users mailing list archive at >>Nabble.com. >> >> >>--------------------------------------------------------------------- >>To unsubscribe, e-mail: users-unsubscribe@... >>For additional commands, e-mail: users-help@... >> >> >>--------------------------------------------------------------------- >>To unsubscribe, e-mail: users-unsubscribe@... >>For additional commands, e-mail: users-help@... >> >> >> > > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@... For additional commands, e-mail: users-help@... |
|
|
Re: Queueing jobs for processing in their own thread/transaction?Thx, Satish Dobes Vandermeer wrote: glassfish-2 wrote: |
| Free embeddable forum powered by Nabble | Forum Help |