deployment taking 45 minutes (a very long time)

[glassfish-2]

We have deployed our second Glassfish cluster.  The application we are deploying is very large. It's well over 100MB.  I got around the size limit by editing /opt/sun/glassfish/lib/install/applications/admingui/adminGUI_war/WEB-INF/web.xml and changing the UploadFilter to 200000000.

Synchronization is taking a very long time.  The problem is intermittent.  Here's an example from the log:
Synchronization for myapp is complete. Total time spent 2783984 milli second(s).

It seems that we can reboot each node, redeploy and deployment is nearly instant.  

All of this is on a gigabit network, so file upload is not an issue.  I'd really like to find out what's causing this.  There are no log entries indicating what's going on during this time.  I really don't want to bow to the pressure from my co-workers to go back to Tomcat.
[Message sent by forum member 'csimm' (csimm@...)]

http://forums.java.net/jive/thread.jspa?messageID=370220

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

Hi,

we have a similar problem.

Maybe it's the same bug?
https://glassfish.dev.java.net/issues/show_bug.cgi?id=10660

-Matthias
[Message sent by forum member 'matthiasfraass' (matthias.fraass@...)]

http://forums.java.net/jive/thread.jspa?messageID=370271

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

I'm not sure.  It sounds similar.  We are seeing the problem on deployments and redeployments, but not always.  Out of the several times we've had the problem, a reboot fixes it.  After a reboot, we can deploy and it's comparatively instant.  We've only seen the problem on our cluster.  We haven't had the problem on any of our stand alone glassfish servers.  I wish I could test the patch, but I'm running glassfish v2.

We also don't see any log entries while the server is churning.
[Message sent by forum member 'csimm' (csimm@...)]

http://forums.java.net/jive/thread.jspa?messageID=370326

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

GF is doing things in the temp directory. I've heard that if there are many files in the temp dir, GF has performance issues during deployment.
So check the temp dir on that particular machine.
[Message sent by forum member 'matthiasfraass' (matthias.fraass@...)]

http://forums.java.net/jive/thread.jspa?messageID=370341

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

Matthias: the problem reported in this email thread should be different than https://glassfish.dev.java.net/issues/show_bug.cgi?id=10660

The problem reported in this email thread seems to be related to the synchronization part of the cluster scenario (in v2) while issue 10660 seems to be related to scanning archive for annotations to find the corresponding archive handler (in v3).

We don't have cluster support in v3 yet and the part to scan annoations to find archive handler is only applicable to v3 also.

Did you get a chance to try with a later nightly build to to see if things improve?
[Message sent by forum member 'hzhang_jn' (hong.zhang@...)]

http://forums.java.net/jive/thread.jspa?messageID=370346

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Huiping]

Hi,
Could this link be fixed, please?

http://download.java.net/javaee5/sailfin/v2/promoted/source/sailfin-v2-b31g-src.zip
Thanks,
Huiping
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

We will not be able to upgrade right now.  We are less than a week from a big roll out.  The rest of the team would not appreciate it if I blew the cluster up trying to upgrade it.  I wish these were virtual servers so I could take a snapshot.

The next time the problem occurs, I will check the temp directory and post the results.
[Message sent by forum member 'csimm' (csimm@...)]

http://forums.java.net/jive/thread.jspa?messageID=370356

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

No, I was not suggesting to upgrade to v3 (and cluster is not supported in v3). I was just pointing out your problem is different than the other issue which happens in v3.
[Message sent by forum member 'hzhang_jn' (hong.zhang@...)]

http://forums.java.net/jive/thread.jspa?messageID=370359

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

Ah.  I thought the "Did you get a chance to try with a later nightly build to to see if things improve? " was directed at me.  I know I can't go to V3 for sure until we have clustering support.
[Message sent by forum member 'csimm' (csimm@...)]

http://forums.java.net/jive/thread.jspa?messageID=370365

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

I checked /tmp before and after deployment and there is nothing there.  Or very little there.
[Message sent by forum member 'csimm' (csimm@...)]

http://forums.java.net/jive/thread.jspa?messageID=370366

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

UPDATE:  If we start and stop each instance, the deployment seems to work.  I'm hesitant to draw conclusions from any one deployment because this problem is intermittent.  

Hopefully we will have time to deploy until it freezes, then restart the instances to see if it resolves it each time.  Rinse and repeat until I'm satisfied this is consistent behavior.

In the mean time, if anyone has a suggestion for troubleshooting steps, I'm open to all suggestions.
[Message sent by forum member 'csimm' (csimm@...)]

http://forums.java.net/jive/thread.jspa?messageID=370367

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[glassfish-2]

Which version do you use v2.1 or v2.1.1?
We have the same problem whith our GF v2.1 cluster and I believe/hope that this is a known bug which is fixed with patch 2 for GF v2.1, see http://blogs.sun.com/GlassFishForBusiness/entry/sjs_as_9_1_u25 
entry "[6774854] AS9.1 very, very slow to sychronize an 80Mb application via node agent, with `cluster' profile".
This patch is only available to customers with subscription however I assume that it is also included in v2.1.1.
If you are using v2.1 and upgrading to 2.1.1 fixes the problem I would be very interested.

Niklas.
[Message sent by forum member 'niklasg' (ngrossmann@...)]

http://forums.java.net/jive/thread.jspa?messageID=370892

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...