Re: Difficult Performance Problem

Hi
60 MB logs would not help. So let us try simulating the issue.

Here is my understating of your scenario.

SOAP-UI->EJB-WS->HTTP-BC->BPEL

EJB-WS is packaged as separate SA. How EJB calls service? Will you share screen shot this SA?

If I create a BPEL as below, is this sufficient?
Receive
Assign (sets value on the PL from variable & sets value for Invoke)
Invoke (external service)

How do I simulate time-out? shall I put huge time out after Invoke?

Is it possible for you to share the WSDL/XSDs/payload? I can use the artifacts supplied by you or could fabricate myself. I like to be create scenario that as real as possible.

Regards
Murali


jsexton0 wrote:

Hello -
I changed the HTTP log level to "fine" and ran the entire process in
question.  The result is about 60 MBs of log files.  If that is of value to
you, I can zip it up and place it somewhere where you can ftp it, and I can
tell you where the key parts are.  Otherwise, I'm not sure quite how to trim
it down right at this point.  As is usually the case, I'm under some time
constraints to work around this issue, but I can try.

Increased payload size is definitely the issue, yes.  The entire message in
about 5 MBs.  And yes, the delay occurs in something like this: EJB->(X)->WS
where "WS" is the entry to a BPEL.  The BPEL has an assign and then a call
out to a remote service to pass on the data using a dynamic address.  The
BPEL and the calling EJB are in separate SAs, so as I understand how this
works (?), the HTTP-BC is involved, rather than a direct call.  The EJB
calls the BPEL using jaxb style code, generated by Netbeans.

The whole process works, but I am going to have to find a way to make it
faster, and avoid the time outs. 

What do you think?
Thank you




Murali Pottlapelli wrote:
  

  


Increased logging may help to understand the issue, change logging
level on BPEL-SE and HTTP-BC to FINEST. 

I do not need actual project, is it possible to build straw man version
of it? let us skip C# services in first version. You can start with
what you suspect most. 

If I understand right call sequence is 

SOAP-UI->EJB->(X)->BPEL-SE 

is this right? X could be Java EE SE or http-bc? which one you are
using? 

Do you think payload size is causing the issue? I did test with 10 MB
pay load. 

You could use visualvm to do monitor memory and threads, will you
please share thread dump? 

Regards 
Murali 


jsexton0 wrote:
  I don't think I can easily post enough material to duplicate the issue
at
this time.  The process in question involves quite a lot of code,
including
a service on an IIS server, in C#, and SAs on remote Glassfish servers. 
I've got quite a large application here...

I did realize that I should have added a few points to my original post. 
1)
The EJB driving the process is in a different SA from the BPEL it is
calling
(but on the same server).  And 2) the amount of data is relatively large. 
The BPEL is being called with an image of about one MB, that has been
marshaled into a string field (a string, for reasons having to do with the
other servers it goes on to from the BPEL).  The large data block seems to
be key.  I have not seen this in other places in the application.

3) Only restarting Glassfish seems to clear it up, reloading the SAs does
not help.

What I'm wondering about is perhaps some resource being consumed in the
HTTP
BC maybe?  I'm not sure how to go about collecting more information, or on
the impacts of increasing any of the default cache and pool settings for
HTTP or EJBs.



Murali Pottlapelli wrote:
  
  
    I am not aware of this issue. Will you please an issue at 
https://open-esb.dev.java.net/servlets/ProjectIssues  and attach the 
project to reproduce the issue?

Regards
Murali

jsexton0 wrote:
    
    
      I have a system of BPELs and EJBs that perform a task.  Most of the
task
is
driven by an EJB, which invokes a few BPEL processes.  One of the BPELs
takes about 2 seconds to run.

The problem is that if for any reason the client (I'm using SoapUI and
Netbeans unit tests) goes away, such as a timeout, then the next time the
process is run, the same process that took 2 seconds to run takes more
like
seven seconds.

Let the client time out again, and that becomes more like 2-3 minutes. 
Do
this 3-5 times and the system becomes unusable, the delay becomes several
hours.

I have used logging to isolate the time being consumed to be just between
the Java/EJB's call to the BPEL's web service, and the start of the
initial
receive action in the BPEL.

How can I avoid this problem?
Thanks