How best to route to many (500+) CXF SOAP outbound endpoints?

[bobs99]

Hi,

I am thinking of using Mule as a kind of centralized outbound SOAP gateway for many hosted applications, on several connected servers, all needing to make various external outbound SOAP requests.

The value of this is that certain aspects, like hosting security, can be more easily controlled than if each application managed its own outbound SOAP calls independently.

I'm wondering what's the most efficient way of setting this up in Mule when potentially large numbers (100+) of different external SOAP endpoints will be called?

The incoming events, from each hosted application, will be some XML containing the SOAP target URI, along with the operation and any parameters needed to make the call. I'm planning to use wsdl2java to generate all the client classes needed by CXF.

Although this is straightforward enough to get working with a few external SOAP endpoints, my main concern is how best to route the XML payload containing the operation and parameters, etc to the correct (out of potentially 100+) outbound CXF endpoint, both for best performance and without cluttering up the mule config file.

Is there a core Mule router to use or should I be thinking in terms of writing a custom router to achieve this?

Any advice (general or specific) on how to go about this would be welcome.
Thanks.

[John D'Emic]

You mentioned you have the SOAP target in the inbound message payload, so you could use an expression evaluator on the outbound-endpoint's address parameter to extract this and use it as the address.

Additionally, take a look at this:

http://www.mulesource.org/display/SAAJ/Home

It includes a pair of transformers that might be more straightforward to deal with then stub classes.

Cheers,
-jd

bobs99 wrote:

Hi,

I am thinking of using Mule as a kind of centralized outbound SOAP gateway for many hosted applications, on several connected servers, all needing to make various external outbound SOAP requests.

The value of this is that certain aspects, like hosting security, can be more easily controlled than if each application managed its own outbound SOAP calls independently.

I'm wondering what's the most efficient way of setting this up in Mule when potentially large numbers (100+) of different external SOAP endpoints will be called?

The incoming events, from each hosted application, will be some XML containing the SOAP target URI, along with the operation and any parameters needed to make the call. I'm planning to use wsdl2java to generate all the client classes needed by CXF.

Although this is straightforward enough to get working with a few external SOAP endpoints, my main concern is how best to route the XML payload containing the operation and parameters, etc to the correct (out of potentially 100+) outbound CXF endpoint, both for best performance and without cluttering up the mule config file.

Is there a core Mule router to use or should I be thinking in terms of writing a custom router to achieve this?

Any advice (general or specific) on how to go about this would be welcome.
Thanks.

[Robin Percy]

bobs99 wrote:

Is there a core Mule router to use or should I be thinking in terms of writing a custom router to achieve this?

I'd suggest having a look at the Recipient List Routers:
http://www.mulesource.org/display/MULE2USER/Outbound+Routers#OutboundRouters-RecipientListRouters

-Robin

[bobs99]

Many thanks John.

I'll look into the SAAJ approach a little later; but first...

Taking you suggestion about using an expression evaluator on the outbound endpoint, I've tried this for the address, wsdlLocation, operation etc, for a simple 'echo' web service. I've created a soapObj bean which, for a test, I've first tried to configure via the mule config file and then, using the 'jxpath' evaluator id, extract these values back from the soapObj (my payload) to make the SOAP call. Like this:

<model name="test">
   <service name="inputService">
      <inbound>
         <inbound-endpoint address="stdio://System.in?promptMessage=Enter string to echo:"
            synchronous="true">

          </inbound-endpoint>
       </inbound>
       <outbound>
           <chaining-router>
              <cxf:outbound-endpoint
                 address="http://#[jxpath:soapObj/address]"
                 clientClass="#[jxpath:soapObj/clientClass]"
                 wsdlPort="#[jxpath:soapObj/wsdlPort]"
                 wsdlLocation="#[jxpath:soapObj/wsdlLocation]"
                 operation="#[jxpath:soapObj/operation]">
                    <properties>
                       <spring:entry key="address" value="soapam.nuwave-tech.com/services/echostring/echostring"/>
                       <spring:entry key="clientClass" value="com.nuwave_tech.soapam.services.echostring.echostring.EchoStringSample"/>
                       <spring:entry key="wsdlPort" value="echoStringSampleSoap"/>
                       <spring:entry key="wsdlLocation" value="stringEcho.wsdl"/>
                       <spring:entry key="operation" value="echoString"/>
                     </properties>
                 </cxf:outbound-endpoint>
               <outbound-endpoint address="stdio://System.out"/>
           </chaining-router>
        </outbound>
    </service>
 </model>

It works fine if I hard code the cxf parameters, but unfortunately it's falling over when the evaluators (i.e #[jxpath:soapObj/address]) are run. I'm assuming, in the 'address case', the getAddress() method of the SoapObj is invoked and this value is used. Is this right, or am I using the evaluators incorrectly?

The output is:

Enter string to echo:
eeeeeeeeeeeee
06-Jul-2009 12:26:36 org.apache.cxf.service.factory.ReflectionServiceFactoryBean
 buildServiceFromWSDL
INFO: Creating Service {http://soapam.nuwave-tech.com/services/echostring/echost
ring/}echoStringSample from WSDL: file:/C:/mule-2.2.2-SNAPSHOT/lib/user/stringEc
ho.wsdl
ERROR 2009-07-06 12:26:36,908 [connector.stdio.0.receiver.1] org.mule.retry.noti
fiers.ConnectNotifier: Failed to connect/reconnect: endpoint.outbound.http://#[j
xpath:(soapObj/address)]. Root Exception was: null. Type: class java.lang.NullPo
interException
ERROR 2009-07-06 12:26:36,924 [connector.stdio.0.receiver.1] org.mule.DefaultExc
eptionStrategy:
********************************************************************************

Message               : Initialisation Failure: null
Type                  : org.mule.retry.RetryPolicyExhaustedException
Code                  : MULE_ERROR-84
JavaDoc               : http://www.mulesource.org/docs/site/current2/apidocs/org
/mule/retry/RetryPolicyExhaustedException.html
Object                : endpoint.outbound.http://#[jxpath:(soapObj/address)]
********************************************************************************

Exception stack is:
1. null (java.lang.NullPointerException)
  java.util.concurrent.ConcurrentHashMap:-1 (null)
2. Initialisation Failure: null (org.mule.retry.RetryPolicyExhaustedException)
  org.mule.retry.policies.AbstractPolicyTemplate:97 (http://www.mulesource.org/d
ocs/site/current2/apidocs/org/mule/retry/RetryPolicyExhaustedException.html)
********************************************************************************

Root Exception stack trace:
java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.put(Unknown Source)
        at org.mule.transport.cxf.ClientWrapper.createClientFromClass(ClientWrap
per.java:334)
        at org.mule.transport.cxf.ClientWrapper.initialize(ClientWrapper.java:11
0)





You mentioned you have the SOAP target in the inbound message payload, so you could use an expression evaluator on the outbound-endpoint's address parameter to extract this and use it as the address.

Additionally, take a look at this:

http://www.mulesource.org/display/SAAJ/Home

It includes a pair of transformers that might be more straightforward to deal with then stub classes.

Cheers,
-jd

bobs99 wrote:

Hi,

I am thinking of using Mule as a kind of centralized outbound SOAP gateway for many hosted applications, on several connected servers, all needing to make various external outbound SOAP requests.

The value of this is that certain aspects, like hosting security, can be more easily controlled than if each application managed its own outbound SOAP calls independently.

I'm wondering what's the most efficient way of setting this up in Mule when potentially large numbers (100+) of different external SOAP endpoints will be called?

The incoming events, from each hosted application, will be some XML containing the SOAP target URI, along with the operation and any parameters needed to make the call. I'm planning to use wsdl2java to generate all the client classes needed by CXF.

Although this is straightforward enough to get working with a few external SOAP endpoints, my main concern is how best to route the XML payload containing the operation and parameters, etc to the correct (out of potentially 100+) outbound CXF endpoint, both for best performance and without cluttering up the mule config file.

Is there a core Mule router to use or should I be thinking in terms of writing a custom router to achieve this?

Any advice (general or specific) on how to go about this would be welcome.
Thanks.

[John D'Emic]

Hey Bobs,

It seems like you're setting the endpoint properties twice, once on the endpoint and again via the spring:entry's, but I don't think that's the problem.  I took a look at org.mule.transport.cxf.ClientWrapper and it seems like the endpoint properties aren't being parsed by the expression evaluator. I'm not familiar enough with the internals of the CXF transport to be sure, but assuming you have a handle on the current event you can probably correct this by parsing the attribute using the parse method of ExpressionManager.

-jd



Many thanks John.

I'll look into the SAAJ approach a little later; but first...

Taking you suggestion about using an expression evaluator on the outbound endpoint, I've tried this for the address, wsdlLocation, operation etc, for a simple 'echo' web service. I've created a soapObj bean which, for a test, I've first tried to configure via the mule config file and then, using the 'jxpath' evaluator id, extract these values back from the soapObj (my payload) to make the SOAP call. Like this:

<model name="test">
   <service name="inputService">
      <inbound>
         <inbound-endpoint address="stdio://System.in?promptMessage=Enter string to echo:"
            synchronous="true">

          </inbound-endpoint>
       </inbound>
       <outbound>
           <chaining-router>
              <cxf:outbound-endpoint
                 address="http://#[jxpath:soapObj/address]"
                 clientClass="#[jxpath:soapObj/clientClass]"
                 wsdlPort="#[jxpath:soapObj/wsdlPort]"
                 wsdlLocation="#[jxpath:soapObj/wsdlLocation]"
                 operation="#[jxpath:soapObj/operation]">
                    <properties>
                       <spring:entry key="address" value="soapam.nuwave-tech.com/services/echostring/echostring"/>
                       <spring:entry key="clientClass" value="com.nuwave_tech.soapam.services.echostring.echostring.EchoStringSample"/>
                       <spring:entry key="wsdlPort" value="echoStringSampleSoap"/>
                       <spring:entry key="wsdlLocation" value="stringEcho.wsdl"/>
                       <spring:entry key="operation" value="echoString"/>
                     </properties>
                 </cxf:outbound-endpoint>
               <outbound-endpoint address="stdio://System.out"/>
           </chaining-router>
        </outbound>
    </service>
 </model>

It works fine if I hard code the cxf parameters, but unfortunately it's falling over when the evaluators (i.e #[jxpath:soapObj/address]) are run. I'm assuming, in the 'address case', the getAddress() method of the SoapObj is invoked and this value is used. Is this right, or am I using the evaluators incorrectly?

The output is:

Enter string to echo:
eeeeeeeeeeeee
06-Jul-2009 12:26:36 org.apache.cxf.service.factory.ReflectionServiceFactoryBean
 buildServiceFromWSDL
INFO: Creating Service {http://soapam.nuwave-tech.com/services/echostring/echost
ring/}echoStringSample from WSDL: file:/C:/mule-2.2.2-SNAPSHOT/lib/user/stringEc
ho.wsdl
ERROR 2009-07-06 12:26:36,908 [connector.stdio.0.receiver.1] org.mule.retry.noti
fiers.ConnectNotifier: Failed to connect/reconnect: endpoint.outbound.http://#[j
xpath:(soapObj/address)]. Root Exception was: null. Type: class java.lang.NullPo
interException
ERROR 2009-07-06 12:26:36,924 [connector.stdio.0.receiver.1] org.mule.DefaultExc
eptionStrategy:
********************************************************************************

Message               : Initialisation Failure: null
Type                  : org.mule.retry.RetryPolicyExhaustedException
Code                  : MULE_ERROR-84
JavaDoc               : http://www.mulesource.org/docs/site/current2/apidocs/org
/mule/retry/RetryPolicyExhaustedException.html
Object                : endpoint.outbound.http://#[jxpath:(soapObj/address)]
********************************************************************************

Exception stack is:
1. null (java.lang.NullPointerException)
  java.util.concurrent.ConcurrentHashMap:-1 (null)
2. Initialisation Failure: null (org.mule.retry.RetryPolicyExhaustedException)
  org.mule.retry.policies.AbstractPolicyTemplate:97 (http://www.mulesource.org/d
ocs/site/current2/apidocs/org/mule/retry/RetryPolicyExhaustedException.html)
********************************************************************************

Root Exception stack trace:
java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.put(Unknown Source)
        at org.mule.transport.cxf.ClientWrapper.createClientFromClass(ClientWrap
per.java:334)
        at org.mule.transport.cxf.ClientWrapper.initialize(ClientWrapper.java:11
0)

John D'Emic wrote:

You mentioned you have the SOAP target in the inbound message payload, so you could use an expression evaluator on the outbound-endpoint's address parameter to extract this and use it as the address.

Additionally, take a look at this:

http://www.mulesource.org/display/SAAJ/Home

It includes a pair of transformers that might be more straightforward to deal with then stub classes.

Cheers,
-jd

bobs99 wrote:

Hi,

I am thinking of using Mule as a kind of centralized outbound SOAP gateway for many hosted applications, on several connected servers, all needing to make various external outbound SOAP requests.

The value of this is that certain aspects, like hosting security, can be more easily controlled than if each application managed its own outbound SOAP calls independently.

I'm wondering what's the most efficient way of setting this up in Mule when potentially large numbers (100+) of different external SOAP endpoints will be called?

The incoming events, from each hosted application, will be some XML containing the SOAP target URI, along with the operation and any parameters needed to make the call. I'm planning to use wsdl2java to generate all the client classes needed by CXF.

Although this is straightforward enough to get working with a few external SOAP endpoints, my main concern is how best to route the XML payload containing the operation and parameters, etc to the correct (out of potentially 100+) outbound CXF endpoint, both for best performance and without cluttering up the mule config file.

Is there a core Mule router to use or should I be thinking in terms of writing a custom router to achieve this?

Any advice (general or specific) on how to go about this would be welcome.
Thanks.

[bobs99]

Hi John, I'm still plugging away on this...

I've left CXF for the time being. I'm trying to get basic HTTP working as a 'dynamic endpoint' before I can begin considering using SAAJ to solve my problem - as you suggested earlier. I'm assuming if I can't even get the basic HTTP transport working dynamically, then using SAAJ on top of the HTTP transport won't be much help to me.  

I'm still looking into this, but while something like this HTTP endpoint works fine using headers for the dynamic data:

<outbound>
   <filtering-router>
       <http:outbound-endpoint address="http://10.12.23.4.5:80#[header:httpDetailsButNoIPAddress]" />
    </filtering-router>
</outbound>

and my expression #[header:httpDetailsButNoIPAddress] is correctly substituted; however, if I try to include the IP address (or host name bit) within the expression itself (to get the dynamic 'bit'), it simply fails at runtime:

<outbound>
   <filtering-router>
       <http:outbound-endpoint address="http://#[header:httpDetailsIncludeIPAddress]" />
    </filtering-router>
</outbound>

It complains about null host exception on HTTP endpoint.

I'm wondering whether it is simply not supported within Mule to submit a host name or IP address to HTTP or CXF outbound endpoints at runtime. Perhaps all these endpoints are created when the config file is read and while some aspects of the endpoint may be dynamically configured at runtime, the actual host name (or IP bit) of the endpoint cannot.

Could this be the case??




Hey Bobs,

It seems like you're setting the endpoint properties twice, once on the endpoint and again via the spring:entry's, but I don't think that's the problem.  I took a look at org.mule.transport.cxf.ClientWrapper and it seems like the endpoint properties aren't being parsed by the expression evaluator. I'm not familiar enough with the internals of the CXF transport to be sure, but assuming you have a handle on the current event you can probably correct this by parsing the attribute using the parse method of ExpressionManager.

-jd



Many thanks John.

I'll look into the SAAJ approach a little later; but first...

Taking you suggestion about using an expression evaluator on the outbound endpoint, I've tried this for the address, wsdlLocation, operation etc, for a simple 'echo' web service. I've created a soapObj bean which, for a test, I've first tried to configure via the mule config file and then, using the 'jxpath' evaluator id, extract these values back from the soapObj (my payload) to make the SOAP call. Like this:

<model name="test">
   <service name="inputService">
      <inbound>
         <inbound-endpoint address="stdio://System.in?promptMessage=Enter string to echo:"
            synchronous="true">

          </inbound-endpoint>
       </inbound>
       <outbound>
           <chaining-router>
              <cxf:outbound-endpoint
                 address="http://#[jxpath:soapObj/address]"
                 clientClass="#[jxpath:soapObj/clientClass]"
                 wsdlPort="#[jxpath:soapObj/wsdlPort]"
                 wsdlLocation="#[jxpath:soapObj/wsdlLocation]"
                 operation="#[jxpath:soapObj/operation]">
                    <properties>
                       <spring:entry key="address" value="soapam.nuwave-tech.com/services/echostring/echostring"/>
                       <spring:entry key="clientClass" value="com.nuwave_tech.soapam.services.echostring.echostring.EchoStringSample"/>
                       <spring:entry key="wsdlPort" value="echoStringSampleSoap"/>
                       <spring:entry key="wsdlLocation" value="stringEcho.wsdl"/>
                       <spring:entry key="operation" value="echoString"/>
                     </properties>
                 </cxf:outbound-endpoint>
               <outbound-endpoint address="stdio://System.out"/>
           </chaining-router>
        </outbound>
    </service>
 </model>

It works fine if I hard code the cxf parameters, but unfortunately it's falling over when the evaluators (i.e #[jxpath:soapObj/address]) are run. I'm assuming, in the 'address case', the getAddress() method of the SoapObj is invoked and this value is used. Is this right, or am I using the evaluators incorrectly?

The output is:

Enter string to echo:
eeeeeeeeeeeee
06-Jul-2009 12:26:36 org.apache.cxf.service.factory.ReflectionServiceFactoryBean
 buildServiceFromWSDL
INFO: Creating Service {http://soapam.nuwave-tech.com/services/echostring/echost
ring/}echoStringSample from WSDL: file:/C:/mule-2.2.2-SNAPSHOT/lib/user/stringEc
ho.wsdl
ERROR 2009-07-06 12:26:36,908 [connector.stdio.0.receiver.1] org.mule.retry.noti
fiers.ConnectNotifier: Failed to connect/reconnect: endpoint.outbound.http://#[j
xpath:(soapObj/address)]. Root Exception was: null. Type: class java.lang.NullPo
interException
ERROR 2009-07-06 12:26:36,924 [connector.stdio.0.receiver.1] org.mule.DefaultExc
eptionStrategy:
********************************************************************************

Message               : Initialisation Failure: null
Type                  : org.mule.retry.RetryPolicyExhaustedException
Code                  : MULE_ERROR-84
JavaDoc               : http://www.mulesource.org/docs/site/current2/apidocs/org
/mule/retry/RetryPolicyExhaustedException.html
Object                : endpoint.outbound.http://#[jxpath:(soapObj/address)]
********************************************************************************

Exception stack is:
1. null (java.lang.NullPointerException)
  java.util.concurrent.ConcurrentHashMap:-1 (null)
2. Initialisation Failure: null (org.mule.retry.RetryPolicyExhaustedException)
  org.mule.retry.policies.AbstractPolicyTemplate:97 (http://www.mulesource.org/d
ocs/site/current2/apidocs/org/mule/retry/RetryPolicyExhaustedException.html)
********************************************************************************

Root Exception stack trace:
java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.put(Unknown Source)
        at org.mule.transport.cxf.ClientWrapper.createClientFromClass(ClientWrap
per.java:334)
        at org.mule.transport.cxf.ClientWrapper.initialize(ClientWrapper.java:11
0)

John D'Emic wrote:

You mentioned you have the SOAP target in the inbound message payload, so you could use an expression evaluator on the outbound-endpoint's address parameter to extract this and use it as the address.

Additionally, take a look at this:

http://www.mulesource.org/display/SAAJ/Home

It includes a pair of transformers that might be more straightforward to deal with then stub classes.

Cheers,
-jd

bobs99 wrote:

Hi,

I am thinking of using Mule as a kind of centralized outbound SOAP gateway for many hosted applications, on several connected servers, all needing to make various external outbound SOAP requests.

The value of this is that certain aspects, like hosting security, can be more easily controlled than if each application managed its own outbound SOAP calls independently.

I'm wondering what's the most efficient way of setting this up in Mule when potentially large numbers (100+) of different external SOAP endpoints will be called?

The incoming events, from each hosted application, will be some XML containing the SOAP target URI, along with the operation and any parameters needed to make the call. I'm planning to use wsdl2java to generate all the client classes needed by CXF.

Although this is straightforward enough to get working with a few external SOAP endpoints, my main concern is how best to route the XML payload containing the operation and parameters, etc to the correct (out of potentially 100+) outbound CXF endpoint, both for best performance and without cluttering up the mule config file.

Is there a core Mule router to use or should I be thinking in terms of writing a custom router to achieve this?

Any advice (general or specific) on how to go about this would be welcome.
Thanks.

[Gerwin Postma]

We have created our own dynamic router by extending the FilteringOutboundRouter

For help on how to do this, take a look at this improvement reported by Dimitar Dimitrov (which unfortunately didn't make it):
http://www.mulesource.org/jira/browse/MULE-3580

HTH

Regards,
Mario

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email