Making SFTP transport working in Mule 2.2

[Richard Swart]

Hi,

I am trying to get the sftp transport to work in Mule version 2.2. However I am stuck at the part the Mule has to find the xsd for the sftp transport. I keep getting the following error when trying to run the unit test 'SftpLargeReceiveFunctionalTestCase':
cvc-complex-type.2.4.a: Invalid content was found starting with element 'sftp:connector'. One of '{"http://www.mulesource.org/schema/mule/core/2.2":description, "http://www.springframework.org/schema/beans":beans, "http://www.springframework.org/schema/beans":bean, "http://www.springframework.org/schema/context":property-placeholder, "http://www.mulesource.org/schema/mule/core/2.2":global-property, "http://www.mulesource.org/schema/mule/core/2.2":configuration, "http://www.mulesource.org/schema/mule/core/2.2":notifications, "http://www.mulesource.org/schema/mule/core/2.2":abstract-extension, "http://www.mulesource.org/schema/mule/core/2.2":abstract-security-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transaction-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-connector, "http://www.mulesource.org/schema/mule/core/2.2":abstract-global-endpoint, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transformer, "http://www.mulesource.org/schema/mule/core/2.2":abstract-filter, "http://www.mulesource.org/schema/mule/core/2.2":abstract-model, "http://www.mulesource.org/schema/mule/core/2.2":abstract-interceptor-stack}' is expected.

So the real question for now is how do I refer to the 'local' location of the xsd in my mule-config file. I have edited the XSD so the namespace match 'http://www.mulesource.org/schema/mule/sftp/2.2'. I hoped that adding it to the classpath would be sufficient but unfortunately that is not the case. Any help is welcome!

Here is part of my config:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesource.org/schema/mule/core/2.2"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:spring="http://www.springframework.org/schema/beans"
        xmlns:sftp="http://www.mulesource.org/schema/mule/sftp/2.2"
        xmlns:vm="http://www.mulesource.org/schema/mule/vm/2.2"
        xsi:schemaLocation="
               http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
               http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd
               http://www.mulesource.org/schema/mule/sftp/2.2  file///mule-sftp.xsd
               http://www.mulesource.org/schema/mule/vm/2.2 http://www.mulesource.org/schema/mule/vm/2.2/mule-vm.xsd">

        <sftp:connector name="sftp" pollingFrequency="20000" autoDelete="true">
                <custom-exception-strategy
                        class="org.mule.tck.functional.QuietExceptionStrategy" />
        </sftp:connector>
 ....
</mule>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[David Dossot-3]

So the real question for now is how do I refer to the 'local' location of the xsd in my mule-config file.

This uses the standard mechanism from Spring XML schema driven configuration. You need to edit:

• /src/main/resources/META-INF/spring.handlers
• /src/main/resources/META-INF/spring.schemas

in order to "redirect" the resolution of http://www.mulesource.org/schema/mule/sftp/2.2/mule-sftp.xsd to your embedded schema file.

D.

[antoine.borg]

Hi,

In the past, when working without internet access, I had the XSDs in the
classpath and removed the line that starts with "xsi:schemalocation"
which worked.  Perhaps you need to remove the schema location for SFTP?

HTH

A

Pascal Alma wrote:

> Hi,
>
> I am trying to get the sftp transport to work in Mule version 2.2. However I am stuck at the part the Mule has to find the xsd for the sftp transport. I keep getting the following error when trying to run the unit test 'SftpLargeReceiveFunctionalTestCase':
> cvc-complex-type.2.4.a: Invalid content was found starting with element 'sftp:connector'. One of '{"http://www.mulesource.org/schema/mule/core/2.2":description, "http://www.springframework.org/schema/beans":beans, "http://www.springframework.org/schema/beans":bean, "http://www.springframework.org/schema/context":property-placeholder, "http://www.mulesource.org/schema/mule/core/2.2":global-property, "http://www.mulesource.org/schema/mule/core/2.2":configuration, "http://www.mulesource.org/schema/mule/core/2.2":notifications, "http://www.mulesource.org/schema/mule/core/2.2":abstract-extension, "http://www.mulesource.org/schema/mule/core/2.2":abstract-security-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transaction-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-connector, "http://www.mulesource.org/schema/mule/core/2.2":abstract-global-endpoint, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transformer, "http://www.mulesource.org/schema/mule/core/2.2":abstract-filter, "http://www.mulesource.org/schema/mule/core/2.2":abstract-model, "http://www.mulesource.org/schema/mule/core/2.2":abstract-interceptor-stack}' is expected.
>
> So the real question for now is how do I refer to the 'local' location of the xsd in my mule-config file. I have edited the XSD so the namespace match 'http://www.mulesource.org/schema/mule/sftp/2.2'. I hoped that adding it to the classpath would be sufficient but unfortunately that is not the case. Any help is welcome!
>
> Here is part of my config:
> <?xml version="1.0" encoding="UTF-8"?>
> <mule xmlns="http://www.mulesource.org/schema/mule/core/2.2"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:spring="http://www.springframework.org/schema/beans"
> xmlns:sftp="http://www.mulesource.org/schema/mule/sftp/2.2"
> xmlns:vm="http://www.mulesource.org/schema/mule/vm/2.2"
> xsi:schemaLocation="
>                http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
>                http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd
>                http://www.mulesource.org/schema/mule/sftp/2.2  file///mule-sftp.xsd
>                http://www.mulesource.org/schema/mule/vm/2.2 http://www.mulesource.org/schema/mule/vm/2.2/mule-vm.xsd">
>
> <sftp:connector name="sftp" pollingFrequency="20000" autoDelete="true">
> <custom-exception-strategy
> class="org.mule.tck.functional.QuietExceptionStrategy" />
> </sftp:connector>
>  ....
> </mule>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>
>
>  

--

Antoine Borg, Senior Consultant | Tel: [phone]
ricston Ltd., BP 2, 1180 Uccle, Brussels, BELGIUM

See our full schedule of Mule and Android courses:
http://www.ricston.com/courses/schedules/

email: _antoine.borg_@... <mailto:antoine.borg@...> |
blog: blog.ricston.com <http://blog.ricston.com> | web: ricston.com
<http://www.ricston.com/>




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Richard Swart]

Hi David,
Thx for the quick response. That was the solution I was looking for!

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Dirk Olmes-5]

Antoine Borg wrote:
> Hi,
>
> In the past, when working without internet access, I had the XSDs in the
> classpath and removed the line that starts with "xsi:schemalocation"
> which worked.  Perhaps you need to remove the schema location for SFTP?

If you're working with Eclipse, the XML Catalog feature may come in very
handy. Read about it here:
http://xanthippe.dyndns.org/blog/index.php?/archives/5-Validating-XML-documents-against-your-own-schemas-with-Eclipse.html

-dirk

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Richard Swart]

Okay,  I think I almost got it working, however when I run my test class I get the following exception:

ERROR 2009-07-01 11:54:47,404 [sftpConnector.receiver.1] org.mule.DefaultExceptionStrategy:
********************************************************************************
Message               : Cannot route event as component "testComponent" is stopped. Component that caused exception is: SedaService{testComponent}. Message payload is of type: SftpInputStream
Type                  : org.mule.api.service.ServiceException
Code                  : MULE_ERROR--2
JavaDoc               : http://www.mulesource.org/docs/site/current2/apidocs/org/mule/api/service/ServiceException.html
Payload               : org.mule.transport.sftp.SftpInputStream@1ba64d
********************************************************************************
Exception stack is:
1. Cannot route event as component "testComponent" is stopped. Component that caused exception is: SedaService{testComponent}. Message payload is of type: SftpInputStream (org.mule.api.service.ServiceException)
  org.mule.service.AbstractService:434 (http://www.mulesource.org/docs/site/current2/apidocs/org/mule/api/service/ServiceException.html)
********************************************************************************
Root Exception stack trace:
org.mule.api.service.ServiceException: Cannot route event as component "testComponent" is stopped. Component that caused exception is: SedaService{testComponent}. Message payload is of type: SftpInputStream
        at org.mule.service.AbstractService.dispatchEvent(AbstractService.java:434)
        at org.mule.DefaultMuleSession.dispatchEvent(DefaultMuleSession.java:274)
        at org.mule.routing.inbound.DefaultInboundRouterCollection.dispatch(DefaultInboundRouterCollection.java:223)
        at org.mule.routing.inbound.DefaultInboundRouterCollection.route(DefaultInboundRouterCollection.java:192)
        at org.mule.transport.AbstractMessageReceiver$DefaultInternalMessageListener.onMessage(AbstractMessageReceiver.java:364)
        at org.mule.transport.AbstractMessageReceiver.routeMessage(AbstractMessageReceiver.java:252)
        at org.mule.transport.AbstractMessageReceiver.routeMessage(AbstractMessageReceiver.java:175)
        at org.mule.transport.sftp.SftpMessageReceiver.routeFile(SftpMessageReceiver.java:137)
        at org.mule.transport.sftp.SftpMessageReceiver.poll(SftpMessageReceiver.java:60)
        at org.mule.transport.PollingReceiverWorker.run(PollingReceiverWorker.java:47)
        at org.mule.work.WorkerContext.run(WorkerContext.java:310)
        at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1061)
        at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:575)
        at java.lang.Thread.run(Thread.java:613)

********************************************************************************

From the debug info I see that the sftp is picking up a file from my remote machine but is somehow unable to pass it back to the running Mule instance?!?!
Here is some debug info just before the error occurs:
DEBUG 2009-07-01 12:04:32,787 [sftpConnector.receiver.1] org.mule.transport.sftp.SftpMessageReceiver: File is stable (not growing), ready for retrieval: 1.xml
INFO  2009-07-01 12:04:32,933 [sftpConnector.receiver.1] org.mule.transport.sftp.SftpConnector: Successfully connected to: sftp://user:pswd@...:22/home/user/test/
INFO  2009-07-01 12:04:32,933 [sftpConnector.receiver.1] org.mule.transport.sftp.SftpClient: wd = /home/user/test/
DEBUG 2009-07-01 12:04:32,935 [sftpConnector.receiver.1] org.mule.transport.sftp.SftpMessageReceiver: Routing file: 1.xml
DEBUG 2009-07-01 12:04:32,945 [sftpConnector.receiver.1] org.mule.transport.sftp.SftpMessageReceiver: Message Received from: sftp://user:pswd@...:22/home/user/test/
DEBUG 2009-07-01 12:04:32,947 [sftpConnector.receiver.1] org.mule.DefaultMuleSession: There is no session id on the request using key: ID. Generating new session id: 930b38e4-6626-11de-a5e9-056ef93084aa
DEBUG 2009-07-01 12:04:32,953 [sftpConnector.receiver.1] org.mule.DefaultMuleSession: dispatching event to service: testComponent, event is: MuleEvent: 930bae15-6626-11de-a5e9-056ef93084aa, sync=false, stop processing=false, DefaultInboundEndpoint{endpointUri=sftp://user:<password>@192.31.12.23/home/user/test/, connector=SftpConnector{this=93b3c, started=false, initialised=true, name='sftpConnector', disposed=false, numberOfConcurrentTransactedReceivers=4, createMultipleTransactedReceivers=true, connected=false, supportedProtocols=[sftp], serviceOverrides=null}, transformer=[], name='endpoint.sftp.192.31.12.23.22.home.user.test', properties={}, transactionConfig=Transaction{factory=null, action=NEVER, timeout=0}, filter=null, deleteUnacceptedMessages=false, securityFilter=null, synchronous=false, initialState=started, responseTimeout=10000, endpointEncoding=UTF-8}

Any help is again appreciated!

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Häggkvist, Lennart]

Hi,

I'm also working on upgrading the SFTP transport to 2.2 (and I got it to work).

Beside the upgrade I have also added support for identity files (ie. use public/private
keys for authentication instead of passwords) and some other small changes.

How do I submit a patch for the above so that it will be included in the official release
Of the transport? The "Contribute" link on http://www.mulesource.org/display/SFTP/Home points
To http://www.mulesource.org/display/MULEFORGE/Contribute - but that is a Mule "core" info page?
Are there any differences between a transport and "core"?

Thanks,
 Lennart
 

-----Ursprungligt meddelande-----
Från: Pascal Alma [mailto:mule.user.relay@...]
Skickat: den 1 juli 2009 08:31
Till: user@...
Ämne: [mule-user] Making SFTP transport working in Mule 2.2

Hi,

I am trying to get the sftp transport to work in Mule version 2.2. However I am stuck at the part the Mule has to find the xsd for the sftp transport. I keep getting the following error when trying to run the unit test 'SftpLargeReceiveFunctionalTestCase':
cvc-complex-type.2.4.a: Invalid content was found starting with element 'sftp:connector'. One of '{"http://www.mulesource.org/schema/mule/core/2.2":description, "http://www.springframework.org/schema/beans":beans, "http://www.springframework.org/schema/beans":bean, "http://www.springframework.org/schema/context":property-placeholder, "http://www.mulesource.org/schema/mule/core/2.2":global-property, "http://www.mulesource.org/schema/mule/core/2.2":configuration, "http://www.mulesource.org/schema/mule/core/2.2":notifications, "http://www.mulesource.org/schema/mule/core/2.2":abstract-extension, "http://www.mulesource.org/schema/mule/core/2.2":abstract-security-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transaction-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-connector, "http://www.mulesource.org/schema/mule/core/2.2":abstract-global-endpoint, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transformer, "http://www.mulesource.org/schema/mule/core/2.2":abstract-filter, "http://www.mulesource.org/schema/mule/core/2.2":abstract-model, "http://www.mulesource.org/schema/mule/core/2.2":abstract-interceptor-stack}' is expected.

So the real question for now is how do I refer to the 'local' location of the xsd in my mule-config file. I have edited the XSD so the namespace match 'http://www.mulesource.org/schema/mule/sftp/2.2'. I hoped that adding it to the classpath would be sufficient but unfortunately that is not the case. Any help is welcome!

Here is part of my config:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesource.org/schema/mule/core/2.2"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:spring="http://www.springframework.org/schema/beans"
        xmlns:sftp="http://www.mulesource.org/schema/mule/sftp/2.2"
        xmlns:vm="http://www.mulesource.org/schema/mule/vm/2.2"
        xsi:schemaLocation="
               http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
               http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd
               http://www.mulesource.org/schema/mule/sftp/2.2  file///mule-sftp.xsd
               http://www.mulesource.org/schema/mule/vm/2.2 http://www.mulesource.org/schema/mule/vm/2.2/mule-vm.xsd">

        <sftp:connector name="sftp" pollingFrequency="20000" autoDelete="true">
                <custom-exception-strategy
                        class="org.mule.tck.functional.QuietExceptionStrategy" />
        </sftp:connector>
 ....
</mule>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Richard Swart]

Hi Lennart,

I have sent you a PM with my email address. Perhaps you can send me the patch directly so I can continue with my work?

Many thanks in advance!

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[steve k]

Could you send me the updated source code too? Thanks

[virtualshock]

I need to update the SFTP transport for a client we are migrating to Mule 2.x.  Can you guys work together to get this put into MuleForge?  If you can at least get the code synced up I would be happy to put it into MuleForge.  I want to make sure that we utilize MuleForge for getting this transport updated.  Since you have already put in the work to get it updated the community would much appreciate it.

Thanks

Häggkvist, Lennart wrote:

Hi,

I'm also working on upgrading the SFTP transport to 2.2 (and I got it to work).

Beside the upgrade I have also added support for identity files (ie. use public/private
keys for authentication instead of passwords) and some other small changes.

How do I submit a patch for the above so that it will be included in the official release
Of the transport? The "Contribute" link on http://www.mulesource.org/display/SFTP/Home points
To http://www.mulesource.org/display/MULEFORGE/Contribute - but that is a Mule "core" info page?
Are there any differences between a transport and "core"?

Thanks,
 Lennart
 

-----Ursprungligt meddelande-----
Från: Pascal Alma [mailto:[e-mail]]
Skickat: den 1 juli 2009 08:31
Till: [e-mail]
Ämne: [mule-user] Making SFTP transport working in Mule 2.2

Hi,

I am trying to get the sftp transport to work in Mule version 2.2. However I am stuck at the part the Mule has to find the xsd for the sftp transport. I keep getting the following error when trying to run the unit test 'SftpLargeReceiveFunctionalTestCase':
cvc-complex-type.2.4.a: Invalid content was found starting with element 'sftp:connector'. One of '{"http://www.mulesource.org/schema/mule/core/2.2":description, "http://www.springframework.org/schema/beans":beans, "http://www.springframework.org/schema/beans":bean, "http://www.springframework.org/schema/context":property-placeholder, "http://www.mulesource.org/schema/mule/core/2.2":global-property, "http://www.mulesource.org/schema/mule/core/2.2":configuration, "http://www.mulesource.org/schema/mule/core/2.2":notifications, "http://www.mulesource.org/schema/mule/core/2.2":abstract-extension, "http://www.mulesource.org/schema/mule/core/2.2":abstract-security-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transaction-manager, "http://www.mulesource.org/schema/mule/core/2.2":abstract-connector, "http://www.mulesource.org/schema/mule/core/2.2":abstract-global-endpoint, "http://www.mulesource.org/schema/mule/core/2.2":abstract-transformer, "http://www.mulesource.org/schema/mule/core/2.2":abstract-filter, "http://www.mulesource.org/schema/mule/core/2.2":abstract-model, "http://www.mulesource.org/schema/mule/core/2.2":abstract-interceptor-stack}' is expected.

So the real question for now is how do I refer to the 'local' location of the xsd in my mule-config file. I have edited the XSD so the namespace match 'http://www.mulesource.org/schema/mule/sftp/2.2'. I hoped that adding it to the classpath would be sufficient but unfortunately that is not the case. Any help is welcome!

Here is part of my config:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesource.org/schema/mule/core/2.2"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:spring="http://www.springframework.org/schema/beans"
        xmlns:sftp="http://www.mulesource.org/schema/mule/sftp/2.2"
        xmlns:vm="http://www.mulesource.org/schema/mule/vm/2.2"
        xsi:schemaLocation="
               http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
               http://www.mulesource.org/schema/mule/core/2.2 http://www.mulesource.org/schema/mule/core/2.2/mule.xsd
               http://www.mulesource.org/schema/mule/sftp/2.2  file///mule-sftp.xsd
               http://www.mulesource.org/schema/mule/vm/2.2 http://www.mulesource.org/schema/mule/vm/2.2/mule-vm.xsd">

        <sftp:connector name="sftp" pollingFrequency="20000" autoDelete="true">
                <custom-exception-strategy
                        class="org.mule.tck.functional.QuietExceptionStrategy" />
        </sftp:connector>
 ....
</mule>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email