FTP in response to mail queue message

[dhenton9000]

I'd like to initiate an FTP transfer after my mule instance receives a message to a POP3 mail message. We've had problems in the past with file transfers occurring while the file is being deposited in the folder, so we'd like to initiate file transfer after we know the file creation step is completed.

I'm looking for a way to wire up Mule to do it, as a standard behavior without custom code.

Here's the scenario. Process A writes a large file out to folder F, and when complete emails a POP3 account that is being monitored by mule. Mule then initiates the FTP transfer.

The only way I can think to do this is to write a custom component that is fired off by Mule, and then creates a temporary new instance of mule, and is configured one time to FTP transfer from the folder F. That mule instance is then destroyed on completion.

There's got to be a better way, and any suggestions would be very welcome!

[antoine.borg]

Hi,

There is a better way!

Two possible solutions:

1) The fileAge attribute.  The File connector now contains an attribute
which will only read files that are XX milliseconds old. The problem that
you describe was a common problem and this is how it has been solved.  So
you can configure the file connector in Mule to read files that are, say, 5
minutes old so that there will be enough time for the initial file to have
been created/written out completely.  Then configure a simple FTP outbound
endpoint. The problem is that you have to calculate the value of the fileAge
attribute in advance. The advantage is that everything is in config.

2) Dynamic endpoints.  Another solution (if you want to preserve the e-mail
part of your description) is to code a router/component that will read the
e-mail and, upon receipt, read from a dynamically created inbound file
endpoint.  The problem here is that you need coding but most things can be
set as properties so you can still have a flexible solution that is pretty
much in config.

Regards

Antoine

Antoine Borg, Senior Consultant | Tel: [phone]
ricston Ltd., BP 2, 1180 Uccle, Brussels, BELGIUM
email: antoine.borg@... | blog: blog.ricston.com | web: ricston.com

-----Original Message-----
From: dhenton9000 [mailto:dhenton@...]
Sent: Tuesday, August 26, 2008 11:04 PM
To: user@...
Subject: [mule-user] FTP in response to mail queue message


I'd like to initiate an FTP transfer after my mule instance receives a
message to a POP3 mail message. We've had problems in the past with file
transfers occurring while the file is being deposited in the folder, so we'd
like to initiate file transfer after we know the file creation step is
completed.

I'm looking for a way to wire up Mule to do it, as a standard behavior
without custom code.

Here's the scenario. Process A writes a large file out to folder F, and when
complete emails a POP3 account that is being monitored by mule. Mule then
initiates the FTP transfer.

The only way I can think to do this is to write a custom component that is
fired off by Mule, and then creates a temporary new instance of mule, and is
configured one time to FTP transfer from the folder F. That mule instance is
then destroyed on completion.

There's got to be a better way, and any suggestions would be very welcome!
--
View this message in context:
http://www.nabble.com/FTP-in-response-to-mail-queue-message-tp19170374p19170
374.html
Sent from the Mule - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[dhenton9000]

I got it, I guess.
I've got one file service that watches a folder, with *.xml as a filter. My external process writes to that folder with extension *.tmp. When its done, it sends an email message to a POP3 account which another service monitors. In the body of the message is the file name  with *.tmp extension. The POP3 process evokes a bean that checks that the file exists, is the write size, does some other testing, and if all is well, renames it to *.xml.

The file component then does its thing.
HTH

[antoine.borg]

Hi

Will work - but would my suggestion to use the fileAge attribute help?

A

Antoine Borg, Senior Consultant | Tel: [phone]
ricston Ltd., BP 2, 1180 Uccle, Brussels, BELGIUM
email: antoine.borg@... | blog: blog.ricston.com | web: ricston.com

-----Original Message-----
From: dhenton9000 [mailto:dhenton@...]
Sent: Wednesday, August 27, 2008 10:44 PM
To: user@...
Subject: Re: [mule-user] FTP in response to mail queue message


I got it, I guess.
I've got one file service that watches a folder, with *.xml as a filter. My
external process writes to that folder with extension *.tmp. When its done,
it sends an email message to a POP3 account which another service monitors.
In the body of the message is the file name  with *.tmp extension. The POP3
process evokes a bean that checks that the file exists, is the write size,
does some other testing, and if all is well, renames it to *.xml.

The file component then does its thing.
HTH



--
View this message in context:
http://www.nabble.com/FTP-in-response-to-mail-queue-message-tp19170374p19189
346.html
Sent from the Mule - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[dhenton9000]

Alas, the file can easily outgrow any time frame that might be suggested for it. There's a problem with using any time related file attribute as the message, it isn't instantaneous. I think this is an inherent problem with folder polling ESB components. The assumption all these systems make is that any message is instantaneously processed, which isn't the case in our files. The presence of the file in the folder is the "message", but really the presence of the COMPLETED file should be the message. Thus, we need a way to allow the sending process to poke along and then message only when it deems that it is complete. Built in measures of completeness (file size, file age) all have the problem that as the process matures, the parameters set change: 6 months from now we are so big that our process takes 2 hours to run instead of 30 minutes, or end of quarter processing takes us out of the planned time range.

[Andrew Perepelytsya]

If you control the sending side, make it right a 'filename.tmp.done' empty file next to the processed file. Then in Mule you could extend the File receiver to check for this marker file, and proceed only if it is there. If not - the file is still being worked on. There are some edge cases where this scenario may need manual intervention (e.g. in some crash scenarios), but otherwise very robust. I have never come around implementing it as an OOTB functionality in Mule, but know that many have done this in their custom deployments (e.g. with ftp too).

HTH,
Andrew