I've been working on some sort of patch and wanted to see if the following would work. Note that it's based on recent changes to the file processor for 1.4, so that the commit on file process strategy gets called even if there's an error on the processing.
So the quick question is, whether it's simply a matter of replacing this code in the Ftp and Sftp consumer components' "pollFile" method:
getProcessor().process(exchange);
...with this, modeled after the file component (note that I only have comments so far related to file processing):
// begin file process strategy
//final FileProcessStrategy processStrategy = endpoint.getFileStrategy();
//if(processStrategy.begin(...
// process async, with callback
getAsyncProcessor().process(exchange, new AsyncCallback(){
public void done(boolean sync) {
boolean failed = exchange.isFailed();
boolean handled = DeadLetterChannel.isFailureHandled(exchange);
if (LOG.isDebugEnabled()) {
LOG.debug("Done processing file: " + ftpFile + ". Status is: " + (failed ? "failed: " + failed + ", handled by failure processor: " + handled : "OK"));
}
if (!failed || handled) {
// commit the file strategy if there was no failure or already handled by the DeadLetterChannel
//processStrategyCommit(...
} else if (failed && !handled) {
// there was an exception but it was not handled by the DeadLetterChannel
handleException(exchange.getException());
}
}
});
The big question is whether or not the switch to an asynch call for processing the message will have any weird effect on the Ftp or Sftp clients.
Of course, the easy way to handle this across the board would be to replace the File, Ftp, and Sftp components with a Vfs component....:)
Gert Vanthienen wrote:
Drew,
No, you are not crazy. Currently, the FTP/SFTP consumers don't support
this. They store the lastPollTimestamp internally and compare that to
the file's timestamps to determine if a file still needs to be
processed. However, the lastPollTime isn't being persisted anywhere, so
restarting the JVM will result in re-reading the files.
If you want, you can always provide a patch for the file-handling
strategies you want to use for your project. Let us know if you need
any help with that... Any (even partial) solution for this issue would
be more than welcome.
If you are using Camel inside ServiceMix, you can also use ServiceMix's
FTP poller component as a (temporary) workaround. That one does already
support most forms of file-handling (delete, move, ...).
Regards,
Gert
Drew McAuliffe wrote:
> Am I crazy or is there no way that the FTP/SFTP consumers do anything to a
> source file once they've read it? There doesn't appear to be any delete or
> move option like with the File consumer. What possible use case would that
> support? Doesn't this pretty much guarantee that an FTP/SFTP consumer will
> continuously re-read files, never stopping?
>
> It appears that there's a JIRA issue to add support for something similar to
> a file renaming strategy, but it doesn't look like it's slated for anything
> in 1.4.
>
-----
---
Gert Vanthienen
http://www.anova.be
