Just as an FYI, I've hacked the following (after adding an "autoDelete" property to the endpoint configuration. This is in "pollFile" in FtpConsumer. It's a little staggered for debugging purposes, and I'm sure the exception throwing isn't ideal, but it seems to work. This is based somewhat on the Mule FTP and SFTP components. I'm working on the SFTP version next. While I think a FileProcessStrategy approach similar to the File component might be nice, for now an autodelete option works just fine (I delete the files read from SFTP after moving them to local storage, where I can handle them with proper backup after reading). Again, I think if you're going to go through the effort of implementing some sort of FileProcessStrategy thing, it would be better to do it once using a VFS component instead of splitting it between FTP and SFTP because of the different clients.
getAsyncProcessor().process(exchange, new AsyncCallback(){
public void done(boolean sync) {
boolean failed = exchange.isFailed();
boolean handled = DeadLetterChannel.isFailureHandled(exchange);
if (LOG.isDebugEnabled()) {
LOG.debug("Done processing file: " + ftpFile + ". Status is: " + (failed ? "failed: " + failed + ", handled by failure processor: " + handled : "OK"));
}
if (!failed || handled) {
// if autodelete, then remove the source file
if (endpoint.getConfiguration().isAutoDelete()){
try {
String toDelete = getFullFileName(ftpFile);
boolean deleted = client.deleteFile(toDelete);
if (! deleted)
throw new IOException(MessageFormat.format("Could not delete remote file at path {0}. Ftp error:{1}",
new Object[]{ftpFile.getName(), client.getReplyCode()}));
} catch (IOException e) {
throw new RuntimeException(e.getMessage(), e);
}
}
} else if (failed && !handled) {
// there was an exception but it was not handled by the DeadLetterChannel
handleException(exchange.getException());
}
}
});
Claus Ibsen wrote:
Hi
Added ticket: CAMEL-570 to track this issue.
Med venlig hilsen
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
-----Original Message-----
From: Claus Ibsen [mailto:ci@silverbullet.dk]
Sent: 3. juni 2008 05:50
To: camel-user@activemq.apache.org
Subject: RE: No way to remove FTP/SFTP files?
Hi
And voting for the tickets also helps the team decide which issues we should tackle first.
Personally I do think file based components needs an overhaul in Camel to improve their standards as this is IMHO still one of the most common integration techniques in my daily fields of work.
Gert good point about that ServiceMix FTP component. Good spot for code resuse.
Med venlig hilsen
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
-----Original Message-----
From: Gert Vanthienen [mailto:gert.vanthienen@skynet.be]
Sent: 3. juni 2008 04:42
To: camel-user@activemq.apache.org
Subject: Re: No way to remove FTP/SFTP files?
Drew,
No, you are not crazy. Currently, the FTP/SFTP consumers don't support
this. They store the lastPollTimestamp internally and compare that to
the file's timestamps to determine if a file still needs to be
processed. However, the lastPollTime isn't being persisted anywhere, so
restarting the JVM will result in re-reading the files.
If you want, you can always provide a patch for the file-handling
strategies you want to use for your project. Let us know if you need
any help with that... Any (even partial) solution for this issue would
be more than welcome.
If you are using Camel inside ServiceMix, you can also use ServiceMix's
FTP poller component as a (temporary) workaround. That one does already
support most forms of file-handling (delete, move, ...).
Regards,
Gert
Drew McAuliffe wrote:
> Am I crazy or is there no way that the FTP/SFTP consumers do anything to a
> source file once they've read it? There doesn't appear to be any delete or
> move option like with the File consumer. What possible use case would that
> support? Doesn't this pretty much guarantee that an FTP/SFTP consumer will
> continuously re-read files, never stopping?
>
> It appears that there's a JIRA issue to add support for something similar to
> a file renaming strategy, but it doesn't look like it's slated for anything
> in 1.4.
>
