RE: No way to remove FTP/SFTP files?

>
> Hi Drew
>
> Wrapping into a runtime exception is okay, but try to find a suitable
> exception in camel-core.
>
> I do think that Camel could be improved here to adhere to a more
> common/strict exception hieracy - but that can be a goal for Camel 2.0
> where the API can be changed slightly.
>
> The FTPConsumer will have it's exception handled by the
> ScheduledPollConsumer#run.
>
>
> About the async, well I do not think so at current time. Is there any real
> life use-case for this requirements?
>
> Let's keep the task simple to remedy the problem with the lack of a
> delete=true|false option on the component.
>
>
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> -----Original Message-----
> From: Drew McAuliffe [mailto:drewmca@...]
> Sent: 6. juni 2008 22:39
> To: camel-user@...
> Subject: RE: No way to remove FTP/SFTP files?
>
>
> Before I submit it, any ideas on how to handle the exception throwing in
> the
> processing? I'm just throwing a RuntimeException because I didn't know
> what
> the normal Camel procedure was. I have to trap the IOException to adhere
> to
> the interface of the async processor.
>
> Also, are there any repercussions for doing this in an async processor
> block, as opposed to the way the component handles it now (synchronously
> with a call directly to "process")?
>
>
>
> Claus Ibsen wrote:
>>
>> Hi Drew
>>
>> Great work. Keep it up and I will later commit it into Camel.
>> When you think its ready then please submit it as a patch to CAMEL-570.
>>
>> Camel does really need this feature to remove consumed FTP files.
>> Transferring messages using FTP is still a very common use case in real
>> life.
>>
>>
>> Med venlig hilsen
>>  
>> Claus Ibsen
>> ......................................
>> Silverbullet
>> Skovsgårdsvænget 21
>> 8362 Hørning
>> Tlf. +45 2962 7576
>> Web: www.silverbullet.dk
>>
>> -----Original Message-----
>> From: Drew McAuliffe [mailto:drewmca@...]
>> Sent: 6. juni 2008 07:35
>> To: camel-user@...
>> Subject: RE: No way to remove FTP/SFTP files?
>>
>>
>> Just as an FYI, I've hacked the following (after adding an "autoDelete"
>> property to the endpoint configuration. This is in "pollFile" in
>> FtpConsumer. It's a little staggered for debugging purposes, and I'm sure
>> the exception throwing isn't ideal, but it seems to work. This is based
>> somewhat on the Mule FTP and SFTP components. I'm working on the SFTP
>> version next. While I think a FileProcessStrategy approach similar to the
>> File component might be nice, for now an autodelete option works just
>> fine
>> (I delete the files read from SFTP after moving them to local storage,
>> where
>> I can handle them with proper backup after reading). Again, I think if
>> you're going to go through the effort of implementing some sort of
>> FileProcessStrategy thing, it would be better to do it once using a VFS
>> component instead of splitting it between FTP and SFTP because of the
>> different clients.
>>
>>                 getAsyncProcessor().process(exchange, new
>> AsyncCallback(){
>>                 public void done(boolean sync) {
>>                         boolean failed = exchange.isFailed();
>>                         boolean handled =
>> DeadLetterChannel.isFailureHandled(exchange);
>>                        
>>                         if (LOG.isDebugEnabled()) {
>>                             LOG.debug("Done processing file: " + ftpFile
>> +
>> ". Status is: " + (failed ? "failed: " + failed + ", handled by failure
>> processor: " + handled : "OK"));
>>                         }
>>
>>                         if (!failed || handled) {
>>                         // if autodelete, then remove the source file
>>                         if (endpoint.getConfiguration().isAutoDelete()){
>>                         try {
>>                         String toDelete = getFullFileName(ftpFile);
>> boolean deleted = client.deleteFile(toDelete);
>> if (! deleted)
>> throw new IOException(MessageFormat.format("Could not delete
>> remote file at path {0}. Ftp error:{1}",
>> new Object[]{ftpFile.getName(), client.getReplyCode()}));
>>
>> } catch (IOException e) {
>> throw new RuntimeException(e.getMessage(), e);
>> }
>>                         }
>>                         } else if (failed && !handled) {
>>                             // there was an exception but it was not
>> handled
>> by the DeadLetterChannel
>>                             handleException(exchange.getException());
>>                         }
>>                 }
>>                 });
>>  
>>
>>
>> Claus Ibsen wrote:
>>>
>>> Hi
>>>
>>> Added ticket: CAMEL-570 to track this issue.
>>>
>>>
>>> Med venlig hilsen
>>>  
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>>
>>> -----Original Message-----
>>> From: Claus Ibsen [mailto:ci@...]
>>> Sent: 3. juni 2008 05:50
>>> To: camel-user@...
>>> Subject: RE: No way to remove FTP/SFTP files?
>>>
>>> Hi
>>>
>>> And voting for the tickets also helps the team decide which issues we
>>> should tackle first.
>>>
>>> Personally I do think file based components needs an overhaul in Camel
>>> to
>>> improve their standards as this is IMHO still one of the most common
>>> integration techniques in my daily fields of work.
>>>
>>> Gert good point about that ServiceMix FTP component. Good spot for code
>>> resuse.
>>>
>>>
>>>
>>> Med venlig hilsen
>>>  
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>> -----Original Message-----
>>> From: Gert Vanthienen [mailto:gert.vanthienen@...]
>>> Sent: 3. juni 2008 04:42
>>> To: camel-user@...
>>> Subject: Re: No way to remove FTP/SFTP files?
>>>
>>> Drew,
>>>
>>>
>>> No, you are not crazy.  Currently, the FTP/SFTP consumers don't support
>>> this.  They store the lastPollTimestamp internally and compare that to
>>> the file's timestamps to determine if a file still needs to be
>>> processed.  However, the lastPollTime isn't being persisted anywhere, so
>>> restarting the JVM will result in re-reading the files.
>>>
>>> If you want, you can always provide a patch for the file-handling
>>> strategies you want to use for your project.  Let us know if you need
>>> any help with that...  Any (even partial) solution for this issue would
>>> be more than welcome.
>>>
>>> If you are using Camel inside ServiceMix, you can also use ServiceMix's
>>> FTP poller component as a (temporary) workaround.  That one does already
>>> support most forms of file-handling (delete, move, ...).
>>>
>>>
>>> Regards,
>>>
>>> Gert
>>>
>>>
>>>
>>> Drew McAuliffe wrote:
>>>> Am I crazy or is there no way that the FTP/SFTP consumers do anything
>>>> to
>>>> a
>>>> source file once they've read it? There doesn't appear to be any delete
>>>> or
>>>> move option like with the File consumer. What possible use case would
>>>> that
>>>> support? Doesn't this pretty much guarantee that an FTP/SFTP consumer
>>>> will
>>>> continuously re-read files, never stopping?
>>>>
>>>> It appears that there's a JIRA issue to add support for something
>>>> similar
>>>> to
>>>> a file renaming strategy, but it doesn't look like it's slated for
>>>> anything
>>>> in 1.4.
>>>>  
>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/No-way-to-remove-FTP-SFTP-files--tp17612896s22882p17684963.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/No-way-to-remove-FTP-SFTP-files--tp17612896s22882p17700718.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>