transport.vfs supports lockfile ?
|
View:
New views
12 Messages
—
Rating Filter:
Alert me
|
 |
transport.vfs supports lockfile ?
by Harm Verhagen-4
::
Rate this Message:
Hi,
I couldn't file the following in the wso2esb manual. Does the ESB transport.vfs support lockfiles ? Say you have one server uploading flat files to a directory (scp). You have the ESB processing that library using the vfs file listener. to prevent that the ESB starts processing a file is is partially uploaded we'd like to put a file (eg: lock.txt) in the directory, that blocks reading that file). Can it do that now using the existing transport.vfs, or would I have to write my own file reader ? esb:1.7.1 Regards, Harm _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Ruwan Linton-3
::
Rate this Message:
Hi Harm,
I am afraid, AFAIK, this is not yet supported by the WSO2 ESB vfs transport by default, but it could be a very good improvement. You may file an enhancement JIRA for that and we could get that into the next release. Thanks, Ruwan Harm Verhagen wrote: > Hi, > > I couldn't file the following in the wso2esb manual. > Does the ESB transport.vfs support lockfiles ? > > Say you have one server uploading flat files to a directory (scp). You > have the ESB processing that library using the vfs file listener. > to prevent that the ESB starts processing a file is is partially > uploaded we'd like to put a file (eg: lock.txt) in the directory, > that blocks reading that file). > Can it do that now using the existing transport.vfs, or would I have > to write my own file reader ? > > esb:1.7.1 > > > > Regards, > Harm > > ------------------------------------------------------------------------ > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > -- Ruwan Linton Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb WSO2 Inc.; http://wso2.org email: ruwan@...; cell: +94 77 341 3097 blog: http://blog.ruwan.org _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Harm Verhagen-4
::
Rate this Message:
So how does wso2esb currently protect against processing partially uploaded files ?
guessing.. Does it somehow/ try to get an exclusive lock on the file ? Regards, Harm On Wed, Aug 5, 2009 at 2:57 PM, Ruwan Linton <ruwan@...> wrote: Hi Harm, _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Ruwan Linton-3
::
Rate this Message:
Harm Verhagen wrote:
> So how does wso2esb currently protect against processing partially > uploaded files ? > guessing.. Does it somehow/ try to get an exclusive lock on the file ? Once again this is not supported in the ESB, but I see this as a critical improvement and hence would like to work on this ASAP. If you are OK with using the latest version of the ESB I could fix this on the trunk and provide a SNAPSHOT build of the vfs transport, which is compatible with the 2.1.0 release, but I am not sure about the compatibility with the 1.7.1. If you cannot move to the new release and still interested in this fix you may buy support from WSO2 to get this fix ported into the 1.7.1 version. Anyway I will fix this on the trunk, created the issue at [1] Thanks, Ruwan [1] - https://wso2.org/jira/browse/ESBJAVA-602 > > Regards, > Harm > > On Wed, Aug 5, 2009 at 2:57 PM, Ruwan Linton <ruwan@... > <mailto:ruwan@...>> wrote: > > Hi Harm, > > I am afraid, AFAIK, this is not yet supported by the WSO2 ESB vfs > transport by default, but it could be a very good improvement. You may > file an enhancement JIRA for that and we could get that into the next > release. > > Thanks, > Ruwan > > Harm Verhagen wrote: > > Hi, > > > > I couldn't file the following in the wso2esb manual. > > Does the ESB transport.vfs support lockfiles ? > > > > Say you have one server uploading flat files to a directory > (scp). You > > have the ESB processing that library using the vfs file listener. > > to prevent that the ESB starts processing a file is is partially > > uploaded we'd like to put a file (eg: lock.txt) in the directory, > > that blocks reading that file). > > Can it do that now using the existing transport.vfs, or would I > have > > to write my own file reader ? > > > > esb:1.7.1 > > > > > > > > Regards, > > Harm > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Esb-java-user mailing list > > Esb-java-user@... <mailto:Esb-java-user@...> > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > > -- > Ruwan Linton > Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org > email: ruwan@... <mailto:ruwan@...>; cell: +94 77 341 3097 > blog: http://blog.ruwan.org > > > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... <mailto:Esb-java-user@...> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > ------------------------------------------------------------------------ > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > -- Ruwan Linton Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb WSO2 Inc.; http://wso2.org email: ruwan@...; cell: +94 77 341 3097 blog: http://blog.ruwan.org _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Ruwan Linton-3
::
Rate this Message:
Ruwan Linton wrote:
> Harm Verhagen wrote: > >> So how does wso2esb currently protect against processing partially >> uploaded files ? >> guessing.. Does it somehow/ try to get an exclusive lock on the file ? >> > Once again this is not supported in the ESB, but I see this as a > critical improvement and hence would like to work on this ASAP. If you > are OK with using the latest version of the ESB I could fix this on the > trunk and provide a SNAPSHOT build of the vfs transport, which is > compatible with the 2.1.0 release, but I am not sure about the > compatibility with the 1.7.1. > > If you cannot move to the new release and still interested in this fix > you may buy support from WSO2 to get this fix ported into the 1.7.1 version. > > Anyway I will fix this on the trunk, created the issue at [1] > you could get this feature. Thanks, Ruwan > Thanks, > Ruwan > > [1] - https://wso2.org/jira/browse/ESBJAVA-602 > > >> Regards, >> Harm >> >> On Wed, Aug 5, 2009 at 2:57 PM, Ruwan Linton <ruwan@... >> <mailto:ruwan@...>> wrote: >> >> Hi Harm, >> >> I am afraid, AFAIK, this is not yet supported by the WSO2 ESB vfs >> transport by default, but it could be a very good improvement. You may >> file an enhancement JIRA for that and we could get that into the next >> release. >> >> Thanks, >> Ruwan >> >> Harm Verhagen wrote: >> > Hi, >> > >> > I couldn't file the following in the wso2esb manual. >> > Does the ESB transport.vfs support lockfiles ? >> > >> > Say you have one server uploading flat files to a directory >> (scp). You >> > have the ESB processing that library using the vfs file listener. >> > to prevent that the ESB starts processing a file is is partially >> > uploaded we'd like to put a file (eg: lock.txt) in the directory, >> > that blocks reading that file). >> > Can it do that now using the existing transport.vfs, or would I >> have >> > to write my own file reader ? >> > >> > esb:1.7.1 >> > >> > >> > >> > Regards, >> > Harm >> > >> > >> ------------------------------------------------------------------------ >> > >> > _______________________________________________ >> > Esb-java-user mailing list >> > Esb-java-user@... <mailto:Esb-java-user@...> >> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user >> > >> >> >> -- >> Ruwan Linton >> Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb >> WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org >> email: ruwan@... <mailto:ruwan@...>; cell: +94 77 341 3097 >> blog: http://blog.ruwan.org >> >> >> >> _______________________________________________ >> Esb-java-user mailing list >> Esb-java-user@... <mailto:Esb-java-user@...> >> https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Esb-java-user mailing list >> Esb-java-user@... >> https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user >> >> > > > -- Ruwan Linton Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb WSO2 Inc.; http://wso2.org email: ruwan@...; cell: +94 77 341 3097 blog: http://blog.ruwan.org _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Harm Verhagen-4
::
Rate this Message:
Great,
I'll check it out. On Sun, Aug 9, 2009 at 3:15 AM, Ruwan Linton <ruwan@...> wrote:
_______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Harm Verhagen-4
::
Rate this Message:
A similar feature would also be a good idea for the VFS sender part too.
That the ESB writes a lockfile, while writing the output. This way an external service can also check whenever the esb is ready writing its output, before it starts processing it. What do you think ? Regards, Harm On Tue, Aug 11, 2009 at 10:51 AM, Harm Verhagen <harm.verhagen@...> wrote: Great, _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Ruwan Linton-3
::
Rate this Message:
Harm Verhagen wrote:
> A similar feature would also be a good idea for the VFS sender part too. > That the ESB writes a lockfile, while writing the output. > > This way an external service can also check whenever the esb is ready > writing its output, before it starts processing it. > > What do you think ? :-) , it is already implemented and part of the VFS locking mechanism. Well without that bit there is no guarantee that the vfs receiver do not process a file which is still being uploaded. So it is there on the trunk :-) What we do is basically, we write a lock file before starting to write the actual file and clears it after it is done. So the lock file is a file named {processing file name}".lock" so the receiver looks for this .lock file before reading a file and if that is there it is not gonna read the file. Thanks, Ruwan > > Regards, > Harm > > On Tue, Aug 11, 2009 at 10:51 AM, Harm Verhagen > <harm.verhagen@... <mailto:harm.verhagen@...>> wrote: > > Great, > I'll check it out. > > > On Sun, Aug 9, 2009 at 3:15 AM, Ruwan Linton <ruwan@... > <mailto:ruwan@...>> wrote: > > Ruwan Linton wrote: > > Harm Verhagen wrote: > > > >> So how does wso2esb currently protect against processing > partially > >> uploaded files ? > >> guessing.. Does it somehow/ try to get an exclusive lock on > the file ? > >> > > Once again this is not supported in the ESB, but I see this as a > > critical improvement and hence would like to work on this > ASAP. If you > > are OK with using the latest version of the ESB I could fix > this on the > > trunk and provide a SNAPSHOT build of the vfs transport, > which is > > compatible with the 2.1.0 release, but I am not sure about the > > compatibility with the 1.7.1. > > > > If you cannot move to the new release and still interested > in this fix > > you may buy support from WSO2 to get this fix ported into > the 1.7.1 version. > > > > Anyway I will fix this on the trunk, created the issue at [1] > > > Fixed on the trunk and if you take a fresh build of the vfs > transport > you could get this feature. > > Thanks, > Ruwan > > Thanks, > > Ruwan > > > > [1] - https://wso2.org/jira/browse/ESBJAVA-602 > > > > > >> Regards, > >> Harm > >> > >> On Wed, Aug 5, 2009 at 2:57 PM, Ruwan Linton > <ruwan@... <mailto:ruwan@...> > >> <mailto:ruwan@... <mailto:ruwan@...>>> wrote: > >> > >> Hi Harm, > >> > >> I am afraid, AFAIK, this is not yet supported by the > WSO2 ESB vfs > >> transport by default, but it could be a very good > improvement. You may > >> file an enhancement JIRA for that and we could get that > into the next > >> release. > >> > >> Thanks, > >> Ruwan > >> > >> Harm Verhagen wrote: > >> > Hi, > >> > > >> > I couldn't file the following in the wso2esb manual. > >> > Does the ESB transport.vfs support lockfiles ? > >> > > >> > Say you have one server uploading flat files to a > directory > >> (scp). You > >> > have the ESB processing that library using the vfs > file listener. > >> > to prevent that the ESB starts processing a file is > is partially > >> > uploaded we'd like to put a file (eg: lock.txt) in > the directory, > >> > that blocks reading that file). > >> > Can it do that now using the existing transport.vfs, > or would I > >> have > >> > to write my own file reader ? > >> > > >> > esb:1.7.1 > >> > > >> > > >> > > >> > Regards, > >> > Harm > >> > > >> > > >> > ------------------------------------------------------------------------ > >> > > >> > _______________________________________________ > >> > Esb-java-user mailing list > >> > Esb-java-user@... > <mailto:Esb-java-user@...> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> > >> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > >> > > >> > >> > >> -- > >> Ruwan Linton > >> Technical Lead & Product Manager; WSO2 ESB; > http://wso2.org/esb > >> WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org > >> email: ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>; cell: +94 77 > 341 3097 > >> blog: http://blog.ruwan.org > >> > >> > >> > >> _______________________________________________ > >> Esb-java-user mailing list > >> Esb-java-user@... <mailto:Esb-java-user@...> > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>> > >> https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > >> > >> > >> > ------------------------------------------------------------------------ > >> > >> _______________________________________________ > >> Esb-java-user mailing list > >> Esb-java-user@... <mailto:Esb-java-user@...> > >> https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > >> > >> > > > > > > > > > -- > Ruwan Linton > Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org > email: ruwan@... <mailto:ruwan@...>; cell: +94 77 > 341 3097 > blog: http://blog.ruwan.org > > > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... <mailto:Esb-java-user@...> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > -- Ruwan Linton Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb WSO2 Inc.; http://wso2.org email: ruwan@...; cell: +94 77 341 3097 blog: http://blog.ruwan.org _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Harm Verhagen-4
::
Rate this Message:
Can you make the lockfilename configurable ?
(handy when interfacing with legacy systems, where the dir structure & lockfile convention is already fixed). Regards, Harm
On Wed, Aug 12, 2009 at 7:15 PM, Ruwan Linton <ruwan@...> wrote:
_______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Ruwan Linton-3
::
Rate this Message:
Harm Verhagen wrote:
> Can you make the lockfilename configurable ? > (handy when interfacing with legacy systems, where the dir structure & > lockfile convention is already fixed). Hhhmmm, good point. Will do that enhancement. Thanks, Ruwan > > Regards, > Harm > > > > On Wed, Aug 12, 2009 at 7:15 PM, Ruwan Linton <ruwan@... > <mailto:ruwan@...>> wrote: > > Harm Verhagen wrote: > > A similar feature would also be a good idea for the VFS sender > part too. > > That the ESB writes a lockfile, while writing the output. > > > > This way an external service can also check whenever the esb is > ready > > writing its output, before it starts processing it. > > > > What do you think ? > :-) , it is already implemented and part of the VFS locking mechanism. > Well without that bit there is no guarantee that the vfs receiver > do not > process a file which is still being uploaded. > > So it is there on the trunk :-) > > What we do is basically, we write a lock file before starting to write > the actual file and clears it after it is done. So the lock file is a > file named {processing file name}".lock" so the receiver looks for > this > .lock file before reading a file and if that is there it is not gonna > read the file. > > Thanks, > Ruwan > > > > Regards, > > Harm > > > > On Tue, Aug 11, 2009 at 10:51 AM, Harm Verhagen > > <harm.verhagen@... <mailto:harm.verhagen@...> > <mailto:harm.verhagen@... <mailto:harm.verhagen@...>>> > wrote: > > > > Great, > > I'll check it out. > > > > > > On Sun, Aug 9, 2009 at 3:15 AM, Ruwan Linton <ruwan@... > <mailto:ruwan@...> > > <mailto:ruwan@... <mailto:ruwan@...>>> wrote: > > > > Ruwan Linton wrote: > > > Harm Verhagen wrote: > > > > > >> So how does wso2esb currently protect against processing > > partially > > >> uploaded files ? > > >> guessing.. Does it somehow/ try to get an exclusive > lock on > > the file ? > > >> > > > Once again this is not supported in the ESB, but I see > this as a > > > critical improvement and hence would like to work on this > > ASAP. If you > > > are OK with using the latest version of the ESB I > could fix > > this on the > > > trunk and provide a SNAPSHOT build of the vfs transport, > > which is > > > compatible with the 2.1.0 release, but I am not sure > about the > > > compatibility with the 1.7.1. > > > > > > If you cannot move to the new release and still interested > > in this fix > > > you may buy support from WSO2 to get this fix ported into > > the 1.7.1 version. > > > > > > Anyway I will fix this on the trunk, created the issue > at [1] > > > > > Fixed on the trunk and if you take a fresh build of the vfs > > transport > > you could get this feature. > > > > Thanks, > > Ruwan > > > Thanks, > > > Ruwan > > > > > > [1] - https://wso2.org/jira/browse/ESBJAVA-602 > > > > > > > > >> Regards, > > >> Harm > > >> > > >> On Wed, Aug 5, 2009 at 2:57 PM, Ruwan Linton > > <ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>> > > >> <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>>> wrote: > > >> > > >> Hi Harm, > > >> > > >> I am afraid, AFAIK, this is not yet supported by the > > WSO2 ESB vfs > > >> transport by default, but it could be a very good > > improvement. You may > > >> file an enhancement JIRA for that and we could > get that > > into the next > > >> release. > > >> > > >> Thanks, > > >> Ruwan > > >> > > >> Harm Verhagen wrote: > > >> > Hi, > > >> > > > >> > I couldn't file the following in the wso2esb > manual. > > >> > Does the ESB transport.vfs support lockfiles ? > > >> > > > >> > Say you have one server uploading flat files to a > > directory > > >> (scp). You > > >> > have the ESB processing that library using the vfs > > file listener. > > >> > to prevent that the ESB starts processing a file is > > is partially > > >> > uploaded we'd like to put a file (eg: lock.txt) in > > the directory, > > >> > that blocks reading that file). > > >> > Can it do that now using the existing > transport.vfs, > > or would I > > >> have > > >> > to write my own file reader ? > > >> > > > >> > esb:1.7.1 > > >> > > > >> > > > >> > > > >> > Regards, > > >> > Harm > > >> > > > >> > > > >> > > > ------------------------------------------------------------------------ > > >> > > > >> > _______________________________________________ > > >> > Esb-java-user mailing list > > >> > Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>>> > > >> > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > >> > > > >> > > >> > > >> -- > > >> Ruwan Linton > > >> Technical Lead & Product Manager; WSO2 ESB; > > http://wso2.org/esb > > >> WSO2 <http://wso2.org/esb%0AWSO2> Inc.; > http://wso2.org > > >> email: ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>> > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>>; cell: +94 77 > > 341 3097 > > >> blog: http://blog.ruwan.org > > >> > > >> > > >> > > >> _______________________________________________ > > >> Esb-java-user mailing list > > >> Esb-java-user@... > <mailto:Esb-java-user@...> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>>> > > >> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > >> > > >> > > >> > > > ------------------------------------------------------------------------ > > >> > > >> _______________________________________________ > > >> Esb-java-user mailing list > > >> Esb-java-user@... > <mailto:Esb-java-user@...> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> > > >> https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > >> > > >> > > > > > > > > > > > > > > > -- > > Ruwan Linton > > Technical Lead & Product Manager; WSO2 ESB; > http://wso2.org/esb > > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org > > email: ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>; cell: +94 77 > > 341 3097 > > blog: http://blog.ruwan.org > > > > > > > > _______________________________________________ > > Esb-java-user mailing list > > Esb-java-user@... <mailto:Esb-java-user@...> > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>> > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Esb-java-user mailing list > > Esb-java-user@... <mailto:Esb-java-user@...> > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > > -- > Ruwan Linton > Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org > email: ruwan@... <mailto:ruwan@...>; cell: +94 77 341 3097 > blog: http://blog.ruwan.org > > > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... <mailto:Esb-java-user@...> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > ------------------------------------------------------------------------ > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > -- Ruwan Linton Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb WSO2 Inc.; http://wso2.org email: ruwan@...; cell: +94 77 341 3097 blog: http://blog.ruwan.org _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Harm Verhagen-4
::
Rate this Message:
Hi,
Ruwan wrote: >Hhhmmm, good point. Will do that enhancement. I have some thoughts on this. You might have legacy systems where a single lockfile is used for a directory. (for example: lockfile.txt) You might have legacy systems where a lockfile per output file is used. (your example of {processing file name}".lock) So ideally you'd cover both, with a simple configuration. What do you think ? ref: https://issues.apache.org/jira/browse/SYNAPSE-565 Regards, Harm On Thu, Aug 13, 2009 at 2:24 AM, Ruwan Linton <ruwan@...> wrote:
_______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
|
|
Re: transport.vfs supports lockfile ?
by Ruwan Linton-3
::
Rate this Message:
Harm Verhagen wrote:
> Hi, > > Ruwan wrote: > >Hhhmmm, good point. Will do that enhancement. > > I have some thoughts on this. > > You might have legacy systems where a single lockfile is used for a > directory. > (for example: lockfile.txt) > You might have legacy systems where a lockfile per output file is used. > (your example of {processing file name}".lock) > > So ideally you'd cover both, with a simple configuration. > What do you think ? default will be to lock the file letting the VFS transport process rest of the files in that directory, also when reading files locking the directory is going to be a huge overhead. Thanks, Ruwan > > ref: https://issues.apache.org/jira/browse/SYNAPSE-565 > > Regards, > Harm > > On Thu, Aug 13, 2009 at 2:24 AM, Ruwan Linton <ruwan@... > <mailto:ruwan@...>> wrote: > > Harm Verhagen wrote: > > Can you make the lockfilename configurable ? > > (handy when interfacing with legacy systems, where the dir > structure & > > lockfile convention is already fixed). > Hhhmmm, good point. Will do that enhancement. > > Thanks, > Ruwan > > > > Regards, > > Harm > > > > > > > > On Wed, Aug 12, 2009 at 7:15 PM, Ruwan Linton <ruwan@... > <mailto:ruwan@...> > > <mailto:ruwan@... <mailto:ruwan@...>>> wrote: > > > > Harm Verhagen wrote: > > > A similar feature would also be a good idea for the VFS sender > > part too. > > > That the ESB writes a lockfile, while writing the output. > > > > > > This way an external service can also check whenever the > esb is > > ready > > > writing its output, before it starts processing it. > > > > > > What do you think ? > > :-) , it is already implemented and part of the VFS locking > mechanism. > > Well without that bit there is no guarantee that the vfs > receiver > > do not > > process a file which is still being uploaded. > > > > So it is there on the trunk :-) > > > > What we do is basically, we write a lock file before > starting to write > > the actual file and clears it after it is done. So the lock > file is a > > file named {processing file name}".lock" so the receiver > looks for > > this > > .lock file before reading a file and if that is there it is > not gonna > > read the file. > > > > Thanks, > > Ruwan > > > > > > Regards, > > > Harm > > > > > > On Tue, Aug 11, 2009 at 10:51 AM, Harm Verhagen > > > <harm.verhagen@... <mailto:harm.verhagen@...> > <mailto:harm.verhagen@... <mailto:harm.verhagen@...>> > > <mailto:harm.verhagen@... > <mailto:harm.verhagen@...> <mailto:harm.verhagen@... > <mailto:harm.verhagen@...>>>> > > wrote: > > > > > > Great, > > > I'll check it out. > > > > > > > > > On Sun, Aug 9, 2009 at 3:15 AM, Ruwan Linton > <ruwan@... <mailto:ruwan@...> > > <mailto:ruwan@... <mailto:ruwan@...>> > > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>>> wrote: > > > > > > Ruwan Linton wrote: > > > > Harm Verhagen wrote: > > > > > > > >> So how does wso2esb currently protect against > processing > > > partially > > > >> uploaded files ? > > > >> guessing.. Does it somehow/ try to get an exclusive > > lock on > > > the file ? > > > >> > > > > Once again this is not supported in the ESB, but > I see > > this as a > > > > critical improvement and hence would like to > work on this > > > ASAP. If you > > > > are OK with using the latest version of the ESB I > > could fix > > > this on the > > > > trunk and provide a SNAPSHOT build of the vfs > transport, > > > which is > > > > compatible with the 2.1.0 release, but I am not sure > > about the > > > > compatibility with the 1.7.1. > > > > > > > > If you cannot move to the new release and still > interested > > > in this fix > > > > you may buy support from WSO2 to get this fix > ported into > > > the 1.7.1 version. > > > > > > > > Anyway I will fix this on the trunk, created the > issue > > at [1] > > > > > > > Fixed on the trunk and if you take a fresh build > of the vfs > > > transport > > > you could get this feature. > > > > > > Thanks, > > > Ruwan > > > > Thanks, > > > > Ruwan > > > > > > > > [1] - https://wso2.org/jira/browse/ESBJAVA-602 > > > > > > > > > > > >> Regards, > > > >> Harm > > > >> > > > >> On Wed, Aug 5, 2009 at 2:57 PM, Ruwan Linton > > > <ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>> > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>> > > > >> <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>> > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>>>> wrote: > > > >> > > > >> Hi Harm, > > > >> > > > >> I am afraid, AFAIK, this is not yet > supported by the > > > WSO2 ESB vfs > > > >> transport by default, but it could be a > very good > > > improvement. You may > > > >> file an enhancement JIRA for that and we could > > get that > > > into the next > > > >> release. > > > >> > > > >> Thanks, > > > >> Ruwan > > > >> > > > >> Harm Verhagen wrote: > > > >> > Hi, > > > >> > > > > >> > I couldn't file the following in the wso2esb > > manual. > > > >> > Does the ESB transport.vfs support > lockfiles ? > > > >> > > > > >> > Say you have one server uploading flat > files to a > > > directory > > > >> (scp). You > > > >> > have the ESB processing that library > using the vfs > > > file listener. > > > >> > to prevent that the ESB starts processing > a file is > > > is partially > > > >> > uploaded we'd like to put a file (eg: > lock.txt) in > > > the directory, > > > >> > that blocks reading that file). > > > >> > Can it do that now using the existing > > transport.vfs, > > > or would I > > > >> have > > > >> > to write my own file reader ? > > > >> > > > > >> > esb:1.7.1 > > > >> > > > > >> > > > > >> > > > > >> > Regards, > > > >> > Harm > > > >> > > > > >> > > > > >> > > > > > > ------------------------------------------------------------------------ > > > >> > > > > >> > > _______________________________________________ > > > >> > Esb-java-user mailing list > > > >> > Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>> > > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>>> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>> > > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>>>> > > > >> > > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > >> > > > > >> > > > >> > > > >> -- > > > >> Ruwan Linton > > > >> Technical Lead & Product Manager; WSO2 ESB; > > > http://wso2.org/esb > > > >> WSO2 <http://wso2.org/esb%0AWSO2> Inc.; > > http://wso2.org > > > >> email: ruwan@... > <mailto:ruwan@...> <mailto:ruwan@... > <mailto:ruwan@...>> > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>> > > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>> > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>>>; cell: +94 77 > > > 341 3097 > > > >> blog: http://blog.ruwan.org > > > >> > > > >> > > > >> > > > >> _______________________________________________ > > > >> Esb-java-user mailing list > > > >> Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>>> > > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>>>> > > > >> > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > >> > > > >> > > > >> > > > > > > ------------------------------------------------------------------------ > > > >> > > > >> _______________________________________________ > > > >> Esb-java-user mailing list > > > >> Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> > > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>>> > > > >> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > Ruwan Linton > > > Technical Lead & Product Manager; WSO2 ESB; > > http://wso2.org/esb > > > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; > http://wso2.org > > > email: ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>> > > <mailto:ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>>; cell: +94 77 > > > 341 3097 > > > blog: http://blog.ruwan.org > > > > > > > > > > > > _______________________________________________ > > > Esb-java-user mailing list > > > Esb-java-user@... > <mailto:Esb-java-user@...> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>> > > <mailto:Esb-java-user@... > <mailto:Esb-java-user@...> <mailto:Esb-java-user@... > <mailto:Esb-java-user@...>>> > > > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > _______________________________________________ > > > Esb-java-user mailing list > > > Esb-java-user@... <mailto:Esb-java-user@...> > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>> > > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > > > > > > -- > > Ruwan Linton > > Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org > > email: ruwan@... <mailto:ruwan@...> > <mailto:ruwan@... <mailto:ruwan@...>>; cell: +94 77 341 3097 > > blog: http://blog.ruwan.org > > > > > > > > _______________________________________________ > > Esb-java-user mailing list > > Esb-java-user@... <mailto:Esb-java-user@...> > <mailto:Esb-java-user@... <mailto:Esb-java-user@...>> > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Esb-java-user mailing list > > Esb-java-user@... <mailto:Esb-java-user@...> > > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > > > -- > Ruwan Linton > Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > WSO2 <http://wso2.org/esb%0AWSO2> Inc.; http://wso2.org > email: ruwan@... <mailto:ruwan@...>; cell: +94 77 341 3097 > blog: http://blog.ruwan.org > > > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... <mailto:Esb-java-user@...> > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > > > ------------------------------------------------------------------------ > > _______________________________________________ > Esb-java-user mailing list > Esb-java-user@... > https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user > -- Ruwan Linton Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb WSO2 Inc.; http://wso2.org email: ruwan@...; cell: +94 77 341 3097 blog: http://blog.ruwan.org _______________________________________________ Esb-java-user mailing list Esb-java-user@... https://wso2.org/cgi-bin/mailman/listinfo/esb-java-user |
| Free embeddable forum powered by Nabble | Forum Help |
