|
View:
New views
19 Messages
—
Rating Filter:
Alert me
|
|
|
smooks (EDI) + talendHi there,
I'm trying to use smooks in a talend flow... with some serious problems and I need some help: 1) The smooks component for talend is based on an older version of smooks (1.2.4). Looking on the project site... it appears to be dead. Has anybody tried to upgrade the component al least at version 1.4? 2) The EDI files i'm trying to read have several types of messages. I've read that it's possible to manage this but in version 1.4 (interchange). Two questions about this: a)can i achieve this in version 1.2.4? b)i do not need to parse all the messages, just on or two types. For example... I've the following type of messages: {file begin} UNH+1+CSMPRR:08:1:1A' ... bla bla bla... UNH+1+BIDBRR:07:2:1A+999999' ... bla bla bla... UNH+1+CCPRRR:09:4:1A+99999999' ... bla bla bla... UNH+1+MHISRR:05:0:1A+999999' ... bla bla bla... {file end} I only want to process the CCPRRR message, who has inside... hundreds of items. I don't want to generate the mappings for the other types. Is it possible? 3) In the above example, i've converted the delimiters. In the real file, the delimiters are hexadecimal characters. Example: instead of plus (+), hay have 1D (hex). Couldn't find to tell smooks config file to use this kind of separators. Any ideas? i've files with more than 1Gb, so replace is not an option. Enough for the moment, any help would be apreciated. The type of messages i'm using are from the air industry (amadeus - altea). thanks max.- |
|
|
Re: smooks (EDI) + talendHello Max,
Concerning Talend-Smooks integration dedicated to EDI parsing I can give you some sneak preview of it. Here are some screenshots I made with upcoming Talend Open Studio 4.2.0: http://screencast.com/t/4mDhoHZw7 http://screencast.com/t/rjijRpjB http://screencast.com/t/2Wlfp10ye UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't think you could backport it easily to 1.2.4, however latest Talend components (tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT interchanges. Once your message is transformed to XML you could select any part that you are interested in. Concerning the specific separator char, I think it's doable, but AFAK separators are specified in UNH headers, I suppose they should be also treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you can programmatically override the default separators. Best regards, Renat Zubairov -- Product Owner ESB Tooling Talend Application Integration Division Am 28.04.11 22:14 schrieb "rmg78" unter <rmgiorgi@...>: > >Hi there, >I'm trying to use smooks in a talend flow... with some serious problems >and >I need some help: > >1) The smooks component for talend is based on an older version of smooks >(1.2.4). Looking on the project site... it appears to be dead. Has anybody >tried to upgrade the component al least at version 1.4? > >2) The EDI files i'm trying to read have several types of messages. I've >read that it's possible to manage this but in version 1.4 (interchange). >Two >questions about this: a)can i achieve this in version 1.2.4? b)i do not >need >to parse all the messages, just on or two types. For example... I've the >following type of messages: > >{file begin} > UNH+1+CSMPRR:08:1:1A' ... bla bla bla... > UNH+1+BIDBRR:07:2:1A+999999' ... bla bla bla... > UNH+1+CCPRRR:09:4:1A+99999999' ... bla bla bla... > UNH+1+MHISRR:05:0:1A+999999' ... bla bla bla... >{file end} > >I only want to process the CCPRRR message, who has inside... hundreds of >items. I don't want to generate the mappings for the other types. Is it >possible? > >3) In the above example, i've converted the delimiters. In the real file, >the delimiters are hexadecimal characters. Example: instead of plus (+), >hay >have 1D (hex). Couldn't find to tell smooks config file to use this kind >of >separators. Any ideas? i've files with more than 1Gb, so replace is not an >option. > >Enough for the moment, any help would be apreciated. The type of messages >i'm using are from the air industry (amadeus - altea). >thanks >max.- >-- >View this message in context: >http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.html >Sent from the milyn - user mailing list archive at Nabble.com. > > >--------------------------------------------------------------------- >To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: smooks (EDI) + talendHi Renat
Like I said before, i wanted to test it before give you an answer, and I did it: awesome. I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with the amadeus mappings and after a few little problems it worked like a charm. I had to touch the mapping a little... for the upgrade from smooks 1.2.4 to 1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more. Going back to the original questions, 1 & 2 are solved (smooks discards the mappings not included in the jar, excelent), and I'm working on number 3. By the way, in the screenshots you uploaded you have the EDIFACT bullet in the metadata and the mapping editor -> the RC3 dons't include this, need to wait to 4.2.0 right? For future questions about talend flow originated from this component, do I need to write them in another forum? can you tell me which one. Thanks again! max.- From
|
|
|
Re: smooks (EDI) + talendOops!!! I made a mistake on the previous test, smooks doesn't discard the unmapped messages.
Is there a solution for my question number two ? i mean... having: {file begin} UNH+1+CSMPRR:08:1:1A' ... bla bla bla... UNH+1+BIDBRR:07:2:1A+999999' ... bla bla bla... UNH+1+CCPRRR:09:4:1A+99999999' ... bla bla bla... UNH+1+MHISRR:05:0:1A+999999' ... bla bla bla... {file end} just parse the CCPRRR discarding the other messages and having no mapping of them in the jar? thanks max.-
|
|
|
Re: smooks (EDI) + talendHello Max,
See my comments below: > I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with >the > amadeus mappings and after a few little problems it worked like a charm. If I understood you correctly you deployed additional UN/EDIFACT mapping JAR file to the component, how did you do it? Do you think we should extend component so that such use-case would be supported. > By the way, in the screenshots you uploaded you have the EDIFACT bullet >in > the metadata and the mapping editor -> the RC3 dons't include this, need >to > wait to 4.2.0 right? These screenshots were made not from the tEDIFACTtoXML component but from the tEDI component which is a part of commercial Talend offering. In commercial edition (TIS) you would be able to define a project-wide common metadata based on UN/EDIFACT mappings which would directly map to flat Talend schemas (without any XML in-between). Generally same Smooks functionality, just a little more convenient. > For future questions about talend flow originated from this component, >do I > need to write them in another forum? can you tell me which one. Sure, you can ask your question on Forum: http://talendforge.org/forum/ Best regards, Renat Zubairov -- Product Owner ESB Tooling Talend Application Integration Division Am 29.04.11 17:08 schrieb "rmg78" unter <rmgiorgi@...>: > >Hi Renat > >Like I said before, i wanted to test it before give you an answer, and I >did >it: awesome. >I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with >the >amadeus mappings and after a few little problems it worked like a charm. >I had to touch the mapping a little... for the upgrade from smooks 1.2.4 >to >1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more. > >Going back to the original questions, 1 & 2 are solved (smooks discards >the >mappings not included in the jar, excelent), and I'm working on number 3. > >By the way, in the screenshots you uploaded you have the EDIFACT bullet in >the metadata and the mapping editor -> the RC3 dons't include this, need >to >wait to 4.2.0 right? > >For future questions about talend flow originated from this component, do >I >need to write them in another forum? can you tell me which one. > >Thanks again! >max.- > >From > > >Renat Zubairov-4 wrote: >> >> Hello Max, >> >> Concerning Talend-Smooks integration dedicated to EDI parsing I can give >> you some sneak preview of it. Here are some screenshots I made with >> upcoming Talend Open Studio 4.2.0: >> >> http://screencast.com/t/4mDhoHZw7 >> http://screencast.com/t/rjijRpjB >> http://screencast.com/t/2Wlfp10ye >> >> >> UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't >>think >> you could backport it easily to 1.2.4, however latest Talend components >> (tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT >> interchanges. >> >> Once your message is transformed to XML you could select any part that >>you >> are interested in. >> >> Concerning the specific separator char, I think it's doable, but AFAK >> separators are specified in UNH headers, I suppose they should be also >> treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you >>can >> programmatically override the default separators. >> >> Best regards, >> Renat Zubairov >> >> -- >> Product Owner ESB Tooling >> Talend Application Integration Division >> >> >> >> >> >> Am 28.04.11 22:14 schrieb "rmg78" unter <rmgiorgi@...>: >> >>> >>>Hi there, >>>I'm trying to use smooks in a talend flow... with some serious problems >>>and >>>I need some help: >>> >>>1) The smooks component for talend is based on an older version of >>>smooks >>>(1.2.4). Looking on the project site... it appears to be dead. Has >>>anybody >>>tried to upgrade the component al least at version 1.4? >>> >>>2) The EDI files i'm trying to read have several types of messages. I've >>>read that it's possible to manage this but in version 1.4 (interchange). >>>Two >>>questions about this: a)can i achieve this in version 1.2.4? b)i do not >>>need >>>to parse all the messages, just on or two types. For example... I've the >>>following type of messages: >>> >>>{file begin} >>> UNH+1+CSMPRR:08:1:1A' ... bla bla bla... >>> UNH+1+BIDBRR:07:2:1A+999999' ... bla bla bla... >>> UNH+1+CCPRRR:09:4:1A+99999999' ... bla bla bla... >>> UNH+1+MHISRR:05:0:1A+999999' ... bla bla bla... >>>{file end} >>> >>>I only want to process the CCPRRR message, who has inside... hundreds of >>>items. I don't want to generate the mappings for the other types. Is it >>>possible? >>> >>>3) In the above example, i've converted the delimiters. In the real >>>file, >>>the delimiters are hexadecimal characters. Example: instead of plus (+), >>>hay >>>have 1D (hex). Couldn't find to tell smooks config file to use this kind >>>of >>>separators. Any ideas? i've files with more than 1Gb, so replace is not >>>an >>>option. >>> >>>Enough for the moment, any help would be apreciated. The type of >>>messages >>>i'm using are from the air industry (amadeus - altea). >>>thanks >>>max.- >>>-- >>>View this message in context: >>>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.ht >>>ml >>>Sent from the milyn - user mailing list archive at Nabble.com. >>> >>> >>>--------------------------------------------------------------------- >>>To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> >> > >-- >View this message in context: >http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31506305.html >Sent from the milyn - user mailing list archive at Nabble.com. > > >--------------------------------------------------------------------- >To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: smooks (EDI) + talendHello Max,
The problem you are running in is a Smooks UN/EDIFACT parser limitation. Right now parser expects one _interchange_ (UNH...UNZ) in one file. So far it was always the case. Is this limitation a significant issue for you? May be if you could describe your use-case it will be more clear for us why and when we could run into that limitation. Best regards, Renat Zubairov -- Product Owner ESB Tooling Talend Application Integration Division Am 29.04.11 20:41 schrieb "rmg78" unter <rmgiorgi@...>: > >Oops!!! I made a mistake on the previous test, smooks doesn't discard the >unmapped messages. >Is there a solution for my question number two ? i mean... having: > >{file begin} > UNH+1+CSMPRR:08:1:1A' ... bla bla bla... > UNH+1+BIDBRR:07:2:1A+999999' ... bla bla bla... > UNH+1+CCPRRR:09:4:1A+99999999' ... bla bla bla... > UNH+1+MHISRR:05:0:1A+999999' ... bla bla bla... >{file end} > >just parse the CCPRRR discarding the other messages and having no mapping >of >them in the jar? > >thanks >max.- > > >rmg78 wrote: >> >> Hi Renat >> >> Like I said before, i wanted to test it before give you an answer, and I >> did it: awesome. >> I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with >> the amadeus mappings and after a few little problems it worked like a >> charm. >> I had to touch the mapping a little... for the upgrade from smooks 1.2.4 >> to 1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more. >> >> Going back to the original questions, 1 & 2 are solved (smooks discards >> the mappings not included in the jar, excelent), and I'm working on >>number >> 3. >> >> By the way, in the screenshots you uploaded you have the EDIFACT bullet >>in >> the metadata and the mapping editor -> the RC3 dons't include this, need >> to wait to 4.2.0 right? >> >> For future questions about talend flow originated from this component, >>do >> I need to write them in another forum? can you tell me which one. >> >> Thanks again! >> max.- >> >> From >> >> >> Renat Zubairov-4 wrote: >>> >>> Hello Max, >>> >>> Concerning Talend-Smooks integration dedicated to EDI parsing I can >>>give >>> you some sneak preview of it. Here are some screenshots I made with >>> upcoming Talend Open Studio 4.2.0: >>> >>> http://screencast.com/t/4mDhoHZw7 >>> http://screencast.com/t/rjijRpjB >>> http://screencast.com/t/2Wlfp10ye >>> >>> >>> UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't >>> think >>> you could backport it easily to 1.2.4, however latest Talend components >>> (tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT >>> interchanges. >>> >>> Once your message is transformed to XML you could select any part that >>> you >>> are interested in. >>> >>> Concerning the specific separator char, I think it's doable, but AFAK >>> separators are specified in UNH headers, I suppose they should be also >>> treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you >>>can >>> programmatically override the default separators. >>> >>> Best regards, >>> Renat Zubairov >>> >>> -- >>> Product Owner ESB Tooling >>> Talend Application Integration Division >>> >>> >>> >>> >>> >>> Am 28.04.11 22:14 schrieb "rmg78" unter <rmgiorgi@...>: >>> >>>> >>>>Hi there, >>>>I'm trying to use smooks in a talend flow... with some serious problems >>>>and >>>>I need some help: >>>> >>>>1) The smooks component for talend is based on an older version of >>>>smooks >>>>(1.2.4). Looking on the project site... it appears to be dead. Has >anybody >>>>tried to upgrade the component al least at version 1.4? >>>> >>>>2) The EDI files i'm trying to read have several types of messages. >>>>I've >>>>read that it's possible to manage this but in version 1.4 >>>>(interchange). >>>>Two >>>>questions about this: a)can i achieve this in version 1.2.4? b)i do not >>>>need >>>>to parse all the messages, just on or two types. For example... I've >>>>the >>>>following type of messages: >>>> >>>>{file begin} >>>> UNH+1+CSMPRR:08:1:1A' ... bla bla bla... >>>> UNH+1+BIDBRR:07:2:1A+999999' ... bla bla bla... >>>> UNH+1+CCPRRR:09:4:1A+99999999' ... bla bla bla... >>>> UNH+1+MHISRR:05:0:1A+999999' ... bla bla bla... >>>>{file end} >>>> >>>>I only want to process the CCPRRR message, who has inside... hundreds >>>>of >>>>items. I don't want to generate the mappings for the other types. Is it >>>>possible? >>>> >>>>3) In the above example, i've converted the delimiters. In the real >>>>file, >>>>the delimiters are hexadecimal characters. Example: instead of plus >>>>(+), >>>>hay >>>>have 1D (hex). Couldn't find to tell smooks config file to use this >>>>kind >>>>of >>>>separators. Any ideas? i've files with more than 1Gb, so replace is not >an >>>>option. >>>> >>>>Enough for the moment, any help would be apreciated. The type of >>>>messages >>>>i'm using are from the air industry (amadeus - altea). >>>>thanks >>>>max.- >>>>-- >>>>View this message in context: >>>>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.h >>>>tml >>>>Sent from the milyn - user mailing list archive at Nabble.com. >>>> >>>> >>>>--------------------------------------------------------------------- >>>>To unsubscribe from this list, please visit: >>>> >>>> http://xircles.codehaus.org/manage_email >>>> >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >>> >> >> > >-- >View this message in context: >http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31507923.html >Sent from the milyn - user mailing list archive at Nabble.com. > > >--------------------------------------------------------------------- >To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: smooks (EDI) + talendHi Renat,
In order to deploy the additional mappings i looked one of the existing jars, and based on that i replicated the content with my mappings. After that, i copied the jar in the talend component and in the lib/java folder and edited the component's files (*.properties and *.xml). The other thing i changed, was the configuration/ComponentsCache.javacache file, appending the new library. With these changes, I can see my "amadeus" package in the component's combo. I do not consider that these mappings should be integrated in TOS because they aren't standard mappings, the mappings are property of amadeus. In regard of TIS, i'll see it with my project manager. Thanks a lot for your support, max.-
|
|
|
Re: smooks (EDI) + talendRenat,
The real life case is the following: 1- I receive a tar-zipped file, with several .data files. These files are sent from the amadeus server directly to our filesystem. 2- each .data file contains several edifact messages, being one of them the CCPRRR that i'm interested in. i need to discard the rest because i don't need them, and i don't have such mappings. As i explained, each file contains the messages as: Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1' Line 2: UNH+1+BIDBRR:07:2:1A+999999' ... UNT+761+1' Line 3: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+762+1' Line 4: UNH+1+MHISRR:05:0:1A+999999' ... UNT+763+1' Line 5: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+764+1' ... With the component that i've now, if the file contains only CCPRRR messages it works fine, but in the above example... fails saying: Starting job EDIToXML_CCPRRR at 15:26 02/05/2011. [statistics] connecting to socket on port 3813 [statistics] connected Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model. Exception in component tFileOutputXML_1 java.lang.NullPointerException at aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDIToXML_CCPRRR.java:649) at aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRRR.java:911) at aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java:779) [statistics] disconnected Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1] What i don't want is to map every kind of message they sent in the files. any ideas? Thanks max
|
|
|
Re: smooks (EDI) + talendHello Max,
Thnx for the info. Your feedback is very important to us, based on it we will prioritize new features for the component, for example ability to upload user-defined mappings :) Best regards, Renat Zubairov -- Product Owner ESB Tooling Talend Application Integration Division Am 02.05.11 18:37 schrieb "rmg78" unter <rmgiorgi@...>: >In order to deploy the additional mappings i looked one of the existing >jars, and based on that i replicated the content with my mappings. >After that, i copied the jar in the talend component and in the lib/java >folder and edited the component's files (*.properties and *.xml). The >other >thing i changed, was the configuration/ComponentsCache.javacache file, >appending the new library. >With these changes, I can see my "amadeus" package in the component's >combo. > >I do not consider that these mappings should be integrated in TOS because >they aren't standard mappings, the mappings are property of amadeus. > >In regard of TIS, i'll see it with my project manager. >Thanks a lot for your support, >max.- --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Ignoring unknown messages within one interchange* Moved discussion to the new subject (thread)
Hello Max, Thank you for the description of your use-case, it's now much more clear for me. It is a very important information for whole Smooks team. Your sample is very interesting, usually we expect following structure: UNB #Interchange start UNH+1+XXX # Message start UNT # Message end UNH+2+XXX # Message start UNT # Message end ... UNZ #Interchange end Apparently in your case you don't have the interchange header but only individual messages, is it a specific variation of the UN/EDIFACT standard? If it is a standard I think it would make sense for Smooks to support it. If it is not a standard we could discuss on how to make our parsing flexible so that Smooks users could declaratively (or with little programming efforts) parse such structures. Do you know wherever it's some specific standard behind this structure? Best regards, Renat Zubairov -- Product Owner ESB Tooling Talend Application Integration Division Am 02.05.11 20:31 schrieb "rmg78" unter <rmgiorgi@...>: >Renat, >The real life case is the following: > >1- I receive a tar-zipped file, with several .data files. These files are >sent from the amadeus server directly to our filesystem. >2- each .data file contains several edifact messages, being one of them >the >CCPRRR that i'm interested in. i need to discard the rest because i don't >need them, and i don't have such mappings. > >As i explained, each file contains the messages as: > >Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1' >Line 2: UNH+1+BIDBRR:07:2:1A+999999' ... UNT+761+1' >Line 3: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+762+1' >Line 4: UNH+1+MHISRR:05:0:1A+999999' ... UNT+763+1' >Line 5: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+764+1' >... >With the component that i've now, if the file contains only CCPRRR >messages >it works fine, but in the above example... fails saying: > >Starting job EDIToXML_CCPRRR at 15:26 02/05/2011. > >[statistics] connecting to socket on port 3813 >[statistics] connected >Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model. >Exception in component tFileOutputXML_1 >java.lang.NullPointerException > at >aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDI >ToXML_CCPRRR.java:649) > at >aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRR >R.java:911) > at >aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java: >779) >[statistics] disconnected >Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1] > >What i don't want is to map every kind of message they sent in the files. >any ideas? > >Thanks >max --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Ignoring unknown messages within one interchangeRenat,
See my comments below: >is it a specific variation of the UN/EDIFACT standard? >Do you know wherever it's some specific standard behind this structure? I really don't know how to answer you, because the only thing we know is that this is the edifact standard for the whole air industry that use amadeus as partner (the other big player is sabre). But looking at the http://www.unece.org/trade/untdid/down_index.htm these messages aren't official. I think they have taken the protocol and implemented their own messages and kept it private. In fact, if you want the layout of these messages you must have access to their website to download them. :( I would be very happy to send you one example... but this is production data and I can't publish such things on the forum. Anyway, i'll try to answer you and post all the data that i can. Tell me what do you need and i'll try to narrow the examples. I'll append the 2 segments you say to see if the parser works fine discarding the messages i don't want. do you think it's going to work? max.-
|
|
|
Re: Ignoring unknown messages within one interchangeHey guys.... Trying to figure out what's going wrong. I was thinking
about it and though Smooks should function OK with just the messages (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest. It worked fine with just the UNH and UNT segments (plus messages inside). So is it just that we can't ignore unknown messages? T. On 02/05/2011 20:02, Renat Zubairov wrote: > * Moved discussion to the new subject (thread) > > Hello Max, > > Thank you for the description of your use-case, it's now much more clear > for me. It is a very important information for whole Smooks team. > > Your sample is very interesting, usually we expect following structure: > > UNB #Interchange start > UNH+1+XXX # Message start > UNT # Message end > UNH+2+XXX # Message start > UNT # Message end > > ... > UNZ #Interchange end > > Apparently in your case you don't have the interchange header but only > individual messages, is it a specific variation of the UN/EDIFACT > standard? If it is a standard I think it would make sense for Smooks to > support it. If it is not a standard we could discuss on how to make our > parsing flexible so that Smooks users could declaratively (or with little > programming efforts) parse such structures. > > Do you know wherever it's some specific standard behind this structure? > > > Best regards, > Renat Zubairov > > -- > Product Owner ESB Tooling > Talend Application Integration Division > > > > Am 02.05.11 20:31 schrieb "rmg78" unter<rmgiorgi@...>: > >> Renat, >> The real life case is the following: >> >> 1- I receive a tar-zipped file, with several .data files. These files are >> sent from the amadeus server directly to our filesystem. >> 2- each .data file contains several edifact messages, being one of them >> the >> CCPRRR that i'm interested in. i need to discard the rest because i don't >> need them, and i don't have such mappings. >> >> As i explained, each file contains the messages as: >> >> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1' >> Line 2: UNH+1+BIDBRR:07:2:1A+999999' ... UNT+761+1' >> Line 3: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+762+1' >> Line 4: UNH+1+MHISRR:05:0:1A+999999' ... UNT+763+1' >> Line 5: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+764+1' >> ... >> With the component that i've now, if the file contains only CCPRRR >> messages >> it works fine, but in the above example... fails saying: >> >> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011. >> >> [statistics] connecting to socket on port 3813 >> [statistics] connected >> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model. >> Exception in component tFileOutputXML_1 >> java.lang.NullPointerException >> at >> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDI >> ToXML_CCPRRR.java:649) >> at >> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRR >> R.java:911) >> at >> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java: >> 779) >> [statistics] disconnected >> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1] >> >> What i don't want is to map every kind of message they sent in the files. >> any ideas? >> >> Thanks >> max > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Ignoring unknown messages within one interchangeHi Tom,
Exactly, it's all about ignoring unknown (or unmapped) messages, the rest seems to be working fine despite i couldn´t test the complete files (because of the mixed messages). Maybe there could be a flag to tell smooks to be strict (fail if exists unmapped messages, the actual behavior) or not to be strict (ignore the unmapped messages). Do you think this could be done? thanks in advance max.-
|
|
|
Re: Ignoring unknown messages within one interchangeHello Max, Tom,
IMO ignoring message types where we can't find mapping information could be a good feature candidate, however with 1.5 release we have a significant difference to 1.4. In 1.4 you would need to declare explicit list of mappings available for UN/EDIFACT parser, in 1.5 we load them lazily based on the information we could find in the classpath. @Max: can you create a JIRA issue. We could use a specific FEATURE flag for the UN/EDIFACT parser just like we do with ignore new-lines. @Tom: what do you think? Best regards, Renat Zubairov Am 03.05.11 03:00 schrieb "rmg78" unter <rmgiorgi@...>: > >Hi Tom, >Exactly, it's all about ignoring unknown (or unmapped) messages, the rest >seems to be working fine despite i couldn´t test the complete files >(because >of the mixed messages). >Maybe there could be a flag to tell smooks to be strict (fail if exists >unmapped messages, the actual behavior) or not to be strict (ignore the >unmapped messages). Do you think this could be done? > >thanks in advance >max.- > > >TomFennelly wrote: >> >> Hey guys.... Trying to figure out what's going wrong. I was thinking >> about it and though Smooks should function OK with just the messages >> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest. It >> worked fine with just the UNH and UNT segments (plus messages inside). >> >> So is it just that we can't ignore unknown messages? >> >> T. >> >> >> >> On 02/05/2011 20:02, Renat Zubairov wrote: >>> * Moved discussion to the new subject (thread) >>> >>> Hello Max, >>> >>> Thank you for the description of your use-case, it's now much more >>>clear >>> for me. It is a very important information for whole Smooks team. >>> >>> Your sample is very interesting, usually we expect following structure: >>> >>> UNB #Interchange start >>> UNH+1+XXX # Message start >>> UNT # Message end >>> UNH+2+XXX # Message start >>> UNT # Message end >>> >>> ... >>> UNZ #Interchange end >>> >>> Apparently in your case you don't have the interchange header but only >>> individual messages, is it a specific variation of the UN/EDIFACT >>> standard? If it is a standard I think it would make sense for Smooks to >>> support it. If it is not a standard we could discuss on how to make our >>> parsing flexible so that Smooks users could declaratively (or with >>>little >>> programming efforts) parse such structures. >>> >>> Do you know wherever it's some specific standard behind this structure? >>> >>> >>> Best regards, >>> Renat Zubairov >>> >>> -- >>> Product Owner ESB Tooling >>> Talend Application Integration Division >>> >>> >>> >>> Am 02.05.11 20:31 schrieb "rmg78" unter<rmgiorgi@...>: >>> >>>> Renat, >>>> The real life case is the following: >>>> >>>> 1- I receive a tar-zipped file, with several .data files. These files >>>> are >>>> sent from the amadeus server directly to our filesystem. >>>> 2- each .data file contains several edifact messages, being one of >>>>them >>>> the >>>> CCPRRR that i'm interested in. i need to discard the rest because i >>>> don't >>>> need them, and i don't have such mappings. >>>> >>>> As i explained, each file contains the messages as: >>>> >>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1' >>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999' ... UNT+761+1' >>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+762+1' >>>> Line 4: UNH+1+MHISRR:05:0:1A+999999' ... UNT+763+1' >>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+764+1' >>>> ... >>>> With the component that i've now, if the file contains only CCPRRR >>>> messages >>>> it works fine, but in the above example... fails saying: >>>> >>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011. >>>> >>>> [statistics] connecting to socket on port 3813 >>>> [statistics] connected >>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping >>>> model. >>>> Exception in component tFileOutputXML_1 >>>> java.lang.NullPointerException >>>> at >>>> >>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process( >>>>EDI >>>> ToXML_CCPRRR.java:649) >>>> at >>>> >>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC >>>>PRR >>>> R.java:911) >>>> at >>>> >>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja >>>>va: >>>> 779) >>>> [statistics] disconnected >>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1] >>>> >>>> What i don't want is to map every kind of message they sent in the >>>> files. >>>> any ideas? >>>> >>>> Thanks >>>> max >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> >> > >-- >View this message in context: >http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html >Sent from the milyn - user mailing list archive at Nabble.com. > > >--------------------------------------------------------------------- >To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Ignoring unknown messages within one interchangeHey guys.
Yes... I'd be cool with adding that. I'll create a JIRA for it. T. On 03/05/2011 07:45, Renat Zubairov wrote: > Hello Max, Tom, > > IMO ignoring message types where we can't find mapping information could > be a good feature candidate, however with 1.5 release we have a > significant difference to 1.4. In 1.4 you would need to declare explicit > list of mappings available for UN/EDIFACT parser, in 1.5 we load them > lazily based on the information we could find in the classpath. > > @Max: can you create a JIRA issue. We could use a specific FEATURE flag > for the UN/EDIFACT parser just like we do with ignore new-lines. > @Tom: what do you think? > > Best regards, > Renat Zubairov > > > > Am 03.05.11 03:00 schrieb "rmg78" unter<rmgiorgi@...>: > >> Hi Tom, >> Exactly, it's all about ignoring unknown (or unmapped) messages, the rest >> seems to be working fine despite i couldn´t test the complete files >> (because >> of the mixed messages). >> Maybe there could be a flag to tell smooks to be strict (fail if exists >> unmapped messages, the actual behavior) or not to be strict (ignore the >> unmapped messages). Do you think this could be done? >> >> thanks in advance >> max.- >> >> >> TomFennelly wrote: >>> Hey guys.... Trying to figure out what's going wrong. I was thinking >>> about it and though Smooks should function OK with just the messages >>> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest. It >>> worked fine with just the UNH and UNT segments (plus messages inside). >>> >>> So is it just that we can't ignore unknown messages? >>> >>> T. >>> >>> >>> >>> On 02/05/2011 20:02, Renat Zubairov wrote: >>>> * Moved discussion to the new subject (thread) >>>> >>>> Hello Max, >>>> >>>> Thank you for the description of your use-case, it's now much more >>>> clear >>>> for me. It is a very important information for whole Smooks team. >>>> >>>> Your sample is very interesting, usually we expect following structure: >>>> >>>> UNB #Interchange start >>>> UNH+1+XXX # Message start >>>> UNT # Message end >>>> UNH+2+XXX # Message start >>>> UNT # Message end >>>> >>>> ... >>>> UNZ #Interchange end >>>> >>>> Apparently in your case you don't have the interchange header but only >>>> individual messages, is it a specific variation of the UN/EDIFACT >>>> standard? If it is a standard I think it would make sense for Smooks to >>>> support it. If it is not a standard we could discuss on how to make our >>>> parsing flexible so that Smooks users could declaratively (or with >>>> little >>>> programming efforts) parse such structures. >>>> >>>> Do you know wherever it's some specific standard behind this structure? >>>> >>>> >>>> Best regards, >>>> Renat Zubairov >>>> >>>> -- >>>> Product Owner ESB Tooling >>>> Talend Application Integration Division >>>> >>>> >>>> >>>> Am 02.05.11 20:31 schrieb "rmg78" unter<rmgiorgi@...>: >>>> >>>>> Renat, >>>>> The real life case is the following: >>>>> >>>>> 1- I receive a tar-zipped file, with several .data files. These files >>>>> are >>>>> sent from the amadeus server directly to our filesystem. >>>>> 2- each .data file contains several edifact messages, being one of >>>>> them >>>>> the >>>>> CCPRRR that i'm interested in. i need to discard the rest because i >>>>> don't >>>>> need them, and i don't have such mappings. >>>>> >>>>> As i explained, each file contains the messages as: >>>>> >>>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1' >>>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999' ... UNT+761+1' >>>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+762+1' >>>>> Line 4: UNH+1+MHISRR:05:0:1A+999999' ... UNT+763+1' >>>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+764+1' >>>>> ... >>>>> With the component that i've now, if the file contains only CCPRRR >>>>> messages >>>>> it works fine, but in the above example... fails saying: >>>>> >>>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011. >>>>> >>>>> [statistics] connecting to socket on port 3813 >>>>> [statistics] connected >>>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping >>>>> model. >>>>> Exception in component tFileOutputXML_1 >>>>> java.lang.NullPointerException >>>>> at >>>>> >>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process( >>>>> EDI >>>>> ToXML_CCPRRR.java:649) >>>>> at >>>>> >>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC >>>>> PRR >>>>> R.java:911) >>>>> at >>>>> >>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja >>>>> va: >>>>> 779) >>>>> [statistics] disconnected >>>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1] >>>>> >>>>> What i don't want is to map every kind of message they sent in the >>>>> files. >>>>> any ideas? >>>>> >>>>> Thanks >>>>> max >>>> --------------------------------------------------------------------- >>>> To unsubscribe from this list, please visit: >>>> >>>> http://xircles.codehaus.org/manage_email >>>> >>>> >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >>> >> -- >> View this message in context: >> http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html >> Sent from the milyn - user mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Ignoring unknown messages within one interchangehttp://jira.codehaus.org/browse/MILYN-602
On 03/05/2011 10:08, Tom Fennelly wrote: > Hey guys. > > Yes... I'd be cool with adding that. I'll create a JIRA for it. > > T. > > > On 03/05/2011 07:45, Renat Zubairov wrote: >> Hello Max, Tom, >> >> IMO ignoring message types where we can't find mapping information could >> be a good feature candidate, however with 1.5 release we have a >> significant difference to 1.4. In 1.4 you would need to declare explicit >> list of mappings available for UN/EDIFACT parser, in 1.5 we load them >> lazily based on the information we could find in the classpath. >> >> @Max: can you create a JIRA issue. We could use a specific FEATURE flag >> for the UN/EDIFACT parser just like we do with ignore new-lines. >> @Tom: what do you think? >> >> Best regards, >> Renat Zubairov >> >> >> >> Am 03.05.11 03:00 schrieb "rmg78" unter<rmgiorgi@...>: >> >>> Hi Tom, >>> Exactly, it's all about ignoring unknown (or unmapped) messages, the >>> rest >>> seems to be working fine despite i couldn´t test the complete files >>> (because >>> of the mixed messages). >>> Maybe there could be a flag to tell smooks to be strict (fail if exists >>> unmapped messages, the actual behavior) or not to be strict (ignore the >>> unmapped messages). Do you think this could be done? >>> >>> thanks in advance >>> max.- >>> >>> >>> TomFennelly wrote: >>>> Hey guys.... Trying to figure out what's going wrong. I was thinking >>>> about it and though Smooks should function OK with just the messages >>>> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest. It >>>> worked fine with just the UNH and UNT segments (plus messages inside). >>>> >>>> So is it just that we can't ignore unknown messages? >>>> >>>> T. >>>> >>>> >>>> >>>> On 02/05/2011 20:02, Renat Zubairov wrote: >>>>> * Moved discussion to the new subject (thread) >>>>> >>>>> Hello Max, >>>>> >>>>> Thank you for the description of your use-case, it's now much more >>>>> clear >>>>> for me. It is a very important information for whole Smooks team. >>>>> >>>>> Your sample is very interesting, usually we expect following >>>>> structure: >>>>> >>>>> UNB #Interchange start >>>>> UNH+1+XXX # Message start >>>>> UNT # Message end >>>>> UNH+2+XXX # Message start >>>>> UNT # Message end >>>>> >>>>> ... >>>>> UNZ #Interchange end >>>>> >>>>> Apparently in your case you don't have the interchange header but >>>>> only >>>>> individual messages, is it a specific variation of the UN/EDIFACT >>>>> standard? If it is a standard I think it would make sense for >>>>> Smooks to >>>>> support it. If it is not a standard we could discuss on how to >>>>> make our >>>>> parsing flexible so that Smooks users could declaratively (or with >>>>> little >>>>> programming efforts) parse such structures. >>>>> >>>>> Do you know wherever it's some specific standard behind this >>>>> structure? >>>>> >>>>> >>>>> Best regards, >>>>> Renat Zubairov >>>>> >>>>> -- >>>>> Product Owner ESB Tooling >>>>> Talend Application Integration Division >>>>> >>>>> >>>>> >>>>> Am 02.05.11 20:31 schrieb "rmg78" unter<rmgiorgi@...>: >>>>> >>>>>> Renat, >>>>>> The real life case is the following: >>>>>> >>>>>> 1- I receive a tar-zipped file, with several .data files. These >>>>>> files >>>>>> are >>>>>> sent from the amadeus server directly to our filesystem. >>>>>> 2- each .data file contains several edifact messages, being one of >>>>>> them >>>>>> the >>>>>> CCPRRR that i'm interested in. i need to discard the rest because i >>>>>> don't >>>>>> need them, and i don't have such mappings. >>>>>> >>>>>> As i explained, each file contains the messages as: >>>>>> >>>>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1' >>>>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999' ... UNT+761+1' >>>>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+762+1' >>>>>> Line 4: UNH+1+MHISRR:05:0:1A+999999' ... UNT+763+1' >>>>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999' ... UNT+764+1' >>>>>> ... >>>>>> With the component that i've now, if the file contains only CCPRRR >>>>>> messages >>>>>> it works fine, but in the above example... fails saying: >>>>>> >>>>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011. >>>>>> >>>>>> [statistics] connecting to socket on port 3813 >>>>>> [statistics] connected >>>>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping >>>>>> model. >>>>>> Exception in component tFileOutputXML_1 >>>>>> java.lang.NullPointerException >>>>>> at >>>>>> >>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process( >>>>>> >>>>>> EDI >>>>>> ToXML_CCPRRR.java:649) >>>>>> at >>>>>> >>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC >>>>>> >>>>>> PRR >>>>>> R.java:911) >>>>>> at >>>>>> >>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja >>>>>> >>>>>> va: >>>>>> 779) >>>>>> [statistics] disconnected >>>>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1] >>>>>> >>>>>> What i don't want is to map every kind of message they sent in the >>>>>> files. >>>>>> any ideas? >>>>>> >>>>>> Thanks >>>>>> max >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe from this list, please visit: >>>>> >>>>> http://xircles.codehaus.org/manage_email >>>>> >>>>> >>>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe from this list, please visit: >>>> >>>> http://xircles.codehaus.org/manage_email >>>> >>>> >>>> >>>> >>> -- >>> View this message in context: >>> http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html >>> >>> Sent from the milyn - user mailing list archive at Nabble.com. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Ignoring unknown messages within one interchangeThanks Tom, it's good to know about the new feature.
Do you think it's possible to have a patch for the 1.4 version? I think Talend 4.2 will be available soon but with smooks 1.4. I don't know the roadmap for having smooks 1.5 in Talend... but i think will be months right? thanks for your support max.-
|
|
|
Re: Ignoring unknown messages within one interchangeHi Renat,
Tom posted the new Jira issue with the feature request (excellent). I didn't understand the problem about smooks 1.5 and the classpath, but it seems that having a flag for ignoring unmapped messages it's a good idea. The feature it's proposed for smooks 1.5, do you have an idea of when it's going to be available in Talend? thanks max.-
|
|
|
Re: smooks (EDI) + talendHey,
I actually do work on the similar edifact syntax and I know it's private to Amadeus, but still i would like to ask you if you can provide me some documentation or immplementation guide that shows this syntax and its meaning I would really appreciate it. I'm working on a governemental project that involves air industry. my email address is mo.mabchour@gmail.com Thank you man, reply to this message in my email not this forum. |
| Free embeddable forum powered by Nabble | Forum Help |