|
»
xdb:store-files-from-pattern skips one directory when recursive pattern is used
|
View:
New views
19 Messages
—
Rating Filter:
Alert me
|
 |
xdb:store-files-from-pattern skips one directory when recursive pattern is used
by thomas.0007
::
Rate this Message:
dev1.3, rev.9095
When a recursive patter is used in xdb:store-files-from-pattern it misses the first directory. Test case: In the directory where the test.xql file is located ( the code is at the bottom of this email) create a file test-data.xml and copy it to the following directory structure: copy-files.xql data/test-data.xml data/d1/test-data.xml data/d2/test-data.xml We want to import these files, keeping the directory structure in the database under a collection /db/test : /db/test/test-data.xml /db/test/d1/test-data.xml /db/test/d2/test-data.xml The result is: /db/test/test-data.xml /db/d1/test-data.xml /db/d2/test-data.xml copy-files.xql: xquery version "1.0" encoding "UTF-8"; declare default function namespace "http://www.w3.org/2005/xpath-functions"; import module namespace xdb="http://exist-db.org/xquery/xmldb"; declare boundary-space strip; declare option exist:serialize "method=xml media-type=text/xml"; let $home := system:get-exist-home() let $pathSep := util:system-property("file.separator") let $current-dir := replace(system:get-module-load-path(),'\\', '/') let $dir-to-copy := 'data' let $coll := '/db/test' let $ping-coll := if( xdb:collection-exists ($coll)) then () else xdb:create-collection( '/db', 'test') let $dir := if( contains($current-dir,':/' ) ) (: c:/eXist - windows os :) then( if( $dir-to-copy = '.' ) then $current-dir else concat($current-dir, '/', $dir-to-copy ) )else if(ends-with($home, "WEB-INF")) then concat(substring-before($home, "WEB-INF"), $coll, $pathSep, $dir-to-copy ) else concat($home, $pathSep, "webapp", $pathSep, $coll, $pathSep, $dir-to-copy ) return element root { element path { $dir }, xdb:store-files-from-pattern( $coll, $dir, '**/*.xml', 'text/xml', true() ) } Regards, Thomas ------------------------------------------------------------------------------ OpenSolaris 2009.06 is a cutting edge operating system for enterprises looking to deploy the next generation of Solaris that includes the latest innovations from Sun and the OpenSource community. Download a copy and enjoy capabilities such as Networking, Storage and Virtualization. Go to: http://p.sf.net/sfu/opensolaris-get _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: xdb:store-files-from-pattern skips one directory whenrecursive pattern is used
by thomas.0007
::
Rate this Message:
I would really appreciate is somebody can have a look at this issue. I have
a ton of installation XQuery files that import a complex directory structure into the database and all of them are broken now. Regards, Thomas ------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
schema documents or proxy problem.
by Remi Malessa-2
::
Rate this Message:
Hi.
I'm using Apache Tomcat 5.5.26 and eXist 1.2.6 build as a block on Cocoon 2.1.11. When uploading my XML documents I get a message: Impossible to store a resource /path_to_resource.xml: Failed to invoke method parse in class org.exist.xmlrpc.RpcConnection: org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema document 'http://www.loc.gov/standards/mets/mets.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>. I have the proxy parameters specified in the /etc/init.d/tomcat file: :export JAVA_OPTS="...-Dhttp.proxyhost=my.host.address.uk -Dhttp.proxyport=80 ..." I've also filled up the build.properties parameters: proxy.nonproxyhosts = proxy.host = cache.myhost.uk proxy.port = 80 proxy.password = proxy.user = proxy.socks.host = cache.myhost.uk proxy.socks.port = 80 Am I missing something ? Any suggestions ? Is there a place like conf.xml where you can change eXist proxy parameters ? Cheers Remi ------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: schema documents or proxy problem.
by Dannes Wessels-2
::
Rate this Message:
Hi,
On 19 Jun 2009, at 18:18 , Remi Malessa wrote: > I have the proxy parameters specified in the /etc/init.d/tomcat file: > > :export JAVA_OPTS="...-Dhttp.proxyhost=my.host.address.uk > -Dhttp.proxyport=80 ..." > > I've also filled up the build.properties parameters: > > Am I missing something ? > Any suggestions ? > > Is there a place like conf.xml where you can change eXist proxy > parameters ? than what your JVM provides (you tried it... but aren't these parameters case sensitive? http://java.sun.com/j2se/1.5.0/docs/guide/net/properties.html ), but still then I am not sure if the exist classes obey the JVM settings..... Kind regards Dannes -- eXist-db Native XML Database - http://exist-db.org Join us on linked-in: http://www.linkedin.com/groups?gid=35624 ------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: schema documents or proxy problem.
by Dannes Wessels-2
::
Rate this Message:
On 19 Jun 2009, at 18:18 , Remi Malessa wrote: > Any suggestions ? the build.properties are only used during...... the build process. alternatively you could download the xsd's and register them in the catalog file..... Kind regards Dannes -- eXist-db Native XML Database - http://exist-db.org Join us on linked-in: http://www.linkedin.com/groups?gid=35624 ------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: schema documents or proxy problem.
by Adam Retter-3
::
Rate this Message:
Two options -
1) Make sure the proxy stuff is working - I would probably use Wireshark to check this. 2) Download the Schemas locally and set them up in the catalog - see the Validation documentation on the eXist website. This is probably the better option as then you are not reliant on an external source. Nice to see eXist being used in Wales :-) Cheers Adam. 2009/6/19 Remi Malessa <rem@...>: > Hi. > > I'm using Apache Tomcat 5.5.26 and eXist 1.2.6 build as a block on > Cocoon 2.1.11. > When uploading my XML documents I get a message: > > Impossible to store a resource /path_to_resource.xml: Failed to invoke > method parse in class org.exist.xmlrpc.RpcConnection: > org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema > document 'http://www.loc.gov/standards/mets/mets.xsd', because 1) could > not find the document; 2) the document could not be read; 3) the root > element of the document is not <xsd:schema>. > > I have the proxy parameters specified in the /etc/init.d/tomcat file: > > :export JAVA_OPTS="...-Dhttp.proxyhost=my.host.address.uk > -Dhttp.proxyport=80 ..." > > I've also filled up the build.properties parameters: > > proxy.nonproxyhosts = > proxy.host = cache.myhost.uk > proxy.port = 80 > proxy.password = > proxy.user = > proxy.socks.host = cache.myhost.uk > proxy.socks.port = 80 > > Am I missing something ? > Any suggestions ? > > Is there a place like conf.xml where you can change eXist proxy parameters ? > > Cheers > Remi > > ------------------------------------------------------------------------------ > Crystal Reports - New Free Runtime and 30 Day Trial > Check out the new simplified licensing option that enables unlimited > royalty-free distribution of the report engine for externally facing > server and web deployment. > http://p.sf.net/sfu/businessobjects > _______________________________________________ > Exist-open mailing list > Exist-open@... > https://lists.sourceforge.net/lists/listinfo/exist-open > -- Adam Retter eXist Developer { United Kingdom } adam@... irc://irc.freenode.net/existdb ------------------------------------------------------------------------------ Are you an open source citizen? Join us for the Open Source Bridge conference! Portland, OR, June 17-19. Two days of sessions, one day of unconference: $250. Need another reason to go? 24-hour hacker lounge. Register today! http://ad.doubleclick.net/clk;215844324;13503038;v?http://opensourcebridge.org _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: xdb:store-files-from-pattern skips one directory when recursive pattern is used
by Adam Retter-3
::
Rate this Message:
Thomas,
I have spent some time looking at this and stepping through the code. I can reproduce your issue, which is good news of course.
The code is not straight forward and/or I am too tired right now to make sense of it - I will try and spend some time fixing this over the weekend if I get a moment.
Can you tell me if this is a recent regression?
Thanks Adam.
2009/6/3 Thomas <thomas.0007@...> dev1.3, rev.9095 -- Adam Retter eXist Developer { United Kingdom } adam@... irc://irc.freenode.net/existdb ------------------------------------------------------------------------------ Are you an open source citizen? Join us for the Open Source Bridge conference! Portland, OR, June 17-19. Two days of sessions, one day of unconference: $250. Need another reason to go? 24-hour hacker lounge. Register today! http://ad.doubleclick.net/clk;215844324;13503038;v?http://opensourcebridge.org _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: xdb:store-files-from-pattern skips one directory when recursive pattern is used
by thomas.0007
::
Rate this Message:
I think it is at least a month old.
2009/6/19 Adam Retter <adam@...>
------------------------------------------------------------------------------ Are you an open source citizen? Join us for the Open Source Bridge conference! Portland, OR, June 17-19. Two days of sessions, one day of unconference: $250. Need another reason to go? 24-hour hacker lounge. Register today! http://ad.doubleclick.net/clk;215844324;13503038;v?http://opensourcebridge.org _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: schema documents or proxy problem.
by Remi Malessa-2
::
Rate this Message:
Hi Dannes.
On 19/06/09 17:39, Dannes Wessels wrote: > the build.properties are only used during...... the build process. I edited the build.properties before building the eXist as Cocoon block WAR archive. > alternatively you could download the xsd's and register them in the > catalog file..... I tried that. It worked with the TEI, but for some reason I could not make it work with METS. Cheers Remi ------------------------------------------------------------------------------ Are you an open source citizen? Join us for the Open Source Bridge conference! Portland, OR, June 17-19. Two days of sessions, one day of unconference: $250. Need another reason to go? 24-hour hacker lounge. Register today! http://ad.doubleclick.net/clk;215844324;13503038;v?http://opensourcebridge.org _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: schema documents or proxy problem.
by Remi Malessa-2
::
Rate this Message:
Hi Dannes
On 19/06/09 17:37, Dannes Wessels wrote: > (you tried it... but aren't these parameters case sensitive? > http://java.sun.com/j2se/1.5.0/docs/guide/net/properties.html ), but > still then I am not sure if the exist classes obey the JVM settings..... That was it. Thanks for the hint. Remi ------------------------------------------------------------------------------ Are you an open source citizen? Join us for the Open Source Bridge conference! Portland, OR, June 17-19. Two days of sessions, one day of unconference: $250. Need another reason to go? 24-hour hacker lounge. Register today! http://ad.doubleclick.net/clk;215844324;13503038;v?http://opensourcebridge.org _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: schema documents or proxy problem.
by Remi Malessa-2
::
Rate this Message:
On 19/06/09 18:47, Adam Retter wrote: > Nice to see eXist being used in Wales :-) > Hi Adam. Thanks for the hints. I've got two projects (so far) in the schedule, that will be using eXist and Cocoon. We'll see how it goes. Cheers Remi ------------------------------------------------------------------------------ Are you an open source citizen? Join us for the Open Source Bridge conference! Portland, OR, June 17-19. Two days of sessions, one day of unconference: $250. Need another reason to go? 24-hour hacker lounge. Register today! http://ad.doubleclick.net/clk;215844324;13503038;v?http://opensourcebridge.org _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: schema documents or proxy problem.
by Remi Malessa-2
::
Rate this Message:
On 22/06/09 12:59, Remi Malessa wrote:
>> alternatively you could download the xsd's and register them in the >> catalog file..... >> > I tried that. It worked with the TEI, but for some reason I could not > make it work with METS. > Ok. I looked at the 'Validation' documentation and I see why it didn't work. You have to configure the validation :-) Thanks Adam. ------------------------------------------------------------------------------ Are you an open source citizen? Join us for the Open Source Bridge conference! Portland, OR, June 17-19. Two days of sessions, one day of unconference: $250. Need another reason to go? 24-hour hacker lounge. Register today! http://ad.doubleclick.net/clk;215844324;13503038;v?http://opensourcebridge.org _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
|
|
|
Re: xdb:store-files-from-pattern skips one directory when recursive pattern is used
by Adam Retter-3
::
Rate this Message:
Its is hopefully fixed now in trunk as SVN revision 9189.
Let me know... Cheers Adam. 2009/6/24 Adam Retter <adam@...>: > I am very short of time at the moment, but its still on my radar. > > Hopefully before the weekend... > > > > 2009/6/24 Thomas White <thomas.0007@...>: >> Adam, >> >> Did you have a chance to go any further with this issue? >> >> Regards, >> Thomas >> >> 2009/6/19 Adam Retter <adam@...> >>> >>> >>> Thomas, >>> >>> I have spent some time looking at this and stepping through the code. I >>> can reproduce your issue, which is good news of course. >>> The code is not straight forward and/or I am too tired right now to make >>> sense of it - I will try and spend some time fixing this over the weekend if >>> I get a moment. >>> >>> Can you tell me if this is a recent regression? >>> >>> >>> Thanks Adam. >>> >>> >>> 2009/6/3 Thomas <thomas.0007@...> >>>> >>>> dev1.3, rev.9095 >>>> >>>> When a recursive patter is used in xdb:store-files-from-pattern it misses >>>> the first directory. >>>> Test case: In the directory where the test.xql file is located ( the code >>>> is >>>> at the bottom of this email) create a file test-data.xml and copy it to >>>> the >>>> following directory structure: >>>> copy-files.xql >>>> data/test-data.xml >>>> data/d1/test-data.xml >>>> data/d2/test-data.xml >>>> >>>> We want to import these files, keeping the directory structure in the >>>> database under a collection /db/test : >>>> /db/test/test-data.xml >>>> /db/test/d1/test-data.xml >>>> /db/test/d2/test-data.xml >>>> >>>> The result is: >>>> /db/test/test-data.xml >>>> /db/d1/test-data.xml >>>> /db/d2/test-data.xml >>>> >>>> >>>> copy-files.xql: >>>> xquery version "1.0" encoding "UTF-8"; >>>> declare default function namespace >>>> "http://www.w3.org/2005/xpath-functions"; >>>> import module namespace xdb="http://exist-db.org/xquery/xmldb"; >>>> declare boundary-space strip; >>>> declare option exist:serialize "method=xml media-type=text/xml"; >>>> let $home := system:get-exist-home() >>>> let $pathSep := util:system-property("file.separator") >>>> let $current-dir := replace(system:get-module-load-path(),'\\', '/') >>>> >>>> let $dir-to-copy := 'data' >>>> let $coll := '/db/test' >>>> let $ping-coll := if( xdb:collection-exists ($coll)) then () >>>> else xdb:create-collection( '/db', 'test') >>>> >>>> let $dir := if( contains($current-dir,':/' ) ) (: c:/eXist - windows os >>>> :) >>>> then( >>>> if( $dir-to-copy = '.' ) then >>>> $current-dir >>>> else >>>> concat($current-dir, '/', $dir-to-copy ) >>>> )else if(ends-with($home, "WEB-INF")) then >>>> concat(substring-before($home, "WEB-INF"), $coll, $pathSep, $dir-to-copy >>>> ) >>>> else >>>> concat($home, $pathSep, "webapp", $pathSep, $coll, $pathSep, $dir-to-copy >>>> ) >>>> >>>> return >>>> element root { >>>> element path { $dir }, >>>> xdb:store-files-from-pattern( $coll, $dir, '**/*.xml', 'text/xml', >>>> true() ) >>>> } >>>> >>>> Regards, >>>> Thomas >>>> >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> OpenSolaris 2009.06 is a cutting edge operating system for enterprises >>>> looking to deploy the next generation of Solaris that includes the latest >>>> innovations from Sun and the OpenSource community. Download a copy and >>>> enjoy capabilities such as Networking, Storage and Virtualization. >>>> Go to: http://p.sf.net/sfu/opensolaris-get >>>> _______________________________________________ >>>> Exist-open mailing list >>>> Exist-open@... >>>> https://lists.sourceforge.net/lists/listinfo/exist-open >>> >>> >>> >>> -- >>> Adam Retter >>> >>> eXist Developer >>> { United Kingdom } >>> adam@... >>> irc://irc.freenode.net/existdb >> >> >> >> -- >> Thomas White >> >> Mobile:+44 7711 922 966 >> Skype: thomaswhite >> gTalk: thomas.0007 >> Linked-In:http://www.linkedin.com/in/thomaswhite0007 >> facebook: http://www.facebook.com/thomas.0007 >> > > > > -- > Adam Retter > > eXist Developer > { United Kingdom } > adam@... > irc://irc.freenode.net/existdb > -- Adam Retter eXist Developer { United Kingdom } adam@... irc://irc.freenode.net/existdb ------------------------------------------------------------------------------ _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
'text/javascript' files parsed as XML by xdb:store-files-from-pattern (was xdb:store-files-from-pattern skips one directorywhen recursive pattern is used)
by thomas.0007
::
Rate this Message:
Adam,
Today I found an interesting case using xdb:store-files-from-pattern. I tried to import JavaScript files but for some reason 'text/javascript' MIME type is not recognized as binary file and the parser gives an error: 2009-06-29 21:14:48,390 [P1-5] DEBUG (XMLDBLoadFromPattern.java [evalWithCollection]:148) - D:\eXist\webapp\o8\js\ie7\ie7-recalc.js 2009-06-29 21:14:48,390 [P1-5] DEBUG (Collection.java [validateXMLResourceInternal]:1197) - Scanning document /db/o8/js/ie7/ie7-recalc.js 2009-06-29 21:14:48,390 [P1-5] DEBUG (Indexer.java [fatalError]:405) - fatal error at (1,1) : Content is not allowed in prolog. 2009-06-29 21:14:48,390 [P1-5] WARN (XMLDBLoadFromPattern.java [evalWithCollection]:182) - Could not store file D:\eXist\webapp\o8\js\ie7\ie7-recalc.js: The XML parser reported a problem: fatal error at (1,1) : Content is not allowed in prolog. How do we set which MIME types are treated as binary and they are not checked by the XML parser? Regards, Thomas "Adam Retter" <adam@...> wrote in message news:3573c61b0906241312h4a516230jc01d46c3b0d438f0@...... > Its is hopefully fixed now in trunk as SVN revision 9189. > > Let me know... > > Cheers Adam. ------------------------------------------------------------------------------ _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: 'text/javascript' files parsed as XML by xdb:store-files-from-pattern (was xdb:store-files-from-pattern skips one directorywhen recursive pattern is used)
by Adam Retter-3
::
Rate this Message:
> Today I found an interesting case using xdb:store-files-from-pattern. I
> tried to import JavaScript files but for some reason 'text/javascript' MIME > type is not recognized as binary file and the parser gives an error: > > 2009-06-29 21:14:48,390 [P1-5] DEBUG (XMLDBLoadFromPattern.java > [evalWithCollection]:148) - D:\eXist\webapp\o8\js\ie7\ie7-recalc.js > 2009-06-29 21:14:48,390 [P1-5] DEBUG (Collection.java > [validateXMLResourceInternal]:1197) - Scanning document > /db/o8/js/ie7/ie7-recalc.js > 2009-06-29 21:14:48,390 [P1-5] DEBUG (Indexer.java [fatalError]:405) - fatal > error at (1,1) : Content is not allowed in prolog. > 2009-06-29 21:14:48,390 [P1-5] WARN (XMLDBLoadFromPattern.java > [evalWithCollection]:182) - Could not store file > D:\eXist\webapp\o8\js\ie7\ie7-recalc.js: The XML parser reported a problem: > fatal error at (1,1) : Content is not allowed in prolog. > > How do we set which MIME types are treated as binary and they are not > checked by the XML parser? You would set the mime type in the mime-types.xml file in EXIST_HOME - changes to this do require a restart. However looking in mime-types.xml I can see we already have a definition for application/x-javascript - so perhaps that is the mime-type you need? Thanks Adam. > Regards, > Thomas > > > "Adam Retter" <adam@...> wrote in message > news:3573c61b0906241312h4a516230jc01d46c3b0d438f0@...... >> Its is hopefully fixed now in trunk as SVN revision 9189. >> >> Let me know... >> >> Cheers Adam. > > > > > ------------------------------------------------------------------------------ > _______________________________________________ > Exist-open mailing list > Exist-open@... > https://lists.sourceforge.net/lists/listinfo/exist-open > -- Adam Retter eXist Developer { United Kingdom } adam@... irc://irc.freenode.net/existdb ------------------------------------------------------------------------------ _______________________________________________ Exist-open mailing list Exist-open@... https://lists.sourceforge.net/lists/listinfo/exist-open |
|
|
Re: 'text/javascript' files parsed as XML byxdb:store-files-from-pattern (wasxdb:store-files-from-patte |
|
|
Re: 'text/javascript' files parsed as XML byxdb:store-files-from-pattern (wasxdb:store-files-from-patte |
|
|
|
smime.p7s (2K) 
| Free embeddable forum powered by Nabble | Forum Help |
