2 000 000 documents in one xml file

View: New views
3 Messages — Rating Filter:   Alert me  

2 000 000 documents in one xml file

by Scorp :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I new to exist db. But i need to test performance on 2 000 000 generated objects.
I generated these objects and put it in one xml file.

every object looks like that

<objectn>
   <prop1>value</prop1>
   ...
   <prop10>value</prop10>
</objectn>

The query that gets all objects took 5-6 sec.
But the query holds on endlessly that receives objects by some properties or just to get the first object.
1GB memory granted to exist db on server.

Help please what i do wrong?

Re: 2 000 000 documents in one xml file

by Wolfgang Meier-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> The query that gets all objects took 5-6 sec.

A query for //objectn should complete within a few milliseconds.
Retrieving the 2 million results could be slow, sure, depending on how
you serialize them.

> But the query holds on endlessly that receives objects by some properties or
> just to get the first object.

I guess you did not define any indexes on prop1...prop10? See

http://exist-db.org/tuning.html

Wolfgang

------------------------------------------------------------------------------
_______________________________________________
Exist-open mailing list
Exist-open@...
https://lists.sourceforge.net/lists/listinfo/exist-open

Re: 2 000 000 documents in one xml file

by Scorp :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks for reply

Wolfgang Meier-2 wrote:
> The query that gets all objects took 5-6 sec.

A query for //objectn should complete within a few milliseconds.
Retrieving the 2 million results could be slow, sure, depending on how
you serialize them.
All objectsn in one big xml file that i put to collection. May be it is wrong?

Wolfgang Meier-2 wrote:
I guess you did not define any indexes on prop1...prop10? See

http://exist-db.org/tuning.html

Wolfgang
Yes for that moment, but later i have read tuning article, and add indexes :-)
And it is became 4 sec to get objects by 1 property and 10 sec to get objects by 5 properties.