Marshalling Large data into xml

View: New views
2 Messages — Rating Filter:   Alert me  

Marshalling Large data into xml

by deepak_scs :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I am using JAXB and I have a large set of data which i have to marshal into a xml.Since marshalling the whole thing into xml in a single step will be using most of the memory , i want to split it into parts and write to the xml file incremently

For example if my generated output xml should be like this:
<Employees>
<employee>......</employee>
<employee>.....</employee>
<employee>.....</employee>
<employee>.....</employee>
..
...
..
</Employees>

I would like to write the <employee> sections separately into a file instead of writing the whole thing together.I am retrieving the employee details from the database and converting to xml.There are almost 8 lakh records.So marshalling the whole thing in single step will use up my memory.How can i do it?????

Thanks

Re: Marshalling Large data into xml

by Cowtowncoder :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wed, Oct 21, 2009 at 7:03 AM, deepak_scs <deepak@...> wrote:

>
> I am using JAXB and I have a large set of data which i have to marshal into a
> xml.Since marshalling the whole thing into xml in a single step will be
> using most of the memory , i want to split it into parts and write to the
> xml file incremently
>
> For example if my generated output xml should be like this:
> <Employees>
> <employee>......</employee>
> <employee>.....</employee>
> <employee>.....</employee>
> <employee>.....</employee>
> ..
> ...
> ..
> </Employees>
>
> I would like to write the <employee> sections separately into a file instead
> of writing the whole thing together.I am retrieving the employee details
> from the database and converting to xml.There are almost 8 lakh records.So
> marshalling the whole thing in single step will use up my memory.How can i
> do it?????

Use Stax API (XMLStreamWriter) as the underlying XML processing thing;
write <Employees> tag using that, and then pass XMLStreamWriter to
JAXB Marshaller, marshall employee by employee.
This is the pattern I use; similarly works well with unmarshalling.
Not sure if this is in FAQ or not, but it probably should be.

Hope this helps,

-+ Tatu +-

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...