« Return to Thread: Stack overflow using PdfWriter
Re: Stack overflow using PdfWriter
by Gylfi Ingvason
::
Rate this Message:
This is a vast improvement.
I took the latest SVN code and compiled under VS 2008 and then ran a file
concatenation job using PdfWriter feeding it just under 12,000 unique files.
The job completed in about 20 min. running in Debug mode on my vanilla dual
core 32-bit XP box using default stack (1 MB) with all operations running in
the same thread.
As a sanity check, I also ran the job using PdfSmartCopy and that also
completed successfully in about the same time, but I was running that in a
thread with 8 MB stack.
I then doubled the file count (to 23,000+ files, roughly 75,000 pages) using
a copy/paste on the original file set and ran the job again with PdfWriter
and default stack. The job completed successfully in about 45 min.
For comparison, using the released version, I pulled the plug after over six
hours with the original job running in a separate thread with 80 MB of
stack. The job overflowed the stack in a few hours with 16 MB stack.
This testing is by no means exhaustive, but it looks to me that operations
on large data are now fairly linear as opposed to exponential in the
previous version when the tree was thrashing the stack.
>From my perspective, the tree/stack problem has been fixed, and I'm looking
forward to getting the official release.
Many thanks to Paulo for taking the time to investigate and fix this. I've
suspected for a long time that something was wrong with the Tree
implementation, but it is darn near impossible to create a test case to
prove it. Thus, all the more kudos to Paulo for not dismissing this as
something that we should expect when dealing with large data.
Best regards - Gylfi
-----Original Message-----
From: Paulo Soares [mailto:psoares@...]
Sent: Wednesday, July 01, 2009 12:30 PM
To: Gylfi.Ingvason@...; Post all your questions about
iTextSharp here
Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
I've replaced the tree and if you could test it with your big files I would
be must obliged.
Paulo
> -----Original Message-----
> From: Gylfi Ingvason [mailto:Gylfi.Ingvason@...]
> Sent: Tuesday, June 23, 2009 4:54 PM
> To: 'Post all your questions about iTextSharp here'
> Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
>
> Thanks Paulo - that's exactly what this smells like.
>
> -----Original Message-----
> From: Paulo Soares [mailto:psoares@...]
> Sent: Tuesday, June 23, 2009 11:44 AM
> To: Post all your questions about iTextSharp here
> Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
>
> It's probably a bug caused by a lousy tree implementation.
> There's another
> problem in all this in that the tree is only needed when altering docs
> in append mode but it's used in all the cases even when 99% of the
> time a simple ArrayList would do. I'll try to find a better tree for
> .NET 1.1.
>
> Paulo
>
> > -----Original Message-----
> > From: Gylfi Ingvason [mailto:Gylfi.Ingvason@...]
> > Sent: Tuesday, June 23, 2009 4:25 PM
> > To: 'Post all your questions about iTextSharp here'
> > Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
> >
> > Thank you guys.
> >
> > The job runs to completion with 8 MB of stack using PdfSmarCopy. It
> > still dies with 16 MB using PdfWriter. I've now increased
> the stack to
> > 80 MB and the job is still cooking.
> >
> > I was aware of the need to bump the stack for large operations, but
> > when you go beyond 20 MB of stack space when processing 200 MB of
> > data, then, in my opinion, something is not right. Keeping a tree
> > balanced should not require that much stack. For my job,
> we're talking
> > roughly 35,000 pages, which is quite a bit, but not astronomical by
> > any standards.
> > I may experiment with increasing the branch size from 10 to
> a 100 and
> > see if that makes a difference, but to me, this sounds like
> a bug, and
> > if fixed, it will likely also increase processing speed
> dramatically
> > for large files.
> >
> > Gylfi
> >
> > ________________________________
> >
> > From: Adrian Rotaru [mailto:adrian.rotaru@...]
> > Sent: Tuesday, June 23, 2009 3:47 AM
> > To: Post all your questions about iTextSharp here
> > Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
> >
> >
> > I also had to increase the stack size to make it work..
> >
> > Adrian
> >
> >
> >
> >
> > ----- Original Message -----
> > From: Fabrizio Accatino <mailto:fhtino@...>
> > To: Gylfi.Ingvason@... ; Post all your questions
> about
> > iTextSharp here <mailto:itextsharp-questions@...>
> > Sent: Tuesday, June 23, 2009 10:38 AM
> > Subject: Re: [itextsharp-questions] Stack overflow
> using PdfWriter
> >
> >
> > I had the same problem. My environment was IIS6 but
> it's the same
> > problem.
> > The only solution I found was increasing the stack size of the
> > process/thread.
> >
> >
> >
> > http://sourceforge.net/mailarchive/forum.php?thread_name=2433d
> 6a90804080219u3c60cf17k8ab46345950764f6>
> @mail.gmail.com&forum_name=itex
> 6a90804080219u3c60cf17k8ab46345950764f6> tsharp-questions
> >
> >
> > http://fhtino.blogspot.com/2008/04/iis6-stackoverflow.html
> >
> >
> > fabrizio
> >
> >
> >
> > On Mon, Jun 22, 2009 at 9:55 PM, Gylfi Ingvason
> > <Gylfi.Ingvason@...> wrote:
> >
> >
> > I'm getting a stack overflow error when concatenating a
> bunch of
> > files using
> > PdfWriter. This occurs in Tree.cs
> > (/trunk/src/coreSystem/util/Collections)
> > where the private Insert member appears to get
> into a state
> where it
> > calls
> > itself recursively until an exception occurs.
> >
> >
> > ........
>
>
> Aviso Legal:
>
> Esta mensagem i destinada exclusivamente ao destinatario. Pode conter
> informagco confidencial ou legalmente protegida. A incorrecta
> transmissco desta mensagem nco significa a perca de confidencialidade.
> Se esta mensagem for recebida por engano, por favor envie-a de volta
> para o remetente e apague-a do seu sistema de imediato. I proibido a
> qualquer pessoa que nco o destinatario de usar, revelar ou distribuir
> qualquer parte desta mensagem.
>
>
>
> Disclaimer:
>
> This message is destined exclusively to the intended receiver. It may
> contain confidential or legally protected information. The incorrect
> transmission of this message does not mean the loss of its
> confidentiality.
> If this message is received by mistake, please send it back to the
> sender and delete it from your system immediately. It is forbidden to
> any person who is not the intended receiver to use, distribute or copy
> any part of this message.
>
>
>
>
>
>
> --------------------------------------------------------------
> ----------------
> Are you an open source citizen? Join us for the Open Source Bridge
> conference!
> Portland, OR, June 17-19. Two days of sessions, one day of
> unconference: $250.
> Need another reason to go? 24-hour hacker lounge. Register today!
> http://ad.doubleclick.net/clk;215844324;13503038;v?http://open
> sourcebridge.org
> _______________________________________________
> itextsharp-questions mailing list
> itextsharp-questions@...
> https://lists.sourceforge.net/lists/listinfo/itextsharp-questions
>
Aviso Legal:
Esta mensagem i destinada exclusivamente ao destinatario. Pode conter
informagco confidencial ou legalmente protegida. A incorrecta transmissco
desta mensagem nco significa a perca de confidencialidade. Se esta mensagem
for recebida por engano, por favor envie-a de volta para o remetente e
apague-a do seu sistema de imediato. I proibido a qualquer pessoa que nco o
destinatario de usar, revelar ou distribuir qualquer parte desta mensagem.
Disclaimer:
This message is destined exclusively to the intended receiver. It may
contain confidential or legally protected information. The incorrect
transmission of this message does not mean the loss of its confidentiality.
If this message is received by mistake, please send it back to the sender
and delete it from your system immediately. It is forbidden to any person
who is not the intended receiver to use, distribute or copy any part of this
message.
------------------------------------------------------------------------------
_______________________________________________
itextsharp-questions mailing list
itextsharp-questions@...
https://lists.sourceforge.net/lists/listinfo/itextsharp-questions
I took the latest SVN code and compiled under VS 2008 and then ran a file
concatenation job using PdfWriter feeding it just under 12,000 unique files.
The job completed in about 20 min. running in Debug mode on my vanilla dual
core 32-bit XP box using default stack (1 MB) with all operations running in
the same thread.
As a sanity check, I also ran the job using PdfSmartCopy and that also
completed successfully in about the same time, but I was running that in a
thread with 8 MB stack.
I then doubled the file count (to 23,000+ files, roughly 75,000 pages) using
a copy/paste on the original file set and ran the job again with PdfWriter
and default stack. The job completed successfully in about 45 min.
For comparison, using the released version, I pulled the plug after over six
hours with the original job running in a separate thread with 80 MB of
stack. The job overflowed the stack in a few hours with 16 MB stack.
This testing is by no means exhaustive, but it looks to me that operations
on large data are now fairly linear as opposed to exponential in the
previous version when the tree was thrashing the stack.
>From my perspective, the tree/stack problem has been fixed, and I'm looking
forward to getting the official release.
Many thanks to Paulo for taking the time to investigate and fix this. I've
suspected for a long time that something was wrong with the Tree
implementation, but it is darn near impossible to create a test case to
prove it. Thus, all the more kudos to Paulo for not dismissing this as
something that we should expect when dealing with large data.
Best regards - Gylfi
-----Original Message-----
From: Paulo Soares [mailto:psoares@...]
Sent: Wednesday, July 01, 2009 12:30 PM
To: Gylfi.Ingvason@...; Post all your questions about
iTextSharp here
Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
I've replaced the tree and if you could test it with your big files I would
be must obliged.
Paulo
> -----Original Message-----
> From: Gylfi Ingvason [mailto:Gylfi.Ingvason@...]
> Sent: Tuesday, June 23, 2009 4:54 PM
> To: 'Post all your questions about iTextSharp here'
> Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
>
> Thanks Paulo - that's exactly what this smells like.
>
> -----Original Message-----
> From: Paulo Soares [mailto:psoares@...]
> Sent: Tuesday, June 23, 2009 11:44 AM
> To: Post all your questions about iTextSharp here
> Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
>
> It's probably a bug caused by a lousy tree implementation.
> There's another
> problem in all this in that the tree is only needed when altering docs
> in append mode but it's used in all the cases even when 99% of the
> time a simple ArrayList would do. I'll try to find a better tree for
> .NET 1.1.
>
> Paulo
>
> > -----Original Message-----
> > From: Gylfi Ingvason [mailto:Gylfi.Ingvason@...]
> > Sent: Tuesday, June 23, 2009 4:25 PM
> > To: 'Post all your questions about iTextSharp here'
> > Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
> >
> > Thank you guys.
> >
> > The job runs to completion with 8 MB of stack using PdfSmarCopy. It
> > still dies with 16 MB using PdfWriter. I've now increased
> the stack to
> > 80 MB and the job is still cooking.
> >
> > I was aware of the need to bump the stack for large operations, but
> > when you go beyond 20 MB of stack space when processing 200 MB of
> > data, then, in my opinion, something is not right. Keeping a tree
> > balanced should not require that much stack. For my job,
> we're talking
> > roughly 35,000 pages, which is quite a bit, but not astronomical by
> > any standards.
> > I may experiment with increasing the branch size from 10 to
> a 100 and
> > see if that makes a difference, but to me, this sounds like
> a bug, and
> > if fixed, it will likely also increase processing speed
> dramatically
> > for large files.
> >
> > Gylfi
> >
> > ________________________________
> >
> > From: Adrian Rotaru [mailto:adrian.rotaru@...]
> > Sent: Tuesday, June 23, 2009 3:47 AM
> > To: Post all your questions about iTextSharp here
> > Subject: Re: [itextsharp-questions] Stack overflow using PdfWriter
> >
> >
> > I also had to increase the stack size to make it work..
> >
> > Adrian
> >
> >
> >
> >
> > ----- Original Message -----
> > From: Fabrizio Accatino <mailto:fhtino@...>
> > To: Gylfi.Ingvason@... ; Post all your questions
> about
> > iTextSharp here <mailto:itextsharp-questions@...>
> > Sent: Tuesday, June 23, 2009 10:38 AM
> > Subject: Re: [itextsharp-questions] Stack overflow
> using PdfWriter
> >
> >
> > I had the same problem. My environment was IIS6 but
> it's the same
> > problem.
> > The only solution I found was increasing the stack size of the
> > process/thread.
> >
> >
> >
> > http://sourceforge.net/mailarchive/forum.php?thread_name=2433d
> 6a90804080219u3c60cf17k8ab46345950764f6>
> @mail.gmail.com&forum_name=itex
> 6a90804080219u3c60cf17k8ab46345950764f6> tsharp-questions
> >
> >
> > http://fhtino.blogspot.com/2008/04/iis6-stackoverflow.html
> >
> >
> > fabrizio
> >
> >
> >
> > On Mon, Jun 22, 2009 at 9:55 PM, Gylfi Ingvason
> > <Gylfi.Ingvason@...> wrote:
> >
> >
> > I'm getting a stack overflow error when concatenating a
> bunch of
> > files using
> > PdfWriter. This occurs in Tree.cs
> > (/trunk/src/coreSystem/util/Collections)
> > where the private Insert member appears to get
> into a state
> where it
> > calls
> > itself recursively until an exception occurs.
> >
> >
> > ........
>
>
> Aviso Legal:
>
> Esta mensagem i destinada exclusivamente ao destinatario. Pode conter
> informagco confidencial ou legalmente protegida. A incorrecta
> transmissco desta mensagem nco significa a perca de confidencialidade.
> Se esta mensagem for recebida por engano, por favor envie-a de volta
> para o remetente e apague-a do seu sistema de imediato. I proibido a
> qualquer pessoa que nco o destinatario de usar, revelar ou distribuir
> qualquer parte desta mensagem.
>
>
>
> Disclaimer:
>
> This message is destined exclusively to the intended receiver. It may
> contain confidential or legally protected information. The incorrect
> transmission of this message does not mean the loss of its
> confidentiality.
> If this message is received by mistake, please send it back to the
> sender and delete it from your system immediately. It is forbidden to
> any person who is not the intended receiver to use, distribute or copy
> any part of this message.
>
>
>
>
>
>
> --------------------------------------------------------------
> ----------------
> Are you an open source citizen? Join us for the Open Source Bridge
> conference!
> Portland, OR, June 17-19. Two days of sessions, one day of
> unconference: $250.
> Need another reason to go? 24-hour hacker lounge. Register today!
> http://ad.doubleclick.net/clk;215844324;13503038;v?http://open
> sourcebridge.org
> _______________________________________________
> itextsharp-questions mailing list
> itextsharp-questions@...
> https://lists.sourceforge.net/lists/listinfo/itextsharp-questions
>
Aviso Legal:
Esta mensagem i destinada exclusivamente ao destinatario. Pode conter
informagco confidencial ou legalmente protegida. A incorrecta transmissco
desta mensagem nco significa a perca de confidencialidade. Se esta mensagem
for recebida por engano, por favor envie-a de volta para o remetente e
apague-a do seu sistema de imediato. I proibido a qualquer pessoa que nco o
destinatario de usar, revelar ou distribuir qualquer parte desta mensagem.
Disclaimer:
This message is destined exclusively to the intended receiver. It may
contain confidential or legally protected information. The incorrect
transmission of this message does not mean the loss of its confidentiality.
If this message is received by mistake, please send it back to the sender
and delete it from your system immediately. It is forbidden to any person
who is not the intended receiver to use, distribute or copy any part of this
message.
------------------------------------------------------------------------------
_______________________________________________
itextsharp-questions mailing list
itextsharp-questions@...
https://lists.sourceforge.net/lists/listinfo/itextsharp-questions
« Return to Thread: Stack overflow using PdfWriter
| Free embeddable forum powered by Nabble | Forum Help |
