[Kde-graphics-devel] KPdf evolution

Hi all,
  this is the first mail to the ML about KPdf (the program I'm co-writing),
but this seems to me the best place to share ideas and take decisions to
steer development.

For who doesn't know, KPdf (aka xpdf on kde) has been refactored completely
starting from Q3-2004 and is now in a really good shape. But since the user
base is growing and "competitors" are coming, it seems more reasonable to
take decisions in the ML, allowing more people to express their thoughts and
letting the developers have many different perspectives on their work.

About pdf readers unveiled during KPdf-ng development:
- evince: gnome project, looks good:
  http://www.gnome.org/projects/evince/
- adobe reader 7.0: "the reference implementation"
  there is a beta floating around. to be released before mar-31.

As of now, KPdf is the best opensource pdf viewer available but I think that
it's unlikely to be included in distributions, replaced by acroread when it
will be released. Unless it has unique features :-)

Looking for unique features and this ML.... 2 things tied very closely!!

- How can be a new generation EBOOK reader (yes, 'cause kpdf can support many
formats, not only pdf, despite of its name) ??
- What can a program offer you to help you reading / browsing through your
documents ?? { Translation of text, Google-like indexing, Summarizing, Page
painting, Animations, OpenGL 3D book-like view, Brain content injectors,
Retinal Beamers } ? :-) Maybe something more?

I'm sure kpdf jurney will be lots of fun here!
Enrico Ros
--
 
 
 --
 Email.it, the professional e-mail, gratis per te: http://www.email.it/f
 
 Sponsor:
 Proteggi la tua moto dalle intemperie e... dagli sguardi indiscreti con il Telo Coprimoto!
 Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=2744&d=13-2

On Sunday 13 February 2005 12:56, Enrico Ros wrote:
> - How can be a new generation EBOOK reader (yes, 'cause kpdf can
> support many formats, not only pdf, despite of its name) ??
> - What can a program offer you to help you reading / browsing through
> your documents ?? { Translation of text, Google-like indexing,
> Summarizing, Page painting, Animations, OpenGL 3D book-like view,
> Brain content injectors, Retinal Beamers } ? :-) Maybe something
> more?

I love KPdf but there's one thing that bothers me and it's really
related to xpdf and it's the rendering speed of any pdf's with images.
When it's all text, everything is fine but when you hit an image the
drawPixel calls are killing it. An example of a simple presentation is
at: http://ktown.kde.org/~zrusin/pbuffers2.pdf. Notice how the cpu jumps
to 100% on both startup and scrolling. I usually compile/run simulation
on my machine in the background so having a document viewer taking so
many cycles.
So essentially, I think that before tackling the really cool features we
could focus on performance. I'm willing to work on that because it's
the last thing that bothers me a lot. The big question is how do we
want to fix it? Improving the xpdf image rendering can be done by
either porting the drawing code to Qt and then optimizing like any
other Qt app/lib (which would probably kill the freedesktop xpdf
initiative) or trying to simply improve the Xpdf drawing code.
Opinions?


Zack

--
With Windows Millenium MS was able to get the boot time down to 25
seconds.
That's almost as short as it's uptime.

[Hans Oischinger]

> - How can be a new generation EBOOK reader (yes, 'cause kpdf can support
> many formats, not only pdf, despite of its name) ??
> - What can a program offer you to help you reading / browsing through your
> documents ?? { Translation of text, Google-like indexing, Summarizing, Page
> painting, Animations, OpenGL 3D book-like view, Brain content injectors,
> Retinal Beamers } ? :-) Maybe something more?

First of all I want to thank you for the new Kpdf. It's absolutely great. I
used it alot during my exams (had to read lots of pdfs).
It's fast (fast enough for me at least) and supports all important basic
features.

What I miss is a good handling of bookmarks.
How about another Dockwindow where you can drag in pages that you want to have
on hand all the time (e.g. the language reference in a pdf book about C
Programming).
As I already mentioned I used kpdf alot and this feature was the one I missed
the most.

Whatever... continue the great work.

Greets,
 Hans
--
Hans Oischinger hans.oischinger@... | hans.oischinger@...

[Wilfried Huss]

Am Sunday, 13. February 2005 18:56 schrieb Enrico Ros:
> - How can be a new generation EBOOK reader (yes, 'cause kpdf can support
> many formats, not only pdf, despite of its name) ??

As you may know KViewShell has the same goal. While its development
stagnated for a few years, Stefan Kebekus and myself started working on it
again last summer.
The main goal for KDE 3.4 was to cleanly separate KViewShell from KDVI.

In KViewShell the fileformat plugins are KParts which are now loaded on demand
at runtime. Each plugin can add its own menu items and configuration options
to the interface.

The plugins are also designed in a way to allow them to be embeded into other
applications like Kile for example. This is still not complete, but we aim at
something similar to the KTextEditor or KMediaPlayer interfaces.

For an universal EBook-Reader it also has to be possible to implement
special features which the different fileformats support. KDVI for example
allows to jump from a DVI-document to the corresponding line in its
TeX-source, and vice verse.

I've created a list of what I think are the relative strengths of the two
programs. Since I haven't looked too close at the code of KPDF this list
might be not completly correct.

KPDF:
* Asyncronous page generation.
* Support for accessibility features.
* Better rendering code.
* Flickerfree and more smooth display.
* Presentation mode.
* Search function works better.

KViewShell:
* Better page layout with 4 different viewmodes. (Different papersizes
   per document are not supported yet, since dvi doesn't support them.)
* Loading of compressed files.
* Guessing of filename extensions. Very convinient when working with LaTeX
  on the commandline.
* More refined code to fit pages to the viewport. Works in all viewmodes and
  scrollbars are only enabled when necessary.
* More refined API for the fileformat plugins (as explained above).
* Text selection works like in a text editor.

I hope we can combine our efforts for KDE4 since two generic document
viewers are overkill for KDE, and in my opinion both KPDF and KDVI could
gain a lot in the process.

Greetings,
Wilfried.

Dnia niedziela, 13 lutego 2005 18:56, Enrico Ros napisa?:

>
> As of now, KPdf is the best opensource pdf viewer available but I think
> that it's unlikely to be included in distributions, replaced by acroread
> when it will be released. Unless it has unique features :-)
>
> Looking for unique features and this ML.... 2 things tied very closely!!
>
> - How can be a new generation EBOOK reader (yes, 'cause kpdf can support
> many formats, not only pdf, despite of its name) ??
> - What can a program offer you to help you reading / browsing through
> your documents ??
> { Translation of text,

Sorry, don't believe in good OSS solution in that matter, scrap it.

> Google-like indexing,
Should be done on another level. Don't be too fast.

> Summarizing,
Some sort of generated table of contents could be useful:
- list of graphics
- maybe list of links (internal, external)

> Page painting,
That could be useful in connection with annotations.

> Animations, OpenGL 3D book-like view, Brain
> content injectors, Retinal Beamers } ? :-) Maybe something more?

Before other things current features should be polished:
- better support for bookmarks
- better support for KTTSD (Read page, Read document)

And your TODO is really good roadmap :)

> I'm sure kpdf jurney will be lots of fun here!

Thanks for great app.

m.

[Gary L. Greene Jr.-2]

On Sunday 13 February 2005 8:32 pm, Mikolaj Machowski wrote:

> Dnia niedziela, 13 lutego 2005 18:56, Enrico Ros napisa?:
> > As of now, KPdf is the best opensource pdf viewer available but I think
> > that it's unlikely to be included in distributions, replaced by acroread
> > when it will be released. Unless it has unique features :-)
> >
> > Looking for unique features and this ML.... 2 things tied very closely!!
> >
> > - How can be a new generation EBOOK reader (yes, 'cause kpdf can support
> > many formats, not only pdf, despite of its name) ??
> > - What can a program offer you to help you reading / browsing through
> > your documents ??
> > { Translation of text,
>
> Sorry, don't believe in good OSS solution in that matter, scrap it.

Your opinion, not fact. No one should shoot an idea down based off current
levels of expectation.

> > Google-like indexing,
>
> Should be done on another level. Don't be too fast.
>
> > Summarizing,
>
> Some sort of generated table of contents could be useful:
> - list of graphics
> - maybe list of links (internal, external)
>
> > Page painting,
>
> That could be useful in connection with annotations.
>
> > Animations, OpenGL 3D book-like view, Brain
> > content injectors, Retinal Beamers } ? :-) Maybe something more?
>
> Before other things current features should be polished:
> - better support for bookmarks
> - better support for KTTSD (Read page, Read document)
>
> And your TODO is really good roadmap :)
>
> > I'm sure kpdf jurney will be lots of fun here!
>
> Thanks for great app.
>
> m.
>
> _______________________________________________
> Kde-graphics-devel mailing list
> Kde-graphics-devel@...
> https://mail.kde.org/mailman/listinfo/kde-graphics-devel

--
Gary L. Greene, Jr.
Sent from uriel
 13:41:43 up 21:52,  4 users,  load average: 0.37, 0.35, 0.28
 
============================================================
Developer and Project Lead for the Ark Linux Project
 check out http://www.arklinux.org/ for more info.
 Also http://www.csis.gvsu.edu/~greeneg/
EMAIL : greeneg@...
============================================================
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 190 bytes
Desc: not available
Url : http://mail.kde.org/pipermail/kde-graphics-devel/attachments/20050214/4b58edb0/attachment.pgp

Dnia poniedzia?ek, 14 lutego 2005 19:43, Gary L. Greene Jr. napisa?:
> > > { Translation of text,
> >
> > Sorry, don't believe in good OSS solution in that matter, scrap it.
>
> Your opinion, not fact. No one should shoot an idea down based off
> current levels of expectation.
>
Such tool requires good, open and free dictionaries. Dict databases
aren't very good and broad. With phrasal and idiomatic things situation
is even worse.

What makes good application aren't only good interface and features but
data to present them. IMO many KDE applications reached
interface/features level equal (or even better) comparing to commercial
programs. There is still lack of data for presentation. And in some
areas data is crucial - here is the case.

Super, there will be great tool but without data to feed it. Why waste
effort for doing that?

EOT for me (at least here, maybe kde-devel would be better place).

m.

On Sunday 13 February 2005 19:24, Zack Rusin wrote:

> On Sunday 13 February 2005 12:56, Enrico Ros wrote:
> > - How can be a new generation EBOOK reader ... ??
>
> I love KPdf but there's one thing that bothers me and it's really
> related to xpdf and it's the rendering speed of any pdf's with images.
> [ ... ]
> So essentially, I think that before tackling the really cool features we
> could focus on performance. The big question is how do we
> want to fix it? Improving the xpdf image rendering can be done by
> either porting the drawing code to Qt and then optimizing like any
> other Qt app/lib (which would probably kill the freedesktop xpdf
> initiative) or trying to simply improve the Xpdf drawing code.
> Opinions?

Finally I'm back from uni and I can post my considerations to the list.
Definitely this is an issue. KPdf overhead over xpdf is less than 5-10% on
mean cases, but when rendering images xpdf has some overkill bottlenecks!

-- problems details:
Btw you are totally right, I have a testcase that takes ages for
rendering [1]. This is mainly because of poor (well, maybe it's the opposite
case) rescaling stuff -> here are my first impressions:
- slow image downscaling: a 1500x2000 px image (pretty normal on 'scanned
   pdfs' or ones with graphical background) rendered to a 50x75 rectangle
   takes every pixel processed and manipulated.
   -> 3M operations (actually function calls) in such a case!
- drawPixel calls: Splash (the rendering class) has lots of memory buffer
   types, even if we (as kpdf) want 32bits only color pixels.
   drawPixel does its best to waste cpu (from 90% to 10% of the total
   time, depending on when the 'slow downscaling' problem tops that).
- bad image upscaling: image is not smoothscaled up.. it's a block
   scaling! like a QImage.scale().. (not,maybe worst) looks bad.

-- current state:
The scaling algorithm is something like a traversal to the DESTINATION
image (scaled/transoformed) but every pixel in the SOURCE image is
processed using a Bresenham like transform.. so:
-downscaling: every pixel on screen is the mean average of a 1 or many
pixel region in the source image. (on the other hand Downscaled images
looks better in xpdf than in acrobat reader)
-upscaling: a pixel in the source image is mapped to one or more pixels on
screen. (looks 'blocky')

-- the balance (IMO!):
-preserve some quality for downscaling (acrobat one can be enough)
-smooth in upscaling (linear iterpolation is way cooler than 'blocks'). note
that upscaling speed of acroread is similar and it doesn't interpolate too!

-- notes on the scaling algorithm [2] and implementation:
- image scaling must be 1) clipped 2) transformed (using available
   parameters).
- I think Mosfet is probably the best guy in the whole planet to ask
   for the fastest transformed image scaling. He can change the rendering
   speed by orders of magnitude I think.
- We already have a wrapper function in place so we 'might' optimize worst
   (and well defined) cases and let the other ones flow through the
   current code.
- The 'Splash' renderer supports many types of buffer formats (as said
   before). contraining its implementation to the 32 bits one will allow
   to remove many 'if's and 'case's and definitely speed up drawPixel.

References:
[1]: slow pdf
http://robotics.dei.unipd.it/~koral/KDE/PhysRevB.26.4199.pdf
[2]: rescaling stuff can be found on
http://webcvs.kde.org/kdegraphics/kpdf/xpdf/splash/Splash.cc?rev=1.3&view=markup
(Splash::drawPixel and Splash::drawImage methods)

Thanks for your work and all inputs,
Enrico
--
 
 
 --
 Email.it, the professional e-mail, gratis per te: http://www.email.it/f
 
 Sponsor:
 Novit? per telefonare risparmiando: Email.it Phone Card, clicca e scopri i vantaggi
 Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=2689&d=19-2

[Albert Astals Cid]

Moreover patterns on PDF are VERY slow, have a look at
http://bugs.kde.org/show_bug.cgi?id=99486 in xpdf (kpdf dies because second
page takes AGES).

In xpdf you can see pages 1, 3, 4, 5, 7, 8 and 9 without problems. Rendering
page 6 takes 1 minute to render in my system due to the graphic hacing a
pattern, i've never been able to finish the rendering of page 2. The problem
is for each small pattern piece it parses all the commands from the pdf and
then executes them. I've been have to reuse the parsing and get it down to 40
seconds but it is still too much. What should we done is render the pattern
once and then repeat the drawing all over the place, but i have not been able
of finding how to do it. If someone achieves this it can be a good reasont to
go on with the commonxpdf library at freedesktop.

Albert

A Dissabte 19 Febrer 2005 13:40, Enrico Ros va escriure:

> On Sunday 13 February 2005 19:24, Zack Rusin wrote:
> > On Sunday 13 February 2005 12:56, Enrico Ros wrote:
> > > - How can be a new generation EBOOK reader ... ??
> >
> > I love KPdf but there's one thing that bothers me and it's really
> > related to xpdf and it's the rendering speed of any pdf's with images.
> > [ ... ]
> > So essentially, I think that before tackling the really cool features we
> > could focus on performance. The big question is how do we
> > want to fix it? Improving the xpdf image rendering can be done by
> > either porting the drawing code to Qt and then optimizing like any
> > other Qt app/lib (which would probably kill the freedesktop xpdf
> > initiative) or trying to simply improve the Xpdf drawing code.
> > Opinions?
>
> Finally I'm back from uni and I can post my considerations to the list.
> Definitely this is an issue. KPdf overhead over xpdf is less than 5-10% on
> mean cases, but when rendering images xpdf has some overkill bottlenecks!
>
> -- problems details:
> Btw you are totally right, I have a testcase that takes ages for
> rendering [1]. This is mainly because of poor (well, maybe it's the
> opposite case) rescaling stuff -> here are my first impressions:
> - slow image downscaling: a 1500x2000 px image (pretty normal on 'scanned
>    pdfs' or ones with graphical background) rendered to a 50x75 rectangle
>    takes every pixel processed and manipulated.
>    -> 3M operations (actually function calls) in such a case!
> - drawPixel calls: Splash (the rendering class) has lots of memory buffer
>    types, even if we (as kpdf) want 32bits only color pixels.
>    drawPixel does its best to waste cpu (from 90% to 10% of the total
>    time, depending on when the 'slow downscaling' problem tops that).
> - bad image upscaling: image is not smoothscaled up.. it's a block
>    scaling! like a QImage.scale().. (not,maybe worst) looks bad.
>
> -- current state:
> The scaling algorithm is something like a traversal to the DESTINATION
> image (scaled/transoformed) but every pixel in the SOURCE image is
> processed using a Bresenham like transform.. so:
> -downscaling: every pixel on screen is the mean average of a 1 or many
> pixel region in the source image. (on the other hand Downscaled images
> looks better in xpdf than in acrobat reader)
> -upscaling: a pixel in the source image is mapped to one or more pixels on
> screen. (looks 'blocky')
>
> -- the balance (IMO!):
> -preserve some quality for downscaling (acrobat one can be enough)
> -smooth in upscaling (linear iterpolation is way cooler than 'blocks').
> note that upscaling speed of acroread is similar and it doesn't interpolate
> too!
>
> -- notes on the scaling algorithm [2] and implementation:
> - image scaling must be 1) clipped 2) transformed (using available
>    parameters).
> - I think Mosfet is probably the best guy in the whole planet to ask
>    for the fastest transformed image scaling. He can change the rendering
>    speed by orders of magnitude I think.
> - We already have a wrapper function in place so we 'might' optimize worst
>    (and well defined) cases and let the other ones flow through the
>    current code.
> - The 'Splash' renderer supports many types of buffer formats (as said
>    before). contraining its implementation to the 32 bits one will allow
>    to remove many 'if's and 'case's and definitely speed up drawPixel.
>
> References:
> [1]: slow pdf
> http://robotics.dei.unipd.it/~koral/KDE/PhysRevB.26.4199.pdf
> [2]: rescaling stuff can be found on
> http://webcvs.kde.org/kdegraphics/kpdf/xpdf/splash/Splash.cc?rev=1.3&view=m
>arkup (Splash::drawPixel and Splash::drawImage methods)
>
> Thanks for your work and all inputs,
> Enrico
> --
>
>
>  --
>  Email.it, the professional e-mail, gratis per te: http://www.email.it/f
>
>  Sponsor:
>  Novit? per telefonare risparmiando: Email.it Phone Card, clicca e scopri i
> vantaggi Clicca qui:
> http://adv.email.it/cgi-bin/foclick.cgi?mid=2689&d=19-2
> _______________________________________________
> Kde-graphics-devel mailing list
> Kde-graphics-devel@...
> https://mail.kde.org/mailman/listinfo/kde-graphics-devel

[Dirk Schönberger]

> The problem is for each small pattern piece it parses all the commands
from the pdf and
> then executes them. I've been have to reuse the parsing and get it down to
40
> seconds but it is still too much. What should we done is render the
pattern
> once and then repeat the drawing all over the place, but i have not been
able
> of finding how to do it.

I am not quite up to date how this is actually implemented in XPDF / KPDF,
but I would think you could either use some kind of metafile, where the
parsed PDF operators are stored, and can be replayed without re-parsing, or
you could build some kind of "nested" context, render the pattern once and
tile the resulting pattern image.

Regards
Dirk