On-the-fly color space conversions

[Stephan Assmus]

Hi all,

has anyone worked with on-the-fly colorspace conversions, or using images with different colorspaces within the same rendering pipeline? I've looked closer at agg_image_accessors.h, but it seems to be meant only for wrapping buffer byte offsets when there is out-of-bounds access. What I am looking for, sort of, is some kind of pixel format that converts into the format of the target buffer when reading pixels, so that image filters and span generators such would all work. Has anyone done something like this already and can share some experience?

Thanks and best regards,
-Stephan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

[Lorne Laliberte]

Hi Stephen,

Can you convert the images to RGB colorspace when they're loaded? Or
do you need to remain in their colorspace during processing?

I haven't done anything like this yet, but I'm planning to do an
on-the-fly RGB to HSL conversion for a colorizing filter I'd like to
implement. I wasn't planning on implementing a full pixel format for
it though -- just doing RGB to HSL and back during the lifetime of the
effect. I don't know if that will perform satisfactorily yet though; I
may need to implement a more generalized convertor, and "cache" the
HSL version so the conversions could be minimized. I'd be happy to
share any code I come up with when I get to that point.

I wonder if it could be done using an adaptor, sort of along the lines
of agg_pixfmt_amask_adaptor.h or agg_pixfmt_transposer.h?

What colorspaces do you need to convert from?

- Lorne Laliberte


On Fri, Nov 28, 2008 at 7:47 AM, Stephan Assmus <superstippi@...> wrote:

> Hi all,
>
> has anyone worked with on-the-fly colorspace conversions, or using images with different colorspaces within the same rendering pipeline? I've looked closer at agg_image_accessors.h, but it seems to be meant only for wrapping buffer byte offsets when there is out-of-bounds access. What I am looking for, sort of, is some kind of pixel format that converts into the format of the target buffer when reading pixels, so that image filters and span generators such would all work. Has anyone done something like this already and can share some experience?
>
> Thanks and best regards,
> -Stephan
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Vector-agg-general mailing list
> Vector-agg-general@...
> https://lists.sourceforge.net/lists/listinfo/vector-agg-general

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

[Stephan Assmus]

Hi Lorne,

> Can you convert the images to RGB colorspace when they're loaded? Or
> do you need to remain in their colorspace during processing?

Yes. I am implementing an API which offers to store and use images in different format. Since there are no functions to directly access image data, but only for importing/exporting data, I could possibly fake the whole thing and pretend to be storing images in the requested format while I really only have one internal format. But I am guessing the whole point of offering different color spaces is to keep the memory requirements in check, so I better honor the requested formats.

> I haven't done anything like this yet, but I'm planning to do an
> on-the-fly RGB to HSL conversion for a colorizing filter I'd like to
> implement. I wasn't planning on implementing a full pixel format for
> it though -- just doing RGB to HSL and back during the lifetime of the
> effect. I don't know if that will perform satisfactorily yet though; I
> may need to implement a more generalized convertor, and "cache" the
> HSL version so the conversions could be minimized. I'd be happy to
> share any code I come up with when I get to that point.
>
> I wonder if it could be done using an adaptor, sort of along the lines
> of agg_pixfmt_amask_adaptor.h or agg_pixfmt_transposer.h?

I looked at agg_pixfmt_transposer.h, but that seems to be for changing the attached buffer at runtime only (because later classes in the pipeline take references to pixel format objects, which cannot be changed once intantiated).

But I'll take another look at agg_pixfmt_amask_adaptor.h.

> What colorspaces do you need to convert from?

Oh, these are different ones... 16 bit RGB spaces with and without differently sized alpha channels and 32 bit RGB with different channel order and all that multiplied with linear and sRGB versions. As well as luminance, grayscale images and even black and white bitmaps. :-)

Best regards,
-Stephan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

[Lorne Laliberte]

On Fri, Nov 28, 2008 at 11:07 AM, Stephan Assmus <superstippi@...> wrote:
>> Can you convert the images to RGB colorspace when they're loaded? Or
>> do you need to remain in their colorspace during processing?
>
> Yes. I am implementing an API which offers to store and use images in different format. Since there are no functions to directly access image data, but only for importing/exporting data, I could possibly fake the whole thing and pretend to be storing images in the requested format while I really only have one internal format. But I am guessing the whole point of offering different color spaces is to keep the memory requirements in check, so I better honor the requested formats.

That does bring up a good question -- is the goal to just be
compatible with loading/saving those kinds of images, or is the goal
to actually operate in their colorspace? If it's the latter,
converting on the fly might not be enough (except for when you're
converting "up" to a more detailed colorspace). It depends whether the
users want the detail (e.g. more bits per channel) to be preserved
during processing.

>> I wonder if it could be done using an adaptor, sort of along the lines
>> of agg_pixfmt_amask_adaptor.h or agg_pixfmt_transposer.h?
>
> I looked at agg_pixfmt_transposer.h, but that seems to be for changing the attached buffer at runtime only (because later classes in the pipeline take references to pixel format objects, which cannot be changed once intantiated).
>
> But I'll take another look at agg_pixfmt_amask_adaptor.h.

Ah, they're both pretty similar. I meant more that the conversion
could be done in an adaptor like that. I imagined it containing a
reference to the buffer containing the image in the other colorspace,
and then converting the spans on the fly into some temporary storage
for the span (scanline_storage_bin perhaps?).

>> What colorspaces do you need to convert from?
>
> Oh, these are different ones... 16 bit RGB spaces with and without differently sized alpha channels and 32 bit RGB with different channel order and all that multiplied with linear and sRGB versions. As well as luminance, grayscale images and even black and white bitmaps. :-)

Ouch, that's a lot of formats.

If you want to fake it, I won't tell... :)

- Lorne Laliberte

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

Hi,

When doing some interpolation between destination pixels, for every
destination pixel multiple source pixels must be converted to the correct
color space.
This means it is faster to convert the source at once to the required
colorspace, then every source pixel just needs to be converted once.

But when you need all of those mentioned format during one combined drawing
action, this would mean a lot of extra memory.

So do you want speed or memory ? :-)

Edgar


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

[Stephan Assmus]

> When doing some interpolation between destination pixels, for every
> destination pixel multiple source pixels must be converted to the correct
> color space.
> This means it is faster to convert the source at once to the required
> colorspace, then every source pixel just needs to be converted once.

It may be enough to convert individual scanlines and clobber them with the next one. It depends. I need to look at some image filters and analyse their access pattern. It could be pretty random, I need to come up with something solid. I was just trying to see if anybody else out there already had to do this and could give some pointers.

Best regards,
-Stephan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

[Lorne Laliberte]

>> When doing some interpolation between destination pixels, for every
>> destination pixel multiple source pixels must be converted to the correct
>> color space.
>> This means it is faster to convert the source at once to the required
>> colorspace, then every source pixel just needs to be converted once.
>
> It may be enough to convert individual scanlines and clobber them with the next one. It depends. I need to look at some image filters and analyse their access pattern. It could be pretty random, I need to come up with something solid. I was just trying to see if anybody else out there already had to do this and could give some pointers.

You could prefetch/cache a number of scanlines, depending on the
vertical distance of the filter.

It would be much easier to just convert the entire image, though --
either on load, or at the time of first necessity. If you don't mind
slowing down any possible "undo" step, you could drop the original
image data after you convert it into the colorspace you use for
processing, or do so conditionally based on available memory.

The best solution would be to have native support for the colorspaces,
i.e. actual pixel formats for them...perhaps doing real-time
conversion only for the blend modes that don't make sense, or turning
them into no-ops.

Sorry if that's just spelling out the obvious. :)

Lorne Laliberte

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

[Lorne Laliberte]

On Fri, Nov 28, 2008 at 8:47 AM, Stephan Assmus <superstippi@...> wrote:
> Hi all,
>
> has anyone worked with on-the-fly colorspace conversions, or using images with different colorspaces within the same rendering pipeline?

Hi Stephan,

I wanted to let you know what I learned when I implemented real-time
RGB->HSL->RGB conversions in my draw engine.

For smaller areas, doing the conversion in real time works quite well,
actually. I implemented it using a span generator "adaptor" very
similar to the span_converter template in AGG. This is what the
generate function looks like:

void generate(agg::rgba8* span, int x, int y, unsigned len)
{
        m_sourceFill->generate(span, x, y, len);

        for(unsigned i = 0; i < len; ++i, ++span)
        {
                if(span->a != 0)
                {
                        span->demultiply();
                        HSL hsl = RGB2HSL(*span);
                        m_op->Adjust(hsl);
                        *span = HSL2RGB(hsl);
                        span->premultiply();
                }
        }
}

In my case I'm using a shared_ptr to the source span generator,
"m_sourceFill" which is essentially a custom span generator derived
from a "Fill" abstract base class. (I needed a generalized "fill" type
in order to support the compound rasterizer in my engine. These fill
types can be strung together to produce compound span generators. For
example, the above code is from an "HSLAdaptorFill" class template,
which I can attach to any other fill type in order to perform a
real-time HSL color model adjustment to each span.)

m_op in the above code points to an "HSL operator," which is
essentially a policy functor implementing some kind of HSL adjustment,
or a unique "chain" of them (similar to AGG's coordinate conversion
pipeline).

The above method -- performing the RGB to HSL to RGB conversion on the
fly -- is pretty quick, all things considered, but it isn't fast
enough for larger areas, or when there is a lot of interpolation going
on. I ended up storing a colorized version of the full source image,
processing each pixel as it's copied over from the original source
image into another buffer. This allows me to just use the colorized
version in place of the original source image and pay only a one-time
processing cost for the color conversion. (At the expense of greater
memory use, of course.)

I still use both methods in my app -- for example the real-time method
is faster than converting the entire source image whenever a
downscaled image is being drawn, so I use that method to improve
responsiveness while the user is adjusting HSL conversion parameters
in the GUI.

Not sure how relevant this is to your situation, Stephan, but
hopefully it helps.

Lorne Laliberte
Senior Software Developer, Indigo Rose Software

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general

[Stephan Assmus]

Hi Lorne,

thanks for the info! In the specific API I was implementing, the client
code can never access image data directly, it always has to go through
import and export functions. This allowed me to use an internal image
format for the bulk of the color spaces that I need to support, so that
conversion only happens at those import/export functions, but never
during the rendering pipeline.

If I had to implement what I originally asked about, I believe it would
have had to do with Image Accessors doing the necessary on-the-fly
conversion when reading and supplying scanlines.

Best regards,
-Stephan

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Vector-agg-general mailing list
Vector-agg-general@...
https://lists.sourceforge.net/lists/listinfo/vector-agg-general