CgBI PNG derivative and patent pending

[Glenn Randers-Pehrson]

Followups on this thread [from png-mng-implement]  to png-mng-misc, please.

Glenn

---------- Forwarded message ----------
From: Bob Friesenhahn <bfriesen@...>
Date: Sun, Apr 26, 2009 at 12:21 PM
Subject: [png-mng-implement] CgBI PNG derivative and patent pending
To: png-mng-implement@...


Via Google I noticed that Apple has a patent pending for their CgBI
chunk PNG hack:

  http://www.freepatentsonline.com/y2008/0177769.html

The patent application is intentionally written to be obtuse but
ultimately it does not describe much innovation and seems to be an
attempt to lock-down the PNG-derived format used in the Apple iPhone
and likely other portable Apple devices.

The level of technology employed is considerable <smirk>.  For
example, if the file has the CgBI header, then CRCs can be ignored,
the RGBA pixels are written in a different order (BGRA), and the data
is pre-multiplied if there is an alpha channel.  In essence, Apple is
trying to obtain a patent for a simplification/optimization of the PNG
format to execute more efficiently on their particular hardware.

IMHO this patent application is shameful.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

------------------------------------------------------------------------------

Thanks.  The patent seems to talk about the CRC "may be skipped" but
really means it "must be skipped" because the encoder is not writing
one.  Libpng provides prior art for "may be skipped" because it
has always had the option for decoders (but not encoders) to
skip the CRC validation.

It is indeed shameful.

Glenn

------------------------------------------------------------------------------

More prior art exists in the PNG specification itself.  It shows that
we thought of, and
rejected, the idea of changing the meaning of samples in a pixel.  We said don't
change it to actually mean CMYK but I think that includes "don't change it to
actually mean BGRA".  But since we documented the fact that we thought
of it years ago, no one should be able to patent the idea now.

Glenn

---------------------------------------

Guidelines for new chunk types

This International Standard allows extension through the addition of
new chunk types and new interlace, filter, and compression methods.
....

Some guidelines for defining private chunks are given below.

  1. Do not define new chunks that redefine the meaning of existing
chunks or change the interpretation of an existing standardized chunk,
e.g., do not add a new chunk to say that RGB and alpha values actually
mean CMYK.
Glenn
------------------------------------------------------------------------------
From: Bob Friesenhahn [mailto:bfriesen@...]
>The patent application is intentionally written to be obtuse but
>ultimately it does not describe much innovation and seems to be an
>attempt to lock-down the PNG-derived format used in the Apple iPhone
>and likely other portable Apple devices.

It's just a dumb patent.  I wrote a long email showing why every single
invention in it is dumb then I deleted it (the email.)

It seems likely to me that Apple have a very reduced PNG decoder, capable
only of handling a small subset of the format - no CRCs, just BGRA
pre-multiplied values, no filtering.  The "invention" is to hack a chunk
into the head of the PNG to indicate that the image does not exceed the
capabilities of such a decoder.

They should have invented a new file format - just a deflate compressed
stream of BGRA values.  They didn't.  It's not PNG, we don't care.

John Bowler <jbowler@...>

------------------------------------------------------------------------------

From: Glenn Randers-Pehrson [mailto:glennrp@...]
>  We said don't
>change it to actually mean CMYK but I think that includes "don't change it
to
>actually mean BGRA".

BGRA (post-multiplied) PNG is perfectly valid - just swap the red and blue
end points in the cHRM chunk.

CMYK isn't because, while "K" behaves a bit like "A", the redefinition of
the values to be subtractive and the reuse of the alpha channel for an ink
component cannot be expressed in either cHRM or iCCP.

BGRA (or RGBA) pre-multiplied is, likewise, invalid because the alpha
compositing change, even if the author of the file format understands it,
can't be expressed even if the gAMA is 1.0

John Bowler <jbowler@...>

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Glenn Randers-Pehrson]

> They should have invented a new file format - just a deflate compressed
> stream of BGRA values.  They didn't.  It's not PNG, we don't care.
>
> John Bowler <jbowler@...>

They call it PNG and use the PNG signature and extension, so we do
care.

Indeed it's not PNG because it uses little-endian byte format,
has a chunk ahead of IHDR, and has invalid CRC checksums.
The compression method is not exactly zlib but we allow that
(but not without changing the compression method byte in IHDR
which apparently they are not doing).

Unreadable CgBI PNGs are already deployed and people
have written various applications to deal with them.  Most of
those are incorrect in one way or another, as far as I can tell --
they don't deal with the premultiplied alpha or the weird deflate
compression.

If the patent is granted, the patent holder could forbid or
otherwise control such third-party applications.  I was
thinking of modifying the decoder in pngcrush and *Magick
to handle them but since Bob pointed out the patent
application I think all I will do is throw an error (we already
do that because of the unknown chunk, but I'd try to
produce a more useful error message).

Glenn

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[John Bowler]

From: Glenn Randers-Pehrson [mailto:glennrp@...]
> John Bowler <jbowler@...>:
>> They should have invented a new file format - just a deflate compressed
>> stream of BGRA values.  They didn't.  It's not PNG, we don't care.


>They call it PNG and use the PNG signature and extension, so we do
>care.

Right; I was referring strictly to the patent.  A valid implementation of
the patent would necessarily change the PNG signature on the data as well,
for the reasons we have discussed.

CgBI is wrong for several reasons.  Not least the chunk apparently has no
Apple specific signature (IRC) - this sets Apple up for a situation where
PNGs using a valid CgBI chunk from another source (e.g. anything produced by
www.cgbi.fr) emerge and cause confusion (for Apple, not CGBI, who would
presumably put a chunk signature in.)

>Unreadable CgBI PNGs are already deployed and people
>have written various applications to deal with them.
>
>If the patent is granted, the patent holder could forbid or
>otherwise control such third-party applications.

I'm not sure that would stand up in a US court (because of the data
interoperability purpose), but then, IRC, the issue of using GIF decoders
for data interoperability never got tested in a court.  The problem is that
the patent can clearly be used to menace weaker competitors into ceasing to
compete should Apple desire.   Alternatively it could be used to simply get
money under threat of court action, should Apple come to think that
extracting a few bucks from every user of a PNG decoder is a way of keeping
its head above water.

>I was thinking of modifying the decoder in pngcrush and *Magick
>to handle them but since Bob pointed out the patent
>application I think all I will do is throw an error (we already
>do that because of the unknown chunk, but I'd try to
>produce a more useful error message).

Perhaps something along the lines of:  "This PNG contains a CgBI chunk that
prevents its display.  Interpretation of data in a CgBI chunk may be patent
protected."

BTW you can't write the correct code: I don't think it is possible to
generate post-multiplied RGBA values from the pre-multiplied values because,
in general, the color component values required depend on the color
components in the *destination*.  (I.e. you want an RGBA PNG that will
produce the same output as display of the Apple pre-multiplied BGRA data,
but the display of the Apple data depends on the values of the pixels onto
which the data is composited in a way that can't be expressed in PNG.)

John Bowler <jbowler@...>



------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[John Bowler]

From: Glenn Randers-Pehrson [mailto:glennrp@...]
>They call it PNG and use the PNG signature and extension, so we do
>care.

To put this in context, according to this thread:

http://discussions.apple.com/message.jspa?messageID=8314150

The IHDR is being changed to CgBI.  So far as I am concerned the PNG file
signature is and always has been 12 bytes - the 8 byte "signature" followed
by "IHDR" - because the specification requires that every valid PNG file
start with those 12 bytes.

The transformation is only happening when the data is packaged for the
target device, apparently people, probably not Apple people, are hacking the
result out and expecting it to work in other contexts.  It won't, it's like
compiled machine code versus the original source code.

Most importantly: *the original PNG cannot be recovered*.  It's a strictly
one way operation and once it is done it cannot be undone.

It's a pity they didn't zap, or just drop, the 8 byte header too, but,
regardless of the merits of what they did, the real problem is a failure of
understanding in people outside Apple as to what they can do with the
compiled data.

John Bowler <jbowler@...>



------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Glenn Randers-Pehrson]

On Mon, Apr 27, 2009 at 2:33 PM, John Bowler <jbowler@...> wrote:

> The IHDR is being changed to CgBI.  So far as I am concerned the PNG file
> signature is and always has been 12 bytes - the 8 byte "signature" followed
> by "IHDR" - because the specification requires that every valid PNG file
> start with those 12 bytes.

Actually 16 bytes: the 8-byte signature, a 4-byte unsigned integer 13 in
network byte order, then "IHDR".  But there are plenty of applications,
including all libpng-based ones, that only check 8 bytes or less.

Simple applications like file(1) are entitled to expect to find the width
and height in the next 8 bytes.  If there is a CgBI chunk where
IHDR is supposed to be, then weird results will be produced.

Glenn

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensign option that enables unlimited
royalty-free distribution of the report engine for externally facing
server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Glenn Randers-Pehrson]

Adding insult to injury, Apple is using a modified version of pngcrush
to create these rogue non-PNGs, but they are still calling the
program pngcrush.  My license seems to allow that but it sux
all the same.  I would not mind seeing a copy of said modified
pngcrush program.

#:-(

Glenn

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Willem van Schaik]

But whoever said Apple was a good Net-Citizen? Even with a MacBook Pro
being my daily workhorse, I'm still absolutely convinced that they have
been doing more "grabbing" from the public domain than that they have
contributed.

Like so many other successful companies, they're the king of proprietary
apps. No illusions there,

Willem


PS. I'm sure they would translate "you grabbed pngcrush" with "oh, we
forked it". Unfortunately, the world isn't fair....



Glenn Randers-Pehrson wrote:

> Adding insult to injury, Apple is using a modified version of pngcrush
> to create these rogue non-PNGs, but they are still calling the
> program pngcrush.  My license seems to allow that but it sux
> all the same.  I would not mind seeing a copy of said modified
> pngcrush program.
>
> #:-(
>
> Glenn
>
> ------------------------------------------------------------------------------
> Register Now & Save for Velocity, the Web Performance & Operations
> Conference from O'Reilly Media. Velocity features a full day of
> expert-led, hands-on workshops and two days of sessions from industry
> leaders in dedicated Performance & Operations tracks. Use code vel09scf
> and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
> _______________________________________________
> png-mng-misc mailing list
> png-mng-misc@...
> https://lists.sourceforge.net/lists/listinfo/png-mng-misc
>

--

Willem van Schaik
<willem@...>
http://www.schaik.com/


------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[glennrp-2]

----- Original Message -----
From: "Willem van Schaik" <willem@...>

PS. I'm sure they would translate "you grabbed pngcrush" with "oh, we
forked it". Unfortunately, the world isn't fair....

    How would they translate "you ignored the license requirements"?

    They are distributing a pngcrush, over my name (and Greg's) that
    produces invalid PNG files.  They failed to abide by the simple
    requirement that they state that they modified the code (by
    adding -iPhone and -speed options and who knows what else).
    I don't know if they distribute source; the license does not
    require them to, so all I know is by applying "strings -a"
    to the executable that they distribute as a part of their SDK.

    They are damaging the reputation of pngcrush and my professional
    reputation.

    IANAL and IDNHAL but I guess I need one now.

    Glenn

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Glenn Randers-Pehrson]

FYI:

"CgBI" appears to mean "CGBitmap" which I believe
is an Apple internal image representation.

Glenn

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[John Bowler]

From: glennrp@... [mailto:glennrp@...]
>    They are damaging the reputation of pngcrush and my professional
>    reputation.

Yes, I would say that too; my immediate reaction when I saw the comments
about pngcrush was to wonder why an application supported by you was doing
this.  Of course that reaction lasted for just a fraction of a second, but I
know much more about libpng than most developers...

I'm sure Apple did not intentionally retain the name to add authenticity to
their product, but nevertheless that seems to be the result.

>    IANAL and IDNHAL but I guess I need one now.

Likewise IANAL, but I'm not sure you really need one - they tend stick
around and get in the way.  I guess the license could require a change to
the executable name on any substantial modification, however I think this is
just an example of sloppy development.  The Apple pngcrush is apparently
doing something new, I'd call it "pngtoxcode" or maybe "xcodepng".

Good development practice requires that a spade be called a spade.  I don't
see why Apple shouldn't be amenable to simply changing the name and maybe
even making it clear that the modifications are irreversible because of the
loss of precision in the pre-multiplication.

John Bowler <jbowler@...>


------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Glenn Randers-Pehrson]

On Tue, Apr 28, 2009 at 11:44 AM, John Bowler <jbowler@...> wrote:
>
>>    IANAL and IDNHAL but I guess I need one now.
>
> Likewise IANAL, but I'm not sure you really need one - they tend stick
> around and get in the way.

Google seems to think I do.  The sidebar is full of IP lawyer ads when
I use Gmail to look at this thread.

Glenn

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Willem van Schaik]

>
>     How would they translate "you ignored the license requirements"?
>
>     They are damaging the reputation of pngcrush and my professional
>     reputation.
>
>     IANAL and IDNHAL but I guess I need one now.
>

As John said, maybe just start with writing them a letter / email and
ask them friendly to reverse their actions.

Besides the issues with PngCrush, isn't it time that as a PNG collective
we write an 'open letter' to Apple simply asking them to use a different
signature and file extension for their new format? Maybe we can get some
publicity by sending an email to someone like Cringely (either of the
two :-) he loves that type of stuff for his weekly column.

--

Willem van Schaik
<willem@...>
http://www.schaik.com/


------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc

[Willem van Schaik]

Willem van Schaik wrote:

>
> Besides the issues with PngCrush, isn't it time that as a PNG collective
> we write an 'open letter' to Apple simply asking them to use a different
> signature and file extension for their new format?
>

Noticed the following piece (written by Greg, I guess) in the PNG
history: "Nevertheless, progress continued. In June of 1995 I set up the
PNG home page, now grown to roughly a dozen pages [7]; Kevin Mitchell
officially registered the ``PNGf'' Macintosh file ID with Apple
Computer. In August Alexander Lehmann and Willem van Schaik released a
fine pair of additions to the NetPBM image-manipulation suite,
particularly handy under Linux: pnmtopng and pngtopnm version 2.0. And
in December at the Fourth International World Wide Web Conference, the
World Wide Web Consortium (W3C) released the PNG Specification version
0.92 as an official standards-track Working Draft."


--

Willem van Schaik
<willem@...>
http://www.schaik.com/


------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
_______________________________________________
png-mng-misc mailing list
png-mng-misc@...
https://lists.sourceforge.net/lists/listinfo/png-mng-misc