Nabble - Tiff / LibTiff

Re: YCbCr

2009-12-12T10:27:15Z

> So, you can store either RGB, YCbCr, or CMYK data in TIFF files.
> However, due to the variable conversion algorithms used for converting
> to/from CMYK or YCbCr, anything other than RGB is probably not likely
> to work outside of whatever application creates them. Is that a
> correct statement?

No, I said only for CMYK. YCbCr has a very precise conversion for RGB.

> Also, if a user employs the automatic conversion process to get from
> YCbCr to RGB, doesn't the wide variety of possible conversion matrices
> for this process make the automatic conversion somewhat inconsistent or
> even incorrect in some cases?
> Does LibTiff implement the most common
> conversion matrix (if that's even clearly defined) for this automatic
> process, or is there some guess at a conversion matrix based on the
> data itself?

Both no, for the same reason.

> It just seems to me that storing image data in anything other than RGB-
> based formats is not a good idea for most general usages and should
> only be employed if the images are used for internal program processing
> only. Otherwise, the matrix employed to generate the YCbCr or CMYK
> format is not known to a reader of the file.

As Toby explained, CMYK is used for publishing workflows. And the conversion is complicated. No it is not commonly used.

On the other hand YCbCr is widely used. Mostly because you can separate luminance and chrominance information, and compress them with different approaches. Almost all JPEG files are stored in this way (I mean *.jpg).

So actually it depends on what your application does. Also depends on what file format/compression you choose.

Best,
Scuri

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-12T10:19:39Z

On Sat, 12 Dec 2009, Gene Amtower wrote:

> So, you can store either RGB, YCbCr, or CMYK data in TIFF files. However, due to the variable conversion
> algorithms used for converting to/from CMYK or YCbCr, anything other than RGB is probably not likely to work
> outside of whatever application creates them. Is that a correct statement?

While CMYK is quite variable since it is device dependent, YCbCr is
defined by several ITU specifications (Rec.601 or Rec.709) and is
therefore quite well defined.

While YCbCr should work quite nicely in TIFF, the only common use
seems to be in conjunction with JPEG compression.

Except for issues related to subsampling (which requires horizontal,
and sometimes even vertical filters), YCbCr is not difficult to work
with.

> It just seems to me that storing image data in anything other than
> RGB-based formats is not a good idea for most general usages and
> should only be employed if the images are used for internal program
> processing only. Otherwise, the matrix employed to generate the
> YCbCr or CMYK format is not known to a reader of the file.

For general usage, you should stick to baseline TIFF.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-12T09:34:30Z

On 12-Dec-09, at 11:37 AM, Gene Amtower wrote:

So, you can store either RGB, YCbCr, or CMYK data in TIFF files. However, due to the variable conversion algorithms used for converting to/from CMYK or YCbCr, anything other than RGB is probably not likely to work outside of whatever application creates them. Is that a correct statement?

Not really with respect to CMYK. TIFF is commonly used as an interchange format, e.g. from scanner to artist's computer. It is also commonly used as an archive/working format for print-oriented images. The actual interpretation of data is either by tacit agreement (e.g. "We assume SWOP inks, 20% dot gain") and corresponding program settings, or by embedded ICC profile. Software in print production has historically used both (specifically Quark XPress, InDesign, Photoshop, etc).

Historically, CMYK workflows were based on passing through the scanned data without profile-based re-interpretation:

– initial transparency scan in CMYK from a drum scanner with 'internal' tacit characteristics (e.g. UCR, dot gain)

– transport to artist's computer

– import to layout where an RGB proxy is generated

– generation of PostScript separations referencing original CMYK scan (e.g. to film)

– production of printing plates

This workflow (typical during 1990s) would assume that the parameters and data of the original scan were correctly tuned for the end product – usually offset lithographic printing. It is also based on film photography. TIFF is ubiquitous in this workflow as given the software available at the time, it was the most efficient and reliable of the alternatives.

Nowadays this flow would include profile based conversions and the original data is much more commonly non-CMYK digital data. Separation occurs late in the process, is done "on the fly", and would also involve a profile for the output device. So in a modern workflow there is in fact very little place for CMYK TIFF except when dealing with 'legacy' data.

...

It just seems to me that storing image data in anything other than RGB-based formats is not a good idea for most general usages

In CMYK common usage would be existing archives of imagery intended for print (of which there must exist an enormous quantity in archives generated over the last 20 years).

and should only be employed if the images are used for internal program processing only. Otherwise, the matrix employed to generate the YCbCr or CMYK format is not known to a reader of the file.

The CMYK would typically be generated by a "black box", for example a drum scanner. It is possible to characterise this by an ICC profile but this is not always done. But then RGB is not really different in this respect.

--Toby

Am I on the right track, or does LibTiff provide some means to communicate the conversion matrix to any other program reading the TIFF file?

Thanks,

Gene

On Sat, 2009-12-12 at 00:37 -0200, Antonio Scuri wrote:
My recommendation for you is to start at the TIFF format specification pdf. It has several interesting explanations that will answer some of your questions.

Anyway, here are some comments.

Yes there is no pre-defined conversion between CMYK and everything else. It really depends on how you should interpret the K component, and this usually involves the printer profile you are going to use.

Using libTIFF you can access images that are in the YCbCr color space in its raw state or with an automatic conversion to RGB. At least in the JPEG compression. The libtiff high level functions to load an RGB image do not use the automatic conversion, they load the image as YCbCr and then convert the raw data to RGB itself. By the way this is a recent change in libtiff.

Best Regards,

Antonio Scuri

From: tiff-bounces@... [tiff-bounces@...] On Behalf Of Gene Amtower
Sent: sexta-feira, 11 de dezembro de 2009 17:11
To: Steve Mills
Cc: tiff list
Subject: Re: [Tiff] YCbCr

Well, I was following this thread thinking I might learn something from the discussion, but somewhere along the way, it went off the tracks for me!

This last post confuses me because the original post mentioned that the TIFFTAG_PHOTOMETRIC tag was reporting YCbCr, where as this one suggests the image is in CMYK. Are we still talking about the original image from the first post? If so, why is there this apparent contradiction between the first and last post?

All of this info got me to reading on the web, and I see that YCbCr is a colorspace primarily used for signal transmission to reduce the amount of transmission bandwidth, hence the downsampling of the Cb and Cr components because they carry less data. Info on Wikipedia provides conversion formulas to/from RGB that depend on conversion factors that are not universal. From these formulas, a red pixel ends up generating a value in each of the Y, Cb, and Cr pixel values because "Y" carries the primary luminance info, while Cb and Cr only communicate a "difference" value for the Red and Blue components. It also suggested to me that you can't really view YCbCr values directly because they are calculated values used in data transmission that aren't directly related to the RGB world that we live in. From what I can tell, YCbCr and CMYK are completely different colorspaces for either signal transmission or printing (respectively), and converting from one to the other actually involves an intermediate pass through the RGB space, with both conversions involving variable formulas that depend on hardware-dependent and implementation-dependent mapping algorithms. It sounds like there is NO universal way to convert between these three representations. There's also the RGBA colorspace that includes transparency info in the Alpha A channel - does this impact YCbCr and CMYK representations at all, or is this impossible in these alternate colorspaces?

If anyone cares to enlighten me, can you help me understand this discussion on LibTiff handling of RGB, CMYK, and YCbCr without going into a long dissertation on images, color spaces, TVs, and printing processes? Is it possible to store YCbCr image data in a TIFF file? If so, are there mechanisms built into LibTiff to convert it automatically to RGB for viewing, or is that up to the primary application to perform necessary conversions to the viewable world? With the inherent variability of YCbCr color values, how would Steve even know if he was getting valid YCbCr data when reading the values from the image file?

I guess a quick summary of the colorspace capabilities in LibTiff and the Tiff standard in general would be really helpful to the uninitiated among us. I know there's a lot of vagueness in the TIFF standard, but explain as much as possible, please!

Thanks,

Gene

On Fri, 2009-12-11 at 10:58 -0600, Steve Mills wrote:
 
On Dec 11, 2009, at 10:15:39, Antonio Scuri wrote:
 
>  As Joris pointed out, you actually get Y Cb and Cr planes. So the first
> component, that looks red, is infact Y, and indeed has the correct size. But
> the Cb and Cr planes are downsampled. At least this is what I got here.
 
What if the file is jpeg-compressed cmyk? The TIFFTAG_PHOTOMETRIC tag says it's cmyk, not YCbCr. I expect cmyk data in this case, but it sure doesn't look correct.
 
_________________________________________________________
Steve Mills                              Me: 952-401-6255
Senior Software Architect                         MultiAd
smills@...                       www.multi-ad.com
 
 
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-12T08:37:58Z

So, you can store either RGB, YCbCr, or CMYK data in TIFF files. However, due to the variable conversion algorithms used for converting to/from CMYK or YCbCr, anything other than RGB is probably not likely to work outside of whatever application creates them. Is that a correct statement?

Also, if a user employs the automatic conversion process to get from YCbCr to RGB, doesn't the wide variety of possible conversion matrices for this process make the automatic conversion somewhat inconsistent or even incorrect in some cases? Does LibTiff implement the most common conversion matrix (if that's even clearly defined) for this automatic process, or is there some guess at a conversion matrix based on the data itself?

It just seems to me that storing image data in anything other than RGB-based formats is not a good idea for most general usages and should only be employed if the images are used for internal program processing only. Otherwise, the matrix employed to generate the YCbCr or CMYK format is not known to a reader of the file.

Am I on the right track, or does LibTiff provide some means to communicate the conversion matrix to any other program reading the TIFF file?

Thanks,

Gene

On Sat, 2009-12-12 at 00:37 -0200, Antonio Scuri wrote:

My recommendation for you is to start at the TIFF format specification pdf. It has several interesting explanations that will answer some of your questions.

Anyway, here are some comments.

Yes there is no pre-defined conversion between CMYK and everything else. It really depends on how you should interpret the K component, and this usually involves the printer profile you are going to use.

Using libTIFF you can access images that are in the YCbCr color space in its raw state or with an automatic conversion to RGB. At least in the JPEG compression. The libtiff high level functions to load an RGB image do not use the automatic conversion, they load the image as YCbCr and then convert the raw data to RGB itself. By the way this is a recent change in libtiff.

Best Regards,

Antonio Scuri

From: tiff-bounces@... [mailto:tiff-bounces@...] On Behalf Of Gene Amtower
Sent: sexta-feira, 11 de dezembro de 2009 17:11
To: Steve Mills
Cc: tiff list
Subject: Re: [Tiff] YCbCr

Well, I was following this thread thinking I might learn something from the discussion, but somewhere along the way, it went off the tracks for me!

This last post confuses me because the original post mentioned that the TIFFTAG_PHOTOMETRIC tag was reporting YCbCr, where as this one suggests the image is in CMYK. Are we still talking about the original image from the first post? If so, why is there this apparent contradiction between the first and last post?

All of this info got me to reading on the web, and I see that YCbCr is a colorspace primarily used for signal transmission to reduce the amount of transmission bandwidth, hence the downsampling of the Cb and Cr components because they carry less data. Info on Wikipedia provides conversion formulas to/from RGB that depend on conversion factors that are not universal. From these formulas, a red pixel ends up generating a value in each of the Y, Cb, and Cr pixel values because "Y" carries the primary luminance info, while Cb and Cr only communicate a "difference" value for the Red and Blue components. It also suggested to me that you can't really view YCbCr values directly because they are calculated values used in data transmission that aren't directly related to the RGB world that we live in. From what I can tell, YCbCr and CMYK are completely different colorspaces for either signal transmission or printing (respectively), and converting from one to the other actually involves an intermediate pass through the RGB space, with both conversions involving variable formulas that depend on hardware-dependent and implementation-dependent mapping algorithms. It sounds like there is NO universal way to convert between these three representations. There's also the RGBA colorspace that includes transparency info in the Alpha A channel - does this impact YCbCr and CMYK representations at all, or is this impossible in these alternate colorspaces?

If anyone cares to enlighten me, can you help me understand this discussion on LibTiff handling of RGB, CMYK, and YCbCr without going into a long dissertation on images, color spaces, TVs, and printing processes? Is it possible to store YCbCr image data in a TIFF file? If so, are there mechanisms built into LibTiff to convert it automatically to RGB for viewing, or is that up to the primary application to perform necessary conversions to the viewable world? With the inherent variability of YCbCr color values, how would Steve even know if he was getting valid YCbCr data when reading the values from the image file?

I guess a quick summary of the colorspace capabilities in LibTiff and the Tiff standard in general would be really helpful to the uninitiated among us. I know there's a lot of vagueness in the TIFF standard, but explain as much as possible, please!

Thanks,

Gene

On Fri, 2009-12-11 at 10:58 -0600, Steve Mills wrote:
 
On Dec 11, 2009, at 10:15:39, Antonio Scuri wrote:
 
>  As Joris pointed out, you actually get Y Cb and Cr planes. So the first
> component, that looks red, is infact Y, and indeed has the correct size. But
> the Cb and Cr planes are downsampled. At least this is what I got here.
 
What if the file is jpeg-compressed cmyk? The TIFFTAG_PHOTOMETRIC tag says it's cmyk, not YCbCr. I expect cmyk data in this case, but it sure doesn't look correct.
 
_________________________________________________________
Steve Mills                              Me: 952-401-6255
Senior Software Architect                         MultiAd
smills@...                       www.multi-ad.com
 
 
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-11T18:37:36Z

My recommendation for you is to start at the TIFF format specification pdf. It has several interesting explanations that will answer some of your questions.

Anyway, here are some comments.

Yes there is no pre-defined conversion between CMYK and everything else. It really depends on how you should interpret the K component, and this usually involves the printer profile you are going to use.

Using libTIFF you can access images that are in the YCbCr color space in its raw state or with an automatic conversion to RGB. At least in the JPEG compression. The libtiff high level functions to load an RGB image do not use the automatic conversion, they load the image as YCbCr and then convert the raw data to RGB itself. By the way this is a recent change in libtiff.

Best Regards,

Antonio Scuri

From: tiff-bounces@... [mailto:tiff-bounces@...] On Behalf Of Gene Amtower
Sent: sexta-feira, 11 de dezembro de 2009 17:11
To: Steve Mills
Cc: tiff list
Subject: Re: [Tiff] YCbCr

Well, I was following this thread thinking I might learn something from the discussion, but somewhere along the way, it went off the tracks for me!

This last post confuses me because the original post mentioned that the TIFFTAG_PHOTOMETRIC tag was reporting YCbCr, where as this one suggests the image is in CMYK. Are we still talking about the original image from the first post? If so, why is there this apparent contradiction between the first and last post?

All of this info got me to reading on the web, and I see that YCbCr is a colorspace primarily used for signal transmission to reduce the amount of transmission bandwidth, hence the downsampling of the Cb and Cr components because they carry less data. Info on Wikipedia provides conversion formulas to/from RGB that depend on conversion factors that are not universal. From these formulas, a red pixel ends up generating a value in each of the Y, Cb, and Cr pixel values because "Y" carries the primary luminance info, while Cb and Cr only communicate a "difference" value for the Red and Blue components. It also suggested to me that you can't really view YCbCr values directly because they are calculated values used in data transmission that aren't directly related to the RGB world that we live in. From what I can tell, YCbCr and CMYK are completely different colorspaces for either signal transmission or printing (respectively), and converting from one to the other actually involves an intermediate pass through the RGB space, with both conversions involving variable formulas that depend on hardware-dependent and implementation-dependent mapping algorithms. It sounds like there is NO universal way to convert between these three representations. There's also the RGBA colorspace that includes transparency info in the Alpha A channel - does this impact YCbCr and CMYK representations at all, or is this impossible in these alternate colorspaces?

If anyone cares to enlighten me, can you help me understand this discussion on LibTiff handling of RGB, CMYK, and YCbCr without going into a long dissertation on images, color spaces, TVs, and printing processes? Is it possible to store YCbCr image data in a TIFF file? If so, are there mechanisms built into LibTiff to convert it automatically to RGB for viewing, or is that up to the primary application to perform necessary conversions to the viewable world? With the inherent variability of YCbCr color values, how would Steve even know if he was getting valid YCbCr data when reading the values from the image file?

I guess a quick summary of the colorspace capabilities in LibTiff and the Tiff standard in general would be really helpful to the uninitiated among us. I know there's a lot of vagueness in the TIFF standard, but explain as much as possible, please!

Thanks,

Gene

On Fri, 2009-12-11 at 10:58 -0600, Steve Mills wrote:

On Dec 11, 2009, at 10:15:39, Antonio Scuri wrote:

>  As Joris pointed out, you actually get Y Cb and Cr planes. So the first

> component, that looks red, is infact Y, and indeed has the correct size. But

> the Cb and Cr planes are downsampled. At least this is what I got here.

What if the file is jpeg-compressed cmyk? The TIFFTAG_PHOTOMETRIC tag says it's cmyk, not YCbCr. I expect cmyk data in this case, but it sure doesn't look correct.

_________________________________________________________

Steve Mills                              Me: 952-401-6255

Senior Software Architect                         MultiAd

smills@...                       www.multi-ad.com

_______________________________________________

Tiff mailing list: Tiff@...

http://lists.maptools.org/mailman/listinfo/tiff

http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-11T12:04:35Z

On Fri, 2009-12-11 at 13:22 -0600, Steve Mills wrote:

On Dec 11, 2009, at 13:11:15, Gene Amtower wrote:

> This last post confuses me because the original post mentioned that the TIFFTAG_PHOTOMETRIC tag was reporting YCbCr, where as this one suggests the image is in CMYK.  Are we still talking about the original image from the first post?  If so, why is there this apparent contradiction between the first and last post?

The original file was in fact YCbCr. The same tester then tried creating a cmyk file and saving that as a jpeg-compressed tiff. That's the one that comes out as cmyk, but displays wrong. To further confuse you, it's only wrong if it was saved in Photoshop CS3. Save it in CS1 or CS4 and it's correct.

Do the message for this list show up in the actual sent order? They do NOT for me, so it's easy to get confused about who's talking about what when.

Yes, I've been seeing them in order, and I now see in your third message that you changed to another file in CMYK! I guess I was looking for your confirmation that you saw the right data in the YCbCr file, and I missed that the discussion switched to the CMYK file because the subject line mentioned YCbCr.

I'll try to keep up with the big dogs from now on - or maybe I just need to stay on the porch!

Thanks,

Gene
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: JPEG compression in 3.9.2

2009-12-11T11:45:26Z

Richard,

I'm finding a lot of problems with JPEG support in libtiff... beginning
with 3.7.3 all the way to 4.0.0 (including 3.9.2)... and I don't think
that I'm doing anything particularly fancy (i.e. simply raw-writing a
JPEG-compressed TIFF forces a JPEGTABLES tag to be written).

In the next few weeks I'll come back to the list with a set of code
problems that I encountered ... I hope to start up (or maybe add-to) a
discussion about resolving those problems. (i.e. I'll have a patch, but
it will be a "hack" - someone who knows how it is supposed to be will
have to sign off on my fix approach or will need to rework things to
address the problem).

Thanks,

Lee.

Richard Nolde wrote:

> Something definitely changed in the 3.9.X JPEG code. I found that some
> of the old libtiffpic sample images cause segfaults for tiffcp, and I
> had to do some workarounds to make tiffcrop handle them at all. Since
> OJEPG compression is now deprecated for writing images, I try to convert
> them to new JPEG compression on the fly but this is not working uniformly.
> Zackthecat and Smalliz come out at half the height they should and the
> colors are way off. The values are probably the subsampled data rather
> than the upsampled data and/or the Cb and Cr only. Quad-jpeg comes out
> OK in my working copy but looks more like a grayscale luminance map in
> the distributed copy. I'm also getting strange results, ie, several
> stripes of color but no image for ycbcr-cat.tif.
>
> tiffcp can no longer handle zackthecat.tif at all and smallliz aborts
> with a complaint about OJPEG not supported for writing (which is why I
> try to convert to new JPEG on the fly in tiffcrop).
>
> I use TIFFReadEncodedStrip/Tile to avoid issues with compression
> algorithms that do not allow random seeks and I found that the
> calculated strip sizes are smaller than you would get by computing
> length * width * spp *bps/8 so I override that when I allocate buffers
> and ask for more data on the read. For the problem images, it only
> brings back the number of bytes that it calculated, which suggests it is
> bringing back the down sampled data without up sampling it for YCbCr.
>
> I fiddled with the JPEG_COLORMODE pseudo-tag at various points when
> writing out the data, not at load time, but it didn't solve the problem
> for these images.
>
> Richard Nolde
> TIffcrop author
>
> Date: Fri, 11 Dec 2009 14:15:39 -0200
>
>> From: "Antonio Scuri"<scuri@...>
>> Subject: Re: [Tiff] YCbCr
>> To: "'Steve Mills'"<smills@...>, "'tiff list'"
>>
>> As Joris pointed out, you actually get Y Cb and Cr planes. So the first
>> component, that looks red, is infact Y, and indeed has the correct size. But
>> the Cb and Cr planes are downsampled. At least this is what I got here.
>>
>> I use libtiff 3.8.2. I have to comment the code at "tiff_jpeg.c" that does
>> "downsampled_output = TRUE" and add "tif->tif_flags |= TIFF_UPSAMPLED;" so I
>> got all YCbCr planes at normal resolution.
>>
>> When I upgraded to libtiff 3.9.2 my code stop working. Now almost all the
>> samples I have with YCbCr and JPEG compression fail to read. This includes
>> the known quad-jpeg, smallliz and zackthecat.
>>
>> It is the same for libjpeg6 or libjpeg7.
>>
>> I have to investigate the changes between 3.8 and 3.9 to see if I'm not
>> missing anything.
>>
>> BTW, I do use TIFFReadScanline and TIFFReadTile to read the image data.
>> I'm not using the high level functions.
>>
>> Best,
>> scuri
>>
>>
>
> _______________________________________________
> Tiff mailing list: Tiff@...
> http://lists.maptools.org/mailman/listinfo/tiff
> http://www.remotesensing.org/libtiff/
>
>

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: JPEG compression in 3.9.2

2009-12-11T11:37:50Z

This might be loosely related to the issues raised in the latest posts,
but in case it might be usefull,
you can have a look at the following ticket :

2009-12-03 Frank Warmerdam <warmerdam@...>

* libtiff/tif_jpeg.c: Fix a couple of issues that trigger failures in
some cases when using TIFFReadScanline() with JPEG compressed
subsampled ycbcr images.
http://bugzilla.maptools.org/show_bug.cgi?id=1936

Richard Nolde a écrit :

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-11T11:22:24Z

On Dec 11, 2009, at 13:11:15, Gene Amtower wrote:

> This last post confuses me because the original post mentioned that the TIFFTAG_PHOTOMETRIC tag was reporting YCbCr, where as this one suggests the image is in CMYK. Are we still talking about the original image from the first post? If so, why is there this apparent contradiction between the first and last post?

The original file was in fact YCbCr. The same tester then tried creating a cmyk file and saving that as a jpeg-compressed tiff. That's the one that comes out as cmyk, but displays wrong. To further confuse you, it's only wrong if it was saved in Photoshop CS3. Save it in CS1 or CS4 and it's correct.

Do the message for this list show up in the actual sent order? They do NOT for me, so it's easy to get confused about who's talking about what when.

_________________________________________________________
Steve Mills Me: 952-401-6255
Senior Software Architect MultiAd
smills@... www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-11T11:11:15Z

On Dec 11, 2009, at 10:15:39, Antonio Scuri wrote:

>  As Joris pointed out, you actually get Y Cb and Cr planes. So the first
> component, that looks red, is infact Y, and indeed has the correct size. But
> the Cb and Cr planes are downsampled. At least this is what I got here.

What if the file is jpeg-compressed cmyk? The TIFFTAG_PHOTOMETRIC tag says it's cmyk, not YCbCr. I expect cmyk data in this case, but it sure doesn't look correct.

_________________________________________________________
Steve Mills                              Me: 952-401-6255
Senior Software Architect                         MultiAd
smills@...                       www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

JPEG compression in 3.9.2

2009-12-11T10:49:44Z

Something definitely changed in the 3.9.X JPEG code. I found that some
of the old libtiffpic sample images cause segfaults for tiffcp, and I
had to do some workarounds to make tiffcrop handle them at all. Since
OJEPG compression is now deprecated for writing images, I try to convert
them to new JPEG compression on the fly but this is not working uniformly.
Zackthecat and Smalliz come out at half the height they should and the
colors are way off. The values are probably the subsampled data rather
than the upsampled data and/or the Cb and Cr only. Quad-jpeg comes out
OK in my working copy but looks more like a grayscale luminance map in
the distributed copy. I'm also getting strange results, ie, several
stripes of color but no image for ycbcr-cat.tif.

tiffcp can no longer handle zackthecat.tif at all and smallliz aborts
with a complaint about OJPEG not supported for writing (which is why I
try to convert to new JPEG on the fly in tiffcrop).

I use TIFFReadEncodedStrip/Tile to avoid issues with compression
algorithms that do not allow random seeks and I found that the
calculated strip sizes are smaller than you would get by computing
length * width * spp *bps/8 so I override that when I allocate buffers
and ask for more data on the read. For the problem images, it only
brings back the number of bytes that it calculated, which suggests it is
bringing back the down sampled data without up sampling it for YCbCr.

I fiddled with the JPEG_COLORMODE pseudo-tag at various points when
writing out the data, not at load time, but it didn't solve the problem
for these images.

Richard Nolde
TIffcrop author

Date: Fri, 11 Dec 2009 14:15:39 -0200

> From: "Antonio Scuri"<scuri@...>
> Subject: Re: [Tiff] YCbCr
> To: "'Steve Mills'"<smills@...>, "'tiff list'"
>
> As Joris pointed out, you actually get Y Cb and Cr planes. So the first
> component, that looks red, is infact Y, and indeed has the correct size. But
> the Cb and Cr planes are downsampled. At least this is what I got here.
>
> I use libtiff 3.8.2. I have to comment the code at "tiff_jpeg.c" that does
> "downsampled_output = TRUE" and add "tif->tif_flags |= TIFF_UPSAMPLED;" so I
> got all YCbCr planes at normal resolution.
>
> When I upgraded to libtiff 3.9.2 my code stop working. Now almost all the
> samples I have with YCbCr and JPEG compression fail to read. This includes
> the known quad-jpeg, smallliz and zackthecat.
>
> It is the same for libjpeg6 or libjpeg7.
>
> I have to investigate the changes between 3.8 and 3.9 to see if I'm not
> missing anything.
>
> BTW, I do use TIFFReadScanline and TIFFReadTile to read the image data.
> I'm not using the high level functions.
>
> Best,
> scuri
>

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-11T10:37:24Z

I don't think so. Because the logic will set downsampled_output to TRUE
only if h_sampling!=1, and that happen only for YCBCR.

If you want to debug the code, add a breakpoint at the line where
"downsampled_output = FALSE;" in "tif_jpeg.c". If it sets
"downsampled_output = TRUE;" then the output is donwsampled.

If you send me a sample, I can test your image here if I can read it. The
test I have supports CMYK images.

Best,
scuri

> -----Original Message-----
> From: tiff-bounces@... [mailto:tiff-
> bounces@...] On Behalf Of Steve Mills
> Sent: sexta-feira, 11 de dezembro de 2009 14:58
> To: tiff list
> Subject: Re: [Tiff] YCbCr
>
> On Dec 11, 2009, at 10:15:39, Antonio Scuri wrote:
>
> > As Joris pointed out, you actually get Y Cb and Cr planes. So the
> first
> > component, that looks red, is infact Y, and indeed has the correct
> size. But
> > the Cb and Cr planes are downsampled. At least this is what I got
> here.
>
> What if the file is jpeg-compressed cmyk? The TIFFTAG_PHOTOMETRIC tag
> says it's cmyk, not YCbCr. I expect cmyk data in this case, but it sure
> doesn't look correct.
>
> _________________________________________________________
> Steve Mills Me: 952-401-6255
> Senior Software Architect MultiAd
> smills@... www.multi-ad.com
>
>
> _______________________________________________
> Tiff mailing list: Tiff@...
> http://lists.maptools.org/mailman/listinfo/tiff
> http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-11T08:58:24Z

On Dec 11, 2009, at 10:15:39, Antonio Scuri wrote:

> As Joris pointed out, you actually get Y Cb and Cr planes. So the first
> component, that looks red, is infact Y, and indeed has the correct size. But
> the Cb and Cr planes are downsampled. At least this is what I got here.

What if the file is jpeg-compressed cmyk? The TIFFTAG_PHOTOMETRIC tag says it's cmyk, not YCbCr. I expect cmyk data in this case, but it sure doesn't look correct.

_________________________________________________________
Steve Mills Me: 952-401-6255
Senior Software Architect MultiAd
smills@... www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-11T08:15:39Z

As Joris pointed out, you actually get Y Cb and Cr planes. So the first
component, that looks red, is infact Y, and indeed has the correct size. But
the Cb and Cr planes are downsampled. At least this is what I got here.

I use libtiff 3.8.2. I have to comment the code at "tiff_jpeg.c" that does
"downsampled_output = TRUE" and add "tif->tif_flags |= TIFF_UPSAMPLED;" so I
got all YCbCr planes at normal resolution.

When I upgraded to libtiff 3.9.2 my code stop working. Now almost all the
samples I have with YCbCr and JPEG compression fail to read. This includes
the known quad-jpeg, smallliz and zackthecat.

It is the same for libjpeg6 or libjpeg7.

I have to investigate the changes between 3.8 and 3.9 to see if I'm not
missing anything.

BTW, I do use TIFFReadScanline and TIFFReadTile to read the image data.
I'm not using the high level functions.

Best,
scuri

> -----Original Message-----
> From: tiff-bounces@... [mailto:tiff-
> bounces@...] On Behalf Of Steve Mills
> Sent: quinta-feira, 10 de dezembro de 2009 18:31
> To: tiff list
> Subject: Re: [Tiff] YCbCr
>
> On Dec 10, 2009, at 14:21:08, Antonio Scuri wrote:
>
> > When loading YCbCr colorspace from JPEG compressed images I had a
> problem
> > with the size of the returned images. The logic in "tiff_jpeg.c"
> returns a
> > downsampled image. I fixed it changing the code so it returns a
> normal size
> > image. Let me know if you are interested.
>
> In what way was it downsamples? Can you give an example? The one test
> file I just tried looks like the correct size (it's an all red image).
> We're asking for the tags we're interested in and reading it row by
> row. Is this different from how you're loading tiff's?
>
> _________________________________________________________
> Steve Mills Me: 952-401-6255
> Senior Software Architect MultiAd
> smills@... www.multi-ad.com
>
>
> _______________________________________________
> Tiff mailing list: Tiff@...
> http://lists.maptools.org/mailman/listinfo/tiff
> http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-10T14:31:54Z

I used the dump function to see what the differences between the jpeg-compressed cmyk files might be. Here's the output for the file saved in Photoshop CS3, the one that produces incorrect results:

TIFFSetField: dummy: Unknown pseudo-tag 65538.
Magic: 0x4d4d <big-endian> Version: 0x2a
Directory 0: offset 8 (0x8) next 0 (0)
SubFileType (254) LONG (4) 1<0>
ImageWidth (256) SHORT (3) 1<4>
ImageLength (257) SHORT (3) 1<4>
BitsPerSample (258) SHORT (3) 4<8 8 8 8>
Compression (259) SHORT (3) 1<7>
Photometric (262) SHORT (3) 1<5>
StripOffsets (273) LONG (4) 1<25360>
Orientation (274) SHORT (3) 1<1>
SamplesPerPixel (277) SHORT (3) 1<4>
RowsPerStrip (278) SHORT (3) 1<4>
StripByteCounts (279) LONG (4) 1<128>
XResolution (282) RATIONAL (5) 1<72>
YResolution (283) RATIONAL (5) 1<72>
PlanarConfig (284) SHORT (3) 1<1>
ResolutionUnit (296) SHORT (3) 1<2>
Software (305) ASCII (2) 30<Adobe Photoshop CS3 Maci ...>
DateTime (306) ASCII (2) 20<2009:12:10 15:08:12\0>
JPEGTables (347) UNDEFINED (7) 558<0xff 0xd8 0xff 0xdb 00 0x84 00 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x3 0x2 0x2 0x2 0x3 0x4 0x3 ...>
700 (0x2bc) BYTE (1) 15890<0x3c 0x3f 0x78 0x70 0x61 0x63 0x6b 0x65 0x74 0x20 0x62 0x65 0x67 0x69 0x6e 0x3d 0x22 0xef 0xbb 0xbf 0x22 0x20 0x69 0x64 ...>
33723 (0x83bb) LONG (4) 2<469893120 33554432>
34377 (0x8649) BYTE (1) 8552<0x38 0x42 0x49 0x4d 0x4 0x4 00 00 00 00 00 0x7 0x1c 0x2 00 00 0x2 00 00 00 0x38 0x42 0x49 0x4d ...>
34665 (0x8769) LONG (4) 1<25488>

And here's the file from CS4, which produces correct results:

TIFFSetField: dummy: Unknown pseudo-tag 65538.
Magic: 0x4d4d <big-endian> Version: 0x2a
Directory 0: offset 8 (0x8) next 0 (0)
SubFileType (254) LONG (4) 1<0>
ImageWidth (256) SHORT (3) 1<4>
ImageLength (257) SHORT (3) 1<4>
BitsPerSample (258) SHORT (3) 4<8 8 8 8>
Compression (259) SHORT (3) 1<7>
Photometric (262) SHORT (3) 1<5>
StripOffsets (273) LONG (4) 1<18814>
Orientation (274) SHORT (3) 1<1>
SamplesPerPixel (277) SHORT (3) 1<4>
RowsPerStrip (278) SHORT (3) 1<4>
StripByteCounts (279) LONG (4) 1<117>
XResolution (282) RATIONAL (5) 1<72>
YResolution (283) RATIONAL (5) 1<72>
PlanarConfig (284) SHORT (3) 1<1>
ResolutionUnit (296) SHORT (3) 1<2>
Software (305) ASCII (2) 30<Adobe Photoshop CS4 Maci ...>
DateTime (306) ASCII (2) 20<2009:12:10 14:35:21\0>
JPEGTables (347) UNDEFINED (7) 285<0xff 0xd8 0xff 0xdb 00 0x43 00 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x2 0x3 0x2 0x2 0x2 0x3 0x4 0x3 ...>
700 (0x2bc) BYTE (1) 15863<0x3c 0x3f 0x78 0x70 0x61 0x63 0x6b 0x65 0x74 0x20 0x62 0x65 0x67 0x69 0x6e 0x3d 0x22 0xef 0xbb 0xbf 0x22 0x20 0x69 0x64 ...>
34377 (0x8649) BYTE (1) 2324<0x38 0x42 0x49 0x4d 0x4 0x25 00 00 00 00 00 0x10 00 00 00 00 00 00 00 00 00 00 00 00 ...>
34665 (0x8769) LONG (4) 1<18932>

The StripOffsets and StripByteCounts are different, as are the JPEGTables. I certainly don't understand any of that, but shouldn't we expect the same row data for each row read?

_________________________________________________________
Steve Mills Me: 952-401-6255
Senior Software Architect MultiAd
smills@... www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-10T12:55:30Z

On Dec 10, 2009, at 14:32:14, Joris Van Damme (AWare Systems) wrote:

> It does, in the sense that it is LibTiff's wrong way to try and support the result that you're after in this particular case. Do thorough testing, and if it works for you, it works for you.
>
> So, what you were getting was not channel-order-inversed RGB, but YCbCr, actually. And that is what the Photometric said you'd get. Only if the Photometric is YCbCr, and the compression is JPEG, can you suddenly turn this into RGB by using this magical out-of-the-blue pseudo-tag. But you'll have to remember to have your code thoroughly check for these conditions, and use the magical pseudo-tag.

So you're saying I can NOT just ALWAYS set this tag? I tried setting it only after reading had begin and we were reading tags to get color space, pixel size, etc, but it did not work. Moving the tag-setting to right after TIFFClientOpen made it work. Are you saying that we will get incorrect results for other cases that are NOT YCbCr jpeg-compressed?

The same tester just sent me a cymk jpeg-compressed tiff (PC byte order, created in Photoshop CS3). It is showing up wrong. Each pixel should be cmyk 0%,99%,99%,0% (that's what I see in Photoshop - the entire file is red, not a great test file, but that's what he sent), but the hex values I see for each pixel are 4d,55,ff,00 after we call TIFFReadScanline.

I tried making my own test document in Photoshop CS4, but my files are showing up correctly. They're much smaller and contain pixels colored cyan, magenta, yellow, and black.

_________________________________________________________
Steve Mills Me: 952-401-6255
Senior Software Architect MultiAd
smills@... www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-10T12:32:14Z

Steve,

>> Are you using the JPEG_COLORMODE pseudo-tag?
>
> No. But I just added this call right after our call to TIFFClientOpen and
> it worked!
>
> TIFFSetField(m_TIFF, TIFFTAG_JPEGCOLORMODE, JPEGCOLORMODE_RGB);
>
> Is that the correct way to handle this?

It does, in the sense that it is LibTiff's wrong way to try and support the
result that you're after in this particular case. Do thorough testing, and
if it works for you, it works for you.

So, what you were getting was not channel-order-inversed RGB, but YCbCr,
actually. And that is what the Photometric said you'd get. Only if the
Photometric is YCbCr, and the compression is JPEG, can you suddenly turn
this into RGB by using this magical out-of-the-blue pseudo-tag. But you'll
have to remember to have your code thoroughly check for these conditions,
and use the magical pseudo-tag.

Best regards,

Joris

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-10T12:31:02Z

On Dec 10, 2009, at 14:21:08, Antonio Scuri wrote:

> When loading YCbCr colorspace from JPEG compressed images I had a problem
> with the size of the returned images. The logic in "tiff_jpeg.c" returns a
> downsampled image. I fixed it changing the code so it returns a normal size
> image. Let me know if you are interested.

In what way was it downsamples? Can you give an example? The one test file I just tried looks like the correct size (it's an all red image). We're asking for the tags we're interested in and reading it row by row. Is this different from how you're loading tiff's?

_________________________________________________________
Steve Mills Me: 952-401-6255
Senior Software Architect MultiAd
smills@... www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-10T12:21:08Z

When loading YCbCr colorspace from JPEG compressed images I had a problem
with the size of the returned images. The logic in "tiff_jpeg.c" returns a
downsampled image. I fixed it changing the code so it returns a normal size
image. Let me know if you are interested.

Best Regards,
Antonio Scuri

> -----Original Message-----
> From: tiff-bounces@... [mailto:tiff-
> bounces@...] On Behalf Of Steve Mills
> Sent: quinta-feira, 10 de dezembro de 2009 17:37
> To: tiff list
> Subject: Re: [Tiff] YCbCr
>
> On Dec 10, 2009, at 13:19:35, Joris Van Damme (AWare Systems) wrote:
>
> > Are you using the JPEG_COLORMODE pseudo-tag?
>
> No. But I just added this call right after our call to TIFFClientOpen
> and it worked!
>
> TIFFSetField(m_TIFF, TIFFTAG_JPEGCOLORMODE, JPEGCOLORMODE_RGB);
>
> Is that the correct way to handle this?
>
> _________________________________________________________
> Steve Mills Me: 952-401-6255
> Senior Software Architect MultiAd
> smills@... www.multi-ad.com
>
>
> _______________________________________________
> Tiff mailing list: Tiff@...
> http://lists.maptools.org/mailman/listinfo/tiff
> http://www.remotesensing.org/libtiff/

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-10T11:37:16Z

On Dec 10, 2009, at 13:19:35, Joris Van Damme (AWare Systems) wrote:

> Are you using the JPEG_COLORMODE pseudo-tag?

No. But I just added this call right after our call to TIFFClientOpen and it worked!

TIFFSetField(m_TIFF, TIFFTAG_JPEGCOLORMODE, JPEGCOLORMODE_RGB);

Is that the correct way to handle this?

_________________________________________________________
Steve Mills Me: 952-401-6255
Senior Software Architect MultiAd
smills@... www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: YCbCr

2009-12-10T11:19:35Z

Steve,

> What should I expect to be get from a call to TIFFReadScanline when the
> TIFFTAG_PHOTOMETRIC tag is PHOTOMETRIC_YCBCR? It appears that we're
> getting rgb, but in the wrong channel order (should be rgb, but is bgr or
> vise versa). I only have one test document that is supposed to be solid
> red. It's coming out blue. I don't know how the file was made or how to
> make another one, otherwise I'd use a file with more colors so I knew for
> sure what was going on.

Are you using the JPEG_COLORMODE pseudo-tag?

It may also be related to a define in LibJpeg, that specifies the channel
order of RGB. It may be that in your own copy of LibJpeg, that define is
tuned, whilst LibTiff does not expect that. I don't remember from the top of
my head the name of the define, but it should be straightforward to find it
in a LibJpeg config file.

Best regards,

Joris

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

YCbCr

2009-12-10T10:28:31Z

What should I expect to be get from a call to TIFFReadScanline when the TIFFTAG_PHOTOMETRIC tag is PHOTOMETRIC_YCBCR? It appears that we're getting rgb, but in the wrong channel order (should be rgb, but is bgr or vise versa). I only have one test document that is supposed to be solid red. It's coming out blue. I don't know how the file was made or how to make another one, otherwise I'd use a file with more colors so I knew for sure what was going on.

_________________________________________________________
Steve Mills Me: 952-401-6255
Senior Software Architect MultiAd
smills@... www.multi-ad.com

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: Problems with multiple strip G4 to single strip G4 tiff.

2009-12-09T00:11:44Z

On Wed, 9 Dec 2009 02:47:52 -0500
Phillip Wiles <pwiles38@...> wrote:

> I am 99.9% certain that it is my code that is causing the crash. I should
> have been clearer on that.

Ok, I thought you had checked that :)

> I downloaded GDB and it looks interesting, but it's likely overkill for this
> problem and likely over my head.

Nah, it's dead simple. Let your program write a coredump (perhaps
setting "ulimit -c unlimited" before), then run "gdb
path/to/your/program xyz.core" and do a "bt" for backtrace. If you
compiled your code with symbols (-g option of gcc) you'll see on which
line it was.

> I'm not asking that anyone do the work for me. I'm looking for direction (or
> explanation as to what I'm doing wrong) to the correct way to save the image
> a single strip, when the original image was multiple strip.

I thought you wanted to do it the other way round? Anyway, here's a
code snippet from my libtiff usage, omitting the setting of all the
other necessary tags:

/* write a striped image */
rowsperstrip = TIFFDefaultStripSize(hTIFF, fmt->image_length);
if (rowsperstrip > fmt->image_length) {
rowsperstrip = fmt->image_length;
LOG(LINFO,(_fun,"clamping to image length: %u\n",
rowsperstrip));
}
rc = TIFFSetField(hTIFF, TIFFTAG_ROWSPERSTRIP, rowsperstrip);
for (y = 0; y < fmt->image_length; y++) {
tdata_t row = (tdata_t)(tiff->mem.data + y * tiff->stride + x);
rc = TIFFWriteScanline(hTIFF, row, y, 0);
if (rc != 1) {
LOG(LERROR,(_fun,"TIFFWriteScanline(%p,%p,%d,%d) "
"failed (%d)\n", tiff, row, y, 0, rc));
goto abort;
}
}
...

HTH,
Juergen
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: Problems with multiple strip G4 to single strip G4 tiff.

2009-12-08T23:47:52Z

On Tue, Dec 8, 2009 at 1:11 PM, Juergen Buchmueller wrote:

> But when I try to write an image that started out as multiple strips it crashes.

Did you verify it crashes in libtiff, not in your code? I.e. use a
coredump and GDB or the debugger on Windows.

Juergen

I am 99.9% certain that it is my code that is causing the crash. I should have been clearer on that.

I downloaded GDB and it looks interesting, but it's likely overkill for this problem and likely over my head.

I'm not asking that anyone do the work for me. I'm looking for direction (or explanation as to what I'm doing wrong) to the correct way to save the image a single strip, when the original image was multiple strip. I have searched the mail list archive at length and searched on Google. But unfortunately there really isn't that much in the way of examples (the IBM articles, “Using LibTIFF” and tons of reposted LibTIFF manuals). I really hoped and tried to avoid posting to the mail list, to get help. The only other opinion really left is to dig around in the source code of tiffcp and see if I can get the answer/understanding there.

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: Problems with multiple strip G4 to single strip G4 tiff.

2009-12-08T10:11:24Z

On Mon, 7 Dec 2009 11:06:24 -0500
Phillip Wiles <pwiles38@...> wrote:

> But when I try to write an image that started out as multiple strips it crashes.

Did you verify it crashes in libtiff, not in your code? I.e. use a
coredump and GDB or the debugger on Windows.

Juergen
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: Problems with multiple strip G4 to single strip G4 tiff.

2009-12-08T09:53:13Z

Phillip wrote:

If I use this below to write a tiff that started as single strip tiff, there is no problem. But when I try to write an image that started out
as multiple strips it crashes. Do I "need" to save the image multiple strips? And if I do, can someone help in the right direction (I have experimented to try and force multiple strips with a group 4 image, with no luck).

TIFFSetField(out,TIFFTAG_PLANARCONFIG,PLANARCONFIG_CONTIG);
if(mbCompression){TIFFSetField(tif, TIFFTAG_COMPRESSION, mTiffCompression);};
if(mbSubfileType){TIFFSetField(tif, TIFFTAG_SUBFILETYPE, muliSubfileType);};

if(mbResolutionUnit){TIFFSetField(tif, TIFFTAG_RESOLUTIONUNIT, musiResolutionUnit);};
if(mbXResolution){TIFFSetField(tif, TIFFTAG_XRESOLUTION, mfTiffXresolution);};
if(mbYResolution){TIFFSetField(tif, TIFFTAG_YRESOLUTION, mfTiffYresolution);};

if(mbImageWidth){TIFFSetField(tif, TIFFTAG_IMAGEWIDTH, muliImageWidth);};
if(mbImageLength){TIFFSetField(tif, TIFFTAG_IMAGELENGTH, muliImageHeight);};

if(mbPhotometricInterpretation){TIFFSetField(tif, TIFFTAG_PHOTOMETRIC ,musiPhotoMetric);};
if(mbBitsPerSample){TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE ,musiBitPerSample);};
if(mbFillOrder){TIFFSetField(tif, TIFFTAG_FILLORDER, musiFillOrder);};
if(mbSamplesPerPixel){TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, musiSamplePerPixel);};

if(mbuf){TIFFWriteEncodedStrip(tif, 0, mbuf, (muliImageWidth * muliImageHeight) / 8);};

//My attempt at creating multiple strip.
//if(mbRowsPerStrip){TIFFSetField(tif, TIFFTAG_ROWSPERSTRIP, muliRowsPerStrip);};

//tdata_t outbuf;
//char * p = (char*)mbuf;

//for(tstrip_t counter = 0; counter < mstrip; counter++){
// outbuf = (void*)&p[counter * (muliImageWidth * muliRowsPerStrip)/8];
// TIFFWriteEncodedStrip(out, counter, outbuf, (muliImageWidth * muliRowsPerStrip)/8);
//};

I have changed the save to:

if(mbuf){TIFFWriteEncodedStrip(out, 0, mbuf, mBufferSize);};

This gives me a black (or all white depending on program used to read it) image as output from a multiple strip original image. A single strip original image still works without problems. Both images are Group 4 (I can post the data from AsTiffTagViewer, if that will help any).

This is my read (which is for the most part the same as the one shown in the "Using the TIFF Library")
mstrip = TIFFNumberOfStrips(tif);
mBufferSize = TIFFStripSize(tif);
mbuf = _TIFFmalloc(mBufferSize);
mstrip = TIFFNumberOfStrips(tif);
for (tstrip_t tempstrip = 0; tempstrip < mstrip; tempstrip++){
TIFFReadEncodedStrip(tif, tempstrip, mbuf, (tsize_t) -1);
};

I'm using LibTIFF 3.8.2

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: TIFFTAG_RICHTIFFIPTC vs my life...

2009-12-07T14:14:11Z

I feel your pain. When I load a TIFF that was written by Photoshop, I get the warning, "wrong data type 7 for "RichTIFFIPTC"; tag ignored". Have you found any way to deal with this?

Tom

Chris Losinger wrote:

and by OkToChangeTag, i meant _TIFFFindFieldInfo.
sorry. :)

problem remains the same, however.
-c

At 12:38 PM 9/24/2009, Chris Losinger wrote:

>..
> I can kindof see a way to do it - starting by teaching the 'wrong
>data type' check in TIFFReadDirectory to allow the exception; but
>then OkToChangeTag tells TIFFVSetField to use the built-in type for
>that tag (TIFF_LONG instead of TIFF_UNDEFINIED), and that means all
>the mallocs and memcpys which follow are too large by a factor of 4.
>
> Without teaching OkToChangeTag about the exception, and then finding
>the next snag, which could take forever, I was hoping to see that
>someone else had found a solid solution to the problem and would be
>willing to share...

----
Chris Losinger
losinger@earthlink.net
smallest@smalleranimals.com http://www.smalleranimals.com

_______________________________________________
Tiff mailing list: Tiff@lists.maptools.org
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: Full strip allocation in TIFFWriteScanline

2009-12-07T10:01:40Z

Lars Uebernickel wrote:

> Hello,
>
> TIFFWriteScanline always allocates memory for a complete strip. When
> writing large TIFF files in a single strip, this can lead to memory
> problems quickly.
>
> I am aware that the whole point of dividing the image data into strips
> is to be able to read and write in manageable chunks of memory.
> However, some applications (e.g. many fax applications) seem to have
> problems with TIFF images with more than a single strip.
>
> I am aware of libtiff's strip-based API, which is capable of writing
> strip data incrementally with TIFFWriteRawStrip. Unfortunately, it
> expects already compressed data and is therefore not making use of the
> built-in compressors. TIFFWriteEncodedStrip on the other hand
> compresses data prior to writing it, but it doesn't work incrementally,
> i.e. it expects the full data of each strip.
>
> I assume that this is because libtiff's implementation of the
> compression algorithms cannot operate on streamed data? I think it'd
> make sense to support this, as the data which needs to be buffered by
> most compression schemes should be substantially less (and never more)
> than the smallest strip size feasible for each algorithm.
>
> What are you general thoughts on changing the compression
> implementations in this regard? Do you think it is doable or am I
> overlooking something? Is somebody already working on it or planning
> to?

Lars,

I inspected the tif_packbits.c code (because it is one of the simplier
encoders) and it *appears* to me that it ought to support writing a scanline
at a time without buffering the whole output strip. I PackBitsEncode appears
to be structured to handle a scanline at a time and it calls TIFFFlushData1()
periodically - presumably to flush out the data compressed so far.

I haven't looked any further, and I must confess to some confusion about the
assumptions and expectations for encoders and decoders despite having fiddled
with them for years. But it seems to me that at least some encoders support
scanline oriented encoding and writing to disk. So if the fax encoders do not,
we can consider it a task local to the fax encoder to improve it to do the
same.

Of course, I may be missing lots of things.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@...
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Full strip allocation in TIFFWriteScanline

2009-12-07T08:47:27Z

Hello,

TIFFWriteScanline always allocates memory for a complete strip. When
writing large TIFF files in a single strip, this can lead to memory
problems quickly.

I am aware that the whole point of dividing the image data into strips
is to be able to read and write in manageable chunks of memory.
However, some applications (e.g. many fax applications) seem to have
problems with TIFF images with more than a single strip.

I am aware of libtiff's strip-based API, which is capable of writing
strip data incrementally with TIFFWriteRawStrip. Unfortunately, it
expects already compressed data and is therefore not making use of the
built-in compressors. TIFFWriteEncodedStrip on the other hand
compresses data prior to writing it, but it doesn't work incrementally,
i.e. it expects the full data of each strip.

I assume that this is because libtiff's implementation of the
compression algorithms cannot operate on streamed data? I think it'd
make sense to support this, as the data which needs to be buffered by
most compression schemes should be substantially less (and never more)
than the smallest strip size feasible for each algorithm.

What are you general thoughts on changing the compression
implementations in this regard? Do you think it is doable or am I
overlooking something? Is somebody already working on it or planning
to?

Lars

P.S.: A bit of context: We ran into these issues with ghostscript, after
porting all of its tiff output devices to use libtiff instead of writing
output files manually (using ghostscript's streamable implementation of
some of the compression schemes). In order to provide backwards
compatibility, the output files are written in a single strip.

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Problems with multiple strip G4 to single strip G4 tiff.

2009-12-07T08:06:24Z

If I use this below to write a tiff that started as single strip tiff, there is no problem. But when I try to write an image that started out
as multiple strips it crashes. Do I "need" to save the image multiple strips? And if I do, can someone help in the right direction (I have experimented to try and force multiple strips with a group 4 image, with no luck).

TIFFSetField(out,TIFFTAG_PLANARCONFIG,PLANARCONFIG_CONTIG);
if(mbCompression){TIFFSetField(tif, TIFFTAG_COMPRESSION, mTiffCompression);};
if(mbSubfileType){TIFFSetField(tif, TIFFTAG_SUBFILETYPE, muliSubfileType);};

if(mbResolutionUnit){TIFFSetField(tif, TIFFTAG_RESOLUTIONUNIT, musiResolutionUnit);};
if(mbXResolution){TIFFSetField(tif, TIFFTAG_XRESOLUTION, mfTiffXresolution);};
if(mbYResolution){TIFFSetField(tif, TIFFTAG_YRESOLUTION, mfTiffYresolution);};

if(mbImageWidth){TIFFSetField(tif, TIFFTAG_IMAGEWIDTH, muliImageWidth);};
if(mbImageLength){TIFFSetField(tif, TIFFTAG_IMAGELENGTH, muliImageHeight);};

if(mbPhotometricInterpretation){TIFFSetField(tif, TIFFTAG_PHOTOMETRIC ,musiPhotoMetric);};
if(mbBitsPerSample){TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE ,musiBitPerSample);};
if(mbFillOrder){TIFFSetField(tif, TIFFTAG_FILLORDER, musiFillOrder);};
if(mbSamplesPerPixel){TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, musiSamplePerPixel);};

if(mbuf){TIFFWriteEncodedStrip(tif, 0, mbuf, (muliImageWidth * muliImageHeight) / 8);};

//My attempt at creating multiple strip.
//if(mbRowsPerStrip){TIFFSetField(tif, TIFFTAG_ROWSPERSTRIP, muliRowsPerStrip);};

//tdata_t outbuf;
//char * p = (char*)mbuf;

//for(tstrip_t counter = 0; counter < mstrip; counter++){
// outbuf = (void*)&p[counter * (muliImageWidth * muliRowsPerStrip)/8];
// TIFFWriteEncodedStrip(out, counter, outbuf, (muliImageWidth * muliRowsPerStrip)/8);
//};

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: Adding tags to a broken(?) TIFF

2009-12-06T07:57:50Z

On Sun, 6 Dec 2009, Juergen Buchmueller wrote:

> FYI: I solved the problem by creating a new TIFF on the fly.
>
> It seems that libtiff doesn't handle extending a directory without
> overwriting data at all. If a directory size increases, libtiff would
> have to relocate the entire directory entry with all existing tags to
> e.g. the end of the file. It doesn't do that now, but writes the new
> tag to the file right after the end of the current directory, thereby
> destroying whatever is there.
>
> That's just my observation without looking into libtiff's source.

What version of libtiff are you using?

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: TIFFFlushData1 return code verification

2009-12-06T07:57:44Z

On 2009-12-06 17:56, Bob Friesenhahn wrote:

> On Sun, 6 Dec 2009, Evgeny Fraimovitch wrote:
>
>> There is a certain inconsistency in TIFFFlushData1 error code handling,
>> which can lead to unreported disk write errors. (such as when we are out
>> of disk space)
>>
>> For possibly related thread please see: *1994.10.21 15:50 "Problems
>> catching write errors in tiff library (v3.3beta021)", by Ben Griffin*
>> <http://www.asmail.be/msg0054980073.html>
>
> This email is from 1994. What version of libtiff are you working with?
>
> Bob

I know the date of this e-mail. (hence I cited the full date) However
the problematic code is found in libtiff 3.9.2. (The newest version)

Evgeny
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: TIFFFlushData1 return code verification

2009-12-06T07:56:29Z

On Sun, 6 Dec 2009, Evgeny Fraimovitch wrote:

> There is a certain inconsistency in TIFFFlushData1 error code handling,
> which can lead to unreported disk write errors. (such as when we are out
> of disk space)
>
> For possibly related thread please see: *1994.10.21 15:50 "Problems
> catching write errors in tiff library (v3.3beta021)", by Ben Griffin*
> <http://www.asmail.be/msg0054980073.html>

This email is from 1994. What version of libtiff are you working
with?

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

Re: Adding tags to a broken(?) TIFF

2009-12-06T07:02:46Z

FYI: I solved the problem by creating a new TIFF on the fly.

It seems that libtiff doesn't handle extending a directory without
overwriting data at all. If a directory size increases, libtiff would
have to relocate the entire directory entry with all existing tags to
e.g. the end of the file. It doesn't do that now, but writes the new
tag to the file right after the end of the current directory, thereby
destroying whatever is there.

That's just my observation without looking into libtiff's source.

Juergen
_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/

TIFFFlushData1 return code verification

2009-12-06T06:04:05Z

Good time of day,

There is a certain inconsistency in TIFFFlushData1 error code handling,
which can lead to unreported disk write errors. (such as when we are out
of disk space)

For possibly related thread please see: *1994.10.21 15:50 "Problems
catching write errors in tiff library (v3.3beta021)", by Ben Griffin*
<http://www.asmail.be/msg0054980073.html>

In short: the tif_dumpmode.c DumpModeEncode returns (-1) when
TIFFFlushData1 fails

if (tif->tif_rawcc >= tif->tif_rawdatasize &&
!TIFFFlushData1(tif))
return (-1);

however its users expect it to return zero or non-zero value.

LogLuvEncode24 and siblings does the same mistake.
as does PackBitsEncode

The fix is to replace 'return (-1)' to 'return 0' in such places.

Sincerely yours,
Evgeny
VisionMap Ltd

_______________________________________________
Tiff mailing list: Tiff@...
http://lists.maptools.org/mailman/listinfo/tiff
http://www.remotesensing.org/libtiff/