Re: Issues due to commit "Fix metrics on size request for scalable fonts."

[Peter Hurley-2]

Werner LEMBERG wrote:

> > We had report of regression in GTK markups rendering in Ubuntu
> > precise, (i.e <u>label</u> would render underlined), [...]
> >
> > We have reverted that commit in Ubuntu for now to workaround the
> > issue but I would like to know how to determine if that's a bug in
> > freetype or if other part of the stack (i.e gtk) would need to be
> > updated to deal with the new freetype?
>
> AFAIK, the problem is in gtk.  For years, FreeType had this metrics
> bug, and unfortunately the users got used to the appearance of far too
> widely spaced lines.  What FreeType now returns is what the font
> designer has had in mind while designing the font, and what can be
> found in the font.
>
> As has been stated in one of the Debian bug report comments, the
> amount of the vertical line spacing is something which *must* be
> configurable or explicitly set by the application.  Additionally, the
> underlying graphical library should provide a better default in case
> the font value is not satisfying (for whatever reason), for example,
> by multiplying the vertical distances by a fixed amount.  Just think
> of TeX which ignores a font's line spacing values completely,
> providing its own default (1.2 * design size).
>
> Until someone can prove that the patch is incorrect, I won't change it
> back.  Simply reverting it, as done by some distributions, is *not*
> the correct solution IMHO, since it hides the problem of gtk.

Hi Werner,

Let me apologize if this reply is not properly threaded. I had to thread
it manually.

Although I have no interest in proving the patch is incorrect, I do have
some relevant comments which may shed additional light on the
complications introduced by "fixing" this bug.

This is not strictly a vertical line spacing problem. The primary impact
of this fix is that the metrics are now integer-scaled rather than
fractionally scaled. (Indeed, this appears to be the reason the TT
driver overrides the default size_request.)

One outcome of integer scaling the metrics is that monospace fonts have
'excessive' space between glyphs. Unlike vertical line spacing, there is
no way to directly control the max_advance metric value from the
application layer -- only the font stack has access to this value.

Integer scaling *may* have been the font designer's requirement.
However, it could be argued that if the font designer was using FreeType
to proof the work, that the 'intention' was for the font to look as it
did before this fix.

Regards,
Peter Hurley

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Alexei Podtelezhnikov-2]

On Tue, Apr 3, 2012 at 9:05 AM, Peter Hurley <peter@...> wrote:

> Werner LEMBERG wrote:
>> > We had report of regression in GTK markups rendering in Ubuntu
>> > precise, (i.e <u>label</u> would render underlined), [...]
>> >
>>
>> AFAIK, the problem is in gtk.  For years, FreeType had this metrics
>> bug, and unfortunately the users got used to the appearance of far too
>> widely spaced lines.  What FreeType now returns is what the font
>> designer has had in mind while designing the font, and what can be
>> found in the font.
>>
>> As has been stated in one of the Debian bug report comments, the
>> amount of the vertical line spacing is something which *must* be
>> configurable or explicitly set by the application.  >
>
> One outcome of integer scaling the metrics is that monospace fonts have
> 'excessive' space between glyphs. Unlike vertical line spacing, there is
> no way to directly control the max_advance metric value from the
> application layer -- only the font stack has access to this value.

I'd like to see some images that show that. Please use ftstring. Also
please remember that freetype renders glyphs and truthfully returns
metrics. The text layout is not done in freetype.

>
> Integer scaling *may* have been the font designer's requirement.
> However, it could be argued that if the font designer was using FreeType
> to proof the work, that the 'intention' was for the font to look as it
> did before this fix.

This is highly unlikely and hypothetical. Freetype is trying to
improve the rendering all the time. Improving meens some changes in
the way glyphs look.

I personally always liked this particular change that made the line
spacing more consistent with the font size and improved the layout of
complex webpages where the font sizes is highly variable:
http://lists.nongnu.org/archive/html/freetype-devel/2011-07/msg00049.html

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Werner LEMBERG]

> This is not strictly a vertical line spacing problem. The primary
> impact of this fix is that the metrics are now integer-scaled rather
> than fractionally scaled. (Indeed, this appears to be the reason the
> TT driver overrides the default size_request.)

Yes.  This is what the TrueType standard mandates for bi-level and
grey-level glyphs.  Note that ClearType is *not* implemented in
FreeType yet!  As soon as this happens, the situation changes.  This
document

  http://www.microsoft.com/typography/cleartype/truetypecleartype.aspx

writes the following about the GETINFO bytecode instruction:

  Compatible Width ClearType (Selector bit 7, Return bit 14)

    This flag is requested by setting selector bit seven.  If the
    return bit fourteen is set, this indicates that the glyphs for
    this font size will be adjusted post hinting in order to return
    advance widths that are exactly the same as bi-level rendering.
    If the return bit fourteen is not set, then the advanced widths
    will be either returned as integers rounded to the nearest pixel
    or rounded to the nearest sixteenth of a pixel.  The former is
    referred to as natural widths ClearType and the later as sub-pixel
    positioned ClearType.

> Integer scaling *may* have been the font designer's requirement.

If you develop non-ClearType TTFs, this *is* a requirement.

> However, it could be argued that if the font designer was using
> FreeType to proof the work, that the 'intention' was for the font to
> look as it did before this fix.

Hmm.  A font designer who relies only on FreeType for proofing a font?


    Werner

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Werner LEMBERG]

> Yes.  This is what the TrueType standard mandates for bi-level and
> grey-level glyphs.  Note that ClearType is *not* implemented in
> FreeType yet!  As soon as this happens, the situation changes.

BTW, section 3.3.2 in the document below gives a nice description of
fractional advance widths and the problems related to it.

  http://www.beatstamm.com/typography/RTRCh3.htm#Sec30


      Werner

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Peter Hurley-2]

On Tue, 2012-04-03 at 12:01 -0400, Werner LEMBERG wrote:
> > Yes.  This is what the TrueType standard mandates for bi-level and
> > grey-level glyphs.  Note that ClearType is *not* implemented in
> > FreeType yet!  As soon as this happens, the situation changes.
>
> BTW, section 3.3.2 in the document below gives a nice description of
> fractional advance widths and the problems related to it.
>
>   http://www.beatstamm.com/typography/RTRCh3.htm#Sec30

Good source! Thanks for the link.

Regards,
Peter


_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Peter Hurley-2]

On Tue, 2012-04-03 at 11:35 -0400, Werner LEMBERG wrote:

> > This is not strictly a vertical line spacing problem. The primary
> > impact of this fix is that the metrics are now integer-scaled rather
> > than fractionally scaled. (Indeed, this appears to be the reason the
> > TT driver overrides the default size_request.)
>
> Yes.  This is what the TrueType standard mandates for bi-level and
> grey-level glyphs.  Note that ClearType is *not* implemented in
> FreeType yet!  As soon as this happens, the situation changes.  This
> document
>
>   http://www.microsoft.com/typography/cleartype/truetypecleartype.aspx
>
> writes the following about the GETINFO bytecode instruction:
>
>   Compatible Width ClearType (Selector bit 7, Return bit 14)
>
>     This flag is requested by setting selector bit seven.  If the
>     return bit fourteen is set, this indicates that the glyphs for
>     this font size will be adjusted post hinting in order to return
>     advance widths that are exactly the same as bi-level rendering.
>     If the return bit fourteen is not set, then the advanced widths
>     will be either returned as integers rounded to the nearest pixel
>     or rounded to the nearest sixteenth of a pixel.  The former is
>     referred to as natural widths ClearType and the later as sub-pixel
>     positioned ClearType.
>
> > Integer scaling *may* have been the font designer's requirement.
>
> If you develop non-ClearType TTFs, this *is* a requirement.

I think I wasn't clear enough here about integer-scaled vs. fractionally
scaled metrics. I'm not referring to "Fractional Advance Widths".

I'm referring to when the x-scale is forced to be an integer-multiple of
units/EM.

For example, let's suppose that I use FT_Set_Char_Size() to specify the
desired character width as 10.66 (really, 10 43/64). This generates a
size_request() to the TT driver, which the TT driver initially forwards
back to FT_Request_Metrics(). Assuming that units_per_EM == 2048, the
computed x_scale will be:

   x_scale = 10.66 * 2048 = 21856

This represents a fractional scaling value.
Let's say this produces a max_advance of 384 (or 6.0).

Now, if the TrueType font header indicates that integer-scaled values
are required (Flags & bit 3 -- see here
https://developer.apple.com/fonts/TTRefMan/RM06/Chap6head.html)
the TT driver recomputes x_scale:

  x_scale = 11 * 2048 = 22528

and then recomputes max_advance. This larger scalar produces a
max_advance of 448 (7.0). Note that the advance width is integral but
now larger by 1 pixel per glyph (for monospace fonts).

Before the fix, these computations did not affect the font metrics for
layout because the base metrics were not changed. Now they are.

Just to be clear here: there is no way for an application to control
this outcome.

> > However, it could be argued that if the font designer was using
> > FreeType to proof the work, that the 'intention' was for the font to
> > look as it did before this fix.
>
> Hmm.  A font designer who relies only on FreeType for proofing a font?

At first glance, perhaps that seems wrong, but some Linux distributions
are providing custom fonts specifically for the distribution. What else
would be the reference standard in this situation but FreeType?

My point here is not to argue for reverting this fix, but rather to
illustrate the impact at a higher level. Hopefully this collective
doesn't consider that to be immaterial.

Regards,
Peter Hurley

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Werner LEMBERG]

> I'm referring to when the x-scale is forced to be an integer-multiple of
> units/EM.

???  This is not the case.

> For example, let's suppose that I use FT_Set_Char_Size() to specify the
> desired character width as 10.66 (really, 10 43/64). This generates a
> size_request() to the TT driver, which the TT driver initially forwards
> back to FT_Request_Metrics(). Assuming that units_per_EM == 2048, the
> computed x_scale will be:
>
>    x_scale = 10.66 * 2048 = 21856
>
> This represents a fractional scaling value.
> Let's say this produces a max_advance of 384 (or 6.0).
>
> Now, if the TrueType font header indicates that integer-scaled values
> are required (Flags & bit 3 -- see here
> https://developer.apple.com/fonts/TTRefMan/RM06/Chap6head.html)
> the TT driver recomputes x_scale:
>
>   x_scale = 11 * 2048 = 22528
>
> and then recomputes max_advance. This larger scalar produces a
> max_advance of 448 (7.0). Note that the advance width is integral but
> now larger by 1 pixel per glyph (for monospace fonts).

Hmm.  You are probably talking about the pixel size, right?  The
formula to convert from points to pixels (per EM) is

  ppem = pt * dpi / 72

and this resulting ppem value must be an integer if bit 3 is set.  So
while you can request a non-integer value in `FT_Set_Char_Size', you
can only request integer values in `FT_Set_Pixel_Sizes'.  Of course,
if you are using a resolution of 72dpi, the character size equals the
pixel size.

> Before the fix, these computations did not affect the font metrics
> for layout because the base metrics were not changed.  Now they are.

Yes.  If you look into

  http://www.microsoft.com/typography/otspec/head.htm

it explicitly says

  Bit 3: Force ppem to integer values for all internal scaler math;
         may use fractional ppem sizes if this bit is clear;

which is a better description than in the Apple specification.  The
old behaviour in FreeType was a bug.

> Just to be clear here: there is no way for an application to control
> this outcome.

Yes.  BTW, can you provide images for your worst-case scenario?
Honestly spoken, it sounds quite artificial, since TrueType bytecode
instructions normally adjust a glyph's advance width to the right
value.  You might also read the following three FreeType bug reports:

  https://savannah.nongnu.org/bugs/?34156
  https://savannah.nongnu.org/bugs/?34507
  https://savannah.nongnu.org/bugs/?33939

>> Hmm.  A font designer who relies only on FreeType for proofing a
>> font?
>
> At first glance, perhaps that seems wrong, but some Linux
> distributions are providing custom fonts specifically for the
> distribution.  What else would be the reference standard in this
> situation but FreeType?

Too much honour, really!  You are thinking of the Ubuntu font family,
right?  AFAIK, its developer, Dalton Maag, is exclusively using
Microsoft's Visual TrueType tool (VTT) for hinting since there is
nothing comparable to this program...

It's always the goal that FreeType yields similar results to the
Microsoft rasterizer.  Everything else would be simply insane.

> My point here is not to argue for reverting this fix, but rather to
> illustrate the impact at a higher level.

I fully understand your concerns, and its unfortunate that many
applications are affected unexpectedly, but I don't see an alternative
to fulfilling the OpenType specification.


    Werner

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Peter Hurley-2]

On Tue, 2012-04-03 at 09:53 -0400, Alexei Podtelezhnikov wrote:

> On Tue, Apr 3, 2012 at 9:05 AM, Peter Hurley <peter@...> wrote:
> > Werner LEMBERG wrote:
> >> > We had report of regression in GTK markups rendering in Ubuntu
> >> > precise, (i.e <u>label</u> would render underlined), [...]
> >> >
> >>
> >> AFAIK, the problem is in gtk.  For years, FreeType had this metrics
> >> bug, and unfortunately the users got used to the appearance of far too
> >> widely spaced lines.  What FreeType now returns is what the font
> >> designer has had in mind while designing the font, and what can be
> >> found in the font.
> >>
> >> As has been stated in one of the Debian bug report comments, the
> >> amount of the vertical line spacing is something which *must* be
> >> configurable or explicitly set by the application.  >
> >
> > One outcome of integer scaling the metrics is that monospace fonts have
> > 'excessive' space between glyphs. Unlike vertical line spacing, there is
> > no way to directly control the max_advance metric value from the
> > application layer -- only the font stack has access to this value.
>
> I'd like to see some images that show that. Please use ftstring. Also
> please remember that freetype renders glyphs and truthfully returns
> metrics. The text layout is not done in freetype.

Sorry, but to me it's pointless to use ftstring as the reference since
it doesn't use FT_Set_Char_Size or metrics.max_advance.

I get that text layout is not done by FreeType; I am saying that higher
layers in the font stack *do* use the metrics returned by FreeType and
now that the way those metrics have changed affects more than vertical
line spacing.

> > Integer scaling *may* have been the font designer's requirement.
> > However, it could be argued that if the font designer was using FreeType
> > to proof the work, that the 'intention' was for the font to look as it
> > did before this fix.
>
> This is highly unlikely and hypothetical. Freetype is trying to
> improve the rendering all the time. Improving meens some changes in
> the way glyphs look.

I agree that my statement is hypothetical -- as was Werner's assertion
that "What FreeType now returns is what the font designer has had in
mind while designing the font".



_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Werner LEMBERG]

> I agree that my statement is hypothetical -- as was Werner's
> assertion that "What FreeType now returns is what the font designer
> has had in mind while designing the font".

My assertion is not hypothetical!  As the goal of FreeType is to be
fully compatible with the MS rasterizer, unexpected differences count
as bugs.


    Werner

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Peter Hurley-2]

On Wed, 2012-04-04 at 01:39 -0400, Werner LEMBERG wrote:
> > I'm referring to when the x-scale is forced to be an integer-multiple of
> > units/EM.
>
> ???  This is not the case.

The description below is from a code trace in GDB. x_scale is
specifically recomputed in tt_size_reset() as:

   metrics->x_scale = FT_DivFix( metrics->x_ppem << 6,
                                 face->root.units_per_EM);

The effect of this line is that "x_scale is forced to be an
integer-multiple of units/EM".

> > For example, let's suppose that I use FT_Set_Char_Size() to specify the
> > desired character width as 10.66 (really, 10 43/64). This generates a
> > size_request() to the TT driver, which the TT driver initially forwards
> > back to FT_Request_Metrics(). Assuming that units_per_EM == 2048, the
> > computed x_scale will be:
> >
> >    x_scale = 10.66 * 2048 = 21856
> >
> > This represents a fractional scaling value.
> > Let's say this produces a max_advance of 384 (or 6.0).
> >
> > Now, if the TrueType font header indicates that integer-scaled values
> > are required (Flags & bit 3 -- see here
> > https://developer.apple.com/fonts/TTRefMan/RM06/Chap6head.html)
> > the TT driver recomputes x_scale:
> >
> >   x_scale = 11 * 2048 = 22528
> >
> > and then recomputes max_advance. This larger scalar produces a
> > max_advance of 448 (7.0). Note that the advance width is integral but
> > now larger by 1 pixel per glyph (for monospace fonts).
>
> Hmm.  You are probably talking about the pixel size, right?  The
> formula to convert from points to pixels (per EM) is
>
>   ppem = pt * dpi / 72
>
> and this resulting ppem value must be an integer if bit 3 is set.  So
> while you can request a non-integer value in `FT_Set_Char_Size', you
> can only request integer values in `FT_Set_Pixel_Sizes'.  Of course,
> if you are using a resolution of 72dpi, the character size equals the
> pixel size.

I tried to be as specific as possible wrt what FreeType actually does; I
can provide a GDB code trace of FT_Set_Char_Size() if that will help.

> > Before the fix, these computations did not affect the font metrics
> > for layout because the base metrics were not changed.  Now they are.
>
> Yes.  If you look into
>
>   http://www.microsoft.com/typography/otspec/head.htm
>
> it explicitly says
>
>   Bit 3: Force ppem to integer values for all internal scaler math;
>          may use fractional ppem sizes if this bit is clear;
>
> which is a better description than in the Apple specification.  The
> old behaviour in FreeType was a bug.

I understand that the old behavior was the result of a bug; I understand
that the specification says to use integer scaling when bit 3 is set
(that's why I included the link -- to help explain what I was referring
to).

> > Just to be clear here: there is no way for an application to control
> > this outcome.
>
> Yes.  BTW, can you provide images for your worst-case scenario?
> Honestly spoken, it sounds quite artificial [...]

There is nothing artificial or contrived about this. 1 pixel per glyph
of wasted space is 132 pixels in a 132-char wide window. 2 windows
side-by-side = 264 pixels. That's valuable screen real-estate.

You can look at
https://bugs.launchpad.net/ubuntu/+source/freetype/+bug/966654

Of course, those images aren't meant to be convincing, just
illustrative. I can prepare more 'convincing' comparative images, but
realistically, that's going to be a waste of my time, don't you think?

> You are thinking of the Ubuntu font family,
> right?  AFAIK, its developer, Dalton Maag, is exclusively using
> Microsoft's Visual TrueType tool (VTT) for hinting since there is
> nothing comparable to this program...

Ok, but they're reverting this so that the designed font appears as they
intended.

> > My point here is not to argue for reverting this fix, but rather to
> > illustrate the impact at a higher level.
>
> I fully understand your concerns, and its unfortunate that many
> applications are affected unexpectedly, but I don't see an alternative
> to fulfilling the OpenType specification.

Thanks for taking the time to appreciate my point-of-view. I too do not
see an alternative but to faithfully represent the specification.

Regards,
Peter Hurley


_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel

[Werner LEMBERG]

>> > I'm referring to when the x-scale is forced to be an
>> > integer-multiple of units/EM.
>>
>> ???  This is not the case.
>
> The description below is from a code trace in GDB. x_scale is
> specifically recomputed in tt_size_reset() as:
>
>    metrics->x_scale = FT_DivFix( metrics->x_ppem << 6,
>                                  face->root.units_per_EM);
>
> The effect of this line is that "x_scale is forced to be an
> integer-multiple of units/EM".

OK, my mistake, I misunderstood.

>> So while you can request a non-integer value in `FT_Set_Char_Size',
>> you can only request integer values in `FT_Set_Pixel_Sizes'.  Of
>> course, if you are using a resolution of 72dpi, the character size
>> equals the pixel size.
>
> I tried to be as specific as possible wrt what FreeType actually
> does; I can provide a GDB code trace of FT_Set_Char_Size() if that
> will help.

Not necessary, thanks :-) I just want to clarify the terminology: If
you use 72dpi (which is the default), there is no difference between
pixels and points.  However, if you use, say, 300dpi, you actually can
have fractional point sizes (which in turn are normally converted to
integer pixel sizes).


    Werner

_______________________________________________
Freetype-devel mailing list
Freetype-devel@...
https://lists.nongnu.org/mailman/listinfo/freetype-devel