Switch on compiler hardening defaults

[Kees Cook-6]

Hello,

I would like to propose enabling[1] the GCC hardening patches that Ubuntu
uses[2].  Ubuntu has used it successfully for 1.5 years now (3 releases),
and many of the issues have already been fixed in packages that needed
adjustment[3].  After all this time, use of the hardening-wrapper[4]
package is still very low, so I think the right thing to do is to just fix
this in the compiler and everyone wins.  I'm not suggesting that there
won't be added work to fix problems, but I believe that for Debian the
benefits now out-weigh the risks.

I do not expect to reach consensus with all developers on this, so I'm
not sure who I need to convince to move it forward.  (Perhaps just the
GCC maintainers?)  Regardless, if you agree with this, please speak up.
I think it's very important that this change happens.

My candid commentary and possible trolling...

Arguments for:
    - users of Debian become safer (real[5] security vulnerabilities are
      either non-issues or reduced to a DoS).
    - security team has less work to do.
    - software quality improves by finding subtle bugs (and not just
      packaged software -- anything compiled with the Debian gcc).

Arguments against:
    - makes the compiler's behavior different than stock compiler.
        Rebuttal: honestly, I don't care -- it seems like such a
                  huge win for safety and is easy to debug.  Debian
                  already carries plenty of patches anyway -- there
                  is no such thing as the "stock compiler".
    - makes more work for dealing with warnings.
        Rebuttal: those warnings are there for a reason -- they can
                  be real security issues, and should be fixed.
    - lacks documentation.
        Rebuttal: that may have been true a while ago, but I've worked
                  hard to document the features and how to handle
                  problems.  See [2].  Even the gcc man pages are patched.
    - makes running Debian slower.
        Rebuttal: no, nothing supports this.  The bulk of _FORTIFY_SOURCE
                  is compile-time.  Run-time checks, including those from
                  -fstack-protector are just not measurable.  The burden of
                  evidence for anyone claiming this is on them.  I'm not
                  suggesting we turn on PIE; that option can be a problem.

Inflammatory observation: Debian may be the single remaining major Linux
distribution that does not use the stack protector and _FORTIFY_SOURCE
when building its packages.  I find this embarrassing.  Check[6] for
yourself.

Thanks,

-Kees

[1] http://outflux.net/hardening-for-all.patch
    (Note that the gcc hardening does NOT turn on PIE, which has
     measurable performance problems on some architectures.)

[2] https://wiki.ubuntu.com/CompilerFlags

[3] Sampling of bugs I've personally filed:
    Closed
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=521108
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=529074
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=479398
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=488456
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=488457
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=497833
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=497865
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505734
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505233
    Open
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=523807
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=488460
        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=488462

[4] http://wiki.debian.org/Hardening

[5] Many vulnerabilities have been blocked in Ubuntu, but I will give one
    good example of a remote root vulnerability with functional exploits
    in the wild that was a non-issue on versions of Ubuntu with the
    hardened compiler defaults:
    http://www.debian.org/security/2009/dsa-1833

[6] Are there _chk functions in common binaries?
    $ objdump -R /bin/df | grep _chk
    0000000000612048 R_X86_64_JUMP_SLOT  __fprintf_chk
    0000000000612068 R_X86_64_JUMP_SLOT  __printf_chk
    00000000006120c0 R_X86_64_JUMP_SLOT  __memcpy_chk
    00000000006121c0 R_X86_64_JUMP_SLOT  __stack_chk_fail
    0000000000612220 R_X86_64_JUMP_SLOT  __sprintf_chk
    0000000000612230 R_X86_64_JUMP_SLOT  __snprintf_chk


--
Kees Cook                                            @debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[James Vega-2]

On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:

> Arguments against:
>     - makes the compiler's behavior different than stock compiler.
>         Rebuttal: honestly, I don't care -- it seems like such a
>                   huge win for safety and is easy to debug.  Debian
>                   already carries plenty of patches anyway -- there
>                   is no such thing as the "stock compiler".
>     - makes more work for dealing with warnings.
>         Rebuttal: those warnings are there for a reason -- they can
>                   be real security issues, and should be fixed.
>     - lacks documentation.
>         Rebuttal: that may have been true a while ago, but I've worked
>                   hard to document the features and how to handle
>                   problems.  See [2].  Even the gcc man pages are patched.
>     - makes running Debian slower.
>         Rebuttal: no, nothing supports this.  The bulk of _FORTIFY_SOURCE
>                   is compile-time.  Run-time checks, including those from
>                   -fstack-protector are just not measurable.  The burden of
>                   evidence for anyone claiming this is on them.  I'm not
>                   suggesting we turn on PIE; that option can be a problem.

- breaks debugging with gdb.  See
  <1256300822.13273.39.camel@...> on this list and #346409.
  You provided a patch for #346409, but there appears to be issues with it as
  noted in the bug log.

--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan@...>

[Ryan Niebur-3]

On Sun, Oct 25, 2009 at 03:21:01PM -0400, James Vega wrote:

> On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:
> > Arguments against:
> >     - makes the compiler's behavior different than stock compiler.
> >         Rebuttal: honestly, I don't care -- it seems like such a
> >                   huge win for safety and is easy to debug.  Debian
> >                   already carries plenty of patches anyway -- there
> >                   is no such thing as the "stock compiler".
> >     - makes more work for dealing with warnings.
> >         Rebuttal: those warnings are there for a reason -- they can
> >                   be real security issues, and should be fixed.
> >     - lacks documentation.
> >         Rebuttal: that may have been true a while ago, but I've worked
> >                   hard to document the features and how to handle
> >                   problems.  See [2].  Even the gcc man pages are patched.
> >     - makes running Debian slower.
> >         Rebuttal: no, nothing supports this.  The bulk of _FORTIFY_SOURCE
> >                   is compile-time.  Run-time checks, including those from
> >                   -fstack-protector are just not measurable.  The burden of
> >                   evidence for anyone claiming this is on them.  I'm not
> >                   suggesting we turn on PIE; that option can be a problem.
>
> - breaks debugging with gdb.  See
>   <1256300822.13273.39.camel@...> on this list and #346409.
>   You provided a patch for #346409, but there appears to be issues with it as
>   noted in the bug log.
>

in the footnotes of Kees's email it said:
    (Note that the gcc hardening does NOT turn on PIE, which has
     measurable performance problems on some architectures.)

so this isn't a problem.

--
_________________________
Ryan Niebur
ryanryan52@...

[Marco d'Itri]

On Oct 25, Kees Cook <kees@...> wrote:

> I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> uses[2].
Seconded.

hardening-wrapper does not looks like a solution to me since it execs
perl for each call to gcc and ld when installed (even when inactive).
And as you noticed, nobody uses it (starting with me).

--
ciao,
Marco

[Russell Coker]

On Monday 26 October 2009 09:22:26 Marco d'Itri wrote:
> > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > uses[2].
>
> Seconded.

Thirded.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Romain Francoise]

Kees Cook <kees@...> writes:

> I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> uses[2].  Ubuntu has used it successfully for 1.5 years now (3 releases),
> and many of the issues have already been fixed in packages that needed
> adjustment[3].  After all this time, use of the hardening-wrapper[4]
> package is still very low, so I think the right thing to do is to just fix
> this in the compiler and everyone wins.  I'm not suggesting that there
> won't be added work to fix problems, but I believe that for Debian the
> benefits now out-weigh the risks.

Agreed. The freeze is months away, there's plenty of time to deal
with the potential fallout of enabling this, so let's just do it.

--
Romain Francoise <rfrancoise@...>
http://people.debian.org/~rfrancoise/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michael Tautschnig-4]

> On Monday 26 October 2009 09:22:26 Marco d'Itri wrote:
> > > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > > uses[2].
> >
> > Seconded.
>
> Thirded.
>

+1.

Thanks for bringing this up,
Michael

[Bastian Blank]

On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:
> I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> uses[2].

How do they work? Do they also change the free-standing compiler or only
the hosted one? There is a lot of software, which (I would say) missuse
the hosted compiler to build non-userspace-code, including the Linux
kernel.

Bastian

--
History tends to exaggerate.
                -- Col. Green, "The Savage Curtain", stardate 5906.4


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Gabor Gombas]

On Mon, Oct 26, 2009 at 11:14:25AM +0100, Bastian Blank wrote:
> On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:
> > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > uses[2].
>
> How do they work? Do they also change the free-standing compiler or only
> the hosted one? There is a lot of software, which (I would say) missuse
> the hosted compiler to build non-userspace-code, including the Linux
> kernel.

It seems the kernel will not be happy if the stack protector is switched
on unconditionally:

http://osdir.com/ml/linux-kernel/2009-10/msg07064.html

Gabor

--
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Florian Weimer]

* Kees Cook:

> I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> uses[2].

Seems a good idea to me.  But I think we should defer the required
full archive rebuild until we've got the hardening patch for operator
new[] (which currently can return a heap block which is smaller than
requested).  I've got a preliminary version, but it's got a hole when
operator new[] is invoked on a variable-length array.  The easy fix
would probably to outlaw heap allocation of VLAs (it's one of those C
GCC extensions that leaked into C++, and it's arguably less needed for
C++).


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Kees Cook-6]

Hi,

On Mon, Oct 26, 2009 at 01:36:28PM +0100, Florian Weimer wrote:

> * Kees Cook:
> > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > uses[2].
>
> Seems a good idea to me.  But I think we should defer the required
> full archive rebuild until we've got the hardening patch for operator
> new[] (which currently can return a heap block which is smaller than
> requested).  I've got a preliminary version, but it's got a hole when
> operator new[] is invoked on a variable-length array.  The easy fix
> would probably to outlaw heap allocation of VLAs (it's one of those C
> GCC extensions that leaked into C++, and it's arguably less needed for
> C++).

Right, I agree with this -- I figure this release can be seen as a
transition release, where not everything is compiled that way.  I don't
want to introduce so much archive churn anyway.

-Kees

--
Kees Cook                                            @debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Christoph Anton Mitterer-2]

Hi.

Ever thought about integrating PaX [0] per default in Debian?
I'm however not sure how much this actually breaks ;)


Cheers,
Chris.


[0] http://pax.grsecurity.net/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Paul Wise-3]

On Tue, Oct 27, 2009 at 4:41 AM, Christoph Anton Mitterer
<calestyo@...> wrote:

> Ever thought about integrating PaX [0] per default in Debian?
> I'm however not sure how much this actually breaks ;)

Any idea if these patches will be merged upstream?

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Yves-Alexis Perez-2]

On mar., 2009-10-27 at 09:32 +0800, Paul Wise wrote:
> On Tue, Oct 27, 2009 at 4:41 AM, Christoph Anton Mitterer
> <calestyo@...> wrote:
>
> > Ever thought about integrating PaX [0] per default in Debian?
> > I'm however not sure how much this actually breaks ;)
>
> Any idea if these patches will be merged upstream?

I don't think so. From the wikipedia page:

----
As of mid 2004, PaX has not been submitted for the mainline kernel tree
because The PaX Team does not think it yet appropriate; although PaX is
fully functional on many CPU architectures, including the popular x86
architecture used by most, it still remains partially or fully
unimplemented on some architectures. Those that PaX is effective on
include IA-32(x86), AMD64, IA-64, Alpha, PA-RISC, and 32 and 64 bit
MIPS, PowerPC, and SPARC architectures
----

And I think there is periodically threads on LKML but can't find a
relevant one quickly.


Cheers,

--
Yves-Alexis

[Paul Wise-3]

On Tue, Oct 27, 2009 at 2:52 PM, Yves-Alexis Perez <corsac@...> wrote:

> On mar., 2009-10-27 at 09:32 +0800, Paul Wise wrote:
>> On Tue, Oct 27, 2009 at 4:41 AM, Christoph Anton Mitterer
>> <calestyo@...> wrote:
>>
>> > Ever thought about integrating PaX [0] per default in Debian?
>> > I'm however not sure how much this actually breaks ;)
>>
>> Any idea if these patches will be merged upstream?
>
> I don't think so.

I guess that answers the question of integrating PaX into Debian:

http://wiki.debian.org/DebianKernelPatchAcceptanceGuidelines
http://kernel-handbook.alioth.debian.org/ch-source.html#s-acceptance

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Henrique de Moraes Holschuh]

On Mon, 26 Oct 2009, Gabor Gombas wrote:

> On Mon, Oct 26, 2009 at 11:14:25AM +0100, Bastian Blank wrote:
> > On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:
> > > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > > uses[2].
> >
> > How do they work? Do they also change the free-standing compiler or only
> > the hosted one? There is a lot of software, which (I would say) missuse
> > the hosted compiler to build non-userspace-code, including the Linux
> > kernel.
>
> It seems the kernel will not be happy if the stack protector is switched
> on unconditionally:
>
> http://osdir.com/ml/linux-kernel/2009-10/msg07064.html

Indeed.  The kernel build system needs to be able to command whether
stackprotect is enabled or not without surprises...

I assume very performance-critical applications will also need it disabled,
if they have hot paths where dcache footprint matters.  But I think we can
safely assume these will be quite rare, so as long as one can disable the
stackprotector easily enough through CFLAGS, we could just do it in a
case-by-case basis on debian/rules.

--
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Kees Cook-6]

On Mon, Oct 26, 2009 at 11:14:25AM +0100, Bastian Blank wrote:
> On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:
> > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > uses[2].
>
> How do they work? Do they also change the free-standing compiler or only
> the hosted one? There is a lot of software, which (I would say) missuse
> the hosted compiler to build non-userspace-code, including the Linux
> kernel.

The stack protector is conditional on being linked with libc, so, if you
build with -nostdlib (as the kernel does), it is implicitly disabled.

--
Kees Cook                                            @debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Samuel Thibault-8]

Kees Cook, le Tue 27 Oct 2009 14:11:43 -0700, a écrit :

> On Mon, Oct 26, 2009 at 11:14:25AM +0100, Bastian Blank wrote:
> > On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:
> > > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > > uses[2].
> >
> > How do they work? Do they also change the free-standing compiler or only
> > the hosted one? There is a lot of software, which (I would say) missuse
> > the hosted compiler to build non-userspace-code, including the Linux
> > kernel.
>
> The stack protector is conditional on being linked with libc, so, if you
> build with -nostdlib (as the kernel does), it is implicitly disabled.

-nostdlib is a linker option, not a compiler option.  The compiler
would still emit references to __stack_chk_fail.  What you probably
mean is -ffreestanding, but it doesn't prevent references to
__stack_chk_fail either, and it even produces TLS references, see
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29838

Samuel


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Kees Cook-6]

Hi,

On Tue, Oct 27, 2009 at 01:30:12PM -0200, Henrique de Moraes Holschuh wrote:

> On Mon, 26 Oct 2009, Gabor Gombas wrote:
> > On Mon, Oct 26, 2009 at 11:14:25AM +0100, Bastian Blank wrote:
> > > On Sun, Oct 25, 2009 at 11:55:25AM -0700, Kees Cook wrote:
> > > > I would like to propose enabling[1] the GCC hardening patches that Ubuntu
> > > > uses[2].
> > >
> > > How do they work? Do they also change the free-standing compiler or only
> > > the hosted one? There is a lot of software, which (I would say) missuse
> > > the hosted compiler to build non-userspace-code, including the Linux
> > > kernel.
> >
> > It seems the kernel will not be happy if the stack protector is switched
> > on unconditionally:
> >
> > http://osdir.com/ml/linux-kernel/2009-10/msg07064.html
>
> Indeed.  The kernel build system needs to be able to command whether
> stackprotect is enabled or not without surprises...
>
> I assume very performance-critical applications will also need it disabled,
> if they have hot paths where dcache footprint matters.  But I think we can
> safely assume these will be quite rare, so as long as one can disable the
> stackprotector easily enough through CFLAGS, we could just do it in a
> case-by-case basis on debian/rules.

Right, -fno-stack-protector via CFLAGS will disable it (as will
-nostdlib).  The work-arounds for the default are all documented both in the
gcc manpage[1] (though this would need tweaking since it currently says
"Ubuntu") and on the Ubuntu wiki page I mentioned earlier[2].

The specific set of patch that would be enabled are:
 - http://patch-tracker.debian.org/patch/series/view/gcc-4.4/4.4.2-1/gcc-default-format-security.diff
 - http://patch-tracker.debian.org/patch/series/view/gcc-4.4/4.4.2-1/gcc-default-fortify-source.diff
 - http://patch-tracker.debian.org/patch/series/view/gcc-4.4/4.4.2-1/gcc-default-relro.diff
 - http://patch-tracker.debian.org/patch/series/view/gcc-4.4/4.4.2-1/gcc-default-ssp.diff
 - http://patch-tracker.debian.org/patch/series/view/gcc-4.4/4.4.2-1/testsuite-hardening-format.diff
 - http://patch-tracker.debian.org/patch/series/view/gcc-4.4/4.4.2-1/testsuite-hardening-fortify.diff
 - http://patch-tracker.debian.org/patch/series/view/gcc-4.4/4.4.2-1/testsuite-hardening-printf-types.diff

(I am trying[3], since they are general improvements, to get the latter
2 accepted by upstream gcc so our gcc package doesn't need to carry them.)

-Kees

[1] http://manpages.ubuntu.com/manpages/karmic/man1/gcc.1.html
    ...
    NOTE: In Ubuntu 6.10 and later versions this option is enabled by default
    for C, C++, ObjC, ObjC++, if neither @option{-fno-stack-protector}
    nor @option{-nostdlib} are found.
    ...

[2] https://wiki.ubuntu.com/CompilerFlags

[3] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=39536
    http://gcc.gnu.org/bugzilla/show_bug.cgi?id=39537

--
Kees Cook                                            @debian.org


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Christoph Anton Mitterer-2]

On Tue, 2009-10-27 at 09:32 +0800, Paul Wise wrote:
> Any idea if these patches will be merged upstream?
It's probably quite unlikely,... although I never understood why,..
Even though it's available for some architectures,.. it would improve
security at least on them.


Cheers,


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...