Protecting against kernel NULL-pointer derefs

[Pieter de Boer]

All,

Given the amount of NULL-pointer dereference vulnerabilities in the
FreeBSD kernel that have been discovered of late, I've started looking
at a way to generically protect against the code execution possibilities
of such bugs.

By disallowing userland to map pages at address 0x0 (and a bit beyond),
it is possible to make such NULL-pointer deref bugs mere DoS'es instead
of code execution bugs. Linux has implemented such a protection for a
long while now, by disallowing page mappings on 0x0 - 0xffff.

On FreeBSD, it appears that simply bumping up VM_MIN_ADDRESS to 65536
downgrades a whole class of code execution vulnerabilities to DoS
vulnerabilities. I've raised that #define to 65536 on a 6.4-RELEASE i386
VM. This made at least the mmap() method to map at 0x0 fail.

So:
- How do you feel about disallowing such mappings to protect against
NULL-pointer deref code executions?
- Is bumping VM_MIN_ADDRESS enough to protect against all methods of
creating such mappings (on all supported platforms)?
- Are there unwanted side-effects of raising VM_MIN_ADDRESS?
- Should I file a PR to get this into FreeBSD?

Lemme know,
Pieter


_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Dag-Erling Smørgrav]

Pieter de Boer <pieter@...> writes:
> Given the amount of NULL-pointer dereference vulnerabilities in the
> FreeBSD kernel that have been discovered of late,

Specify "amount" and define "of late".

> By disallowing userland to map pages at address 0x0 (and a bit beyond),
> it is possible to make such NULL-pointer deref bugs mere DoS'es instead
> of code execution bugs. Linux has implemented such a protection for a
> long while now, by disallowing page mappings on 0x0 - 0xffff.

Yes, that really worked out great for them:

http://isc.sans.org/diary.html?storyid=6820

DES
--
Dag-Erling Smørgrav - des@...
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Pieter de Boer]

Dag-Erling Smørgrav wrote:

>> Given the amount of NULL-pointer dereference vulnerabilities in the
>> FreeBSD kernel that have been discovered of late,
> Specify "amount" and define "of late".
'amount' => 2, 'of late' is more figure of speech than anything else.
For me, amount was high enough to get interested and 'of late' may be
because I've not been looking long enough.

>> By disallowing userland to map pages at address 0x0 (and a bit beyond),
>> it is possible to make such NULL-pointer deref bugs mere DoS'es instead
>> of code execution bugs. Linux has implemented such a protection for a
>> long while now, by disallowing page mappings on 0x0 - 0xffff.
>
> Yes, that really worked out great for them:
>
> http://isc.sans.org/diary.html?storyid=6820
I was aware of that issue, and was expecting your comment as well. While
SELinux (and iirc SysV compatibility) effectively killed the "don't map
at 0x0" feature, that does not mean such a feature is useless in of
itself. If it is possible to attain a high enough level of confidence
that such a feature would actually work, without negative side-effects,
I feel that it would be beneficial to FreeBSD.

I'd be interested in hearing your and other's opinions, specifically on
the topics my original questions hinted at.

--
Pieter
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Dag-Erling Smørgrav]

Pieter de Boer <pieter@...> writes:
> Dag-Erling Smørgrav <des@...> writes:
> > Pieter de Boer <pieter@...> writes:
> > > Given the amount of NULL-pointer dereference vulnerabilities in
> > > the FreeBSD kernel that have been discovered of late,
> > Specify "amount" and define "of late".
> 'amount' => 2, 'of late' is more figure of speech than anything
> else. For me, amount was high enough to get interested and 'of late'
> may be because I've not been looking long enough.

A search of FreeBSD security advisories shows two in the last four
years, plus the current unreleased issue.  I agree that there is no
reason to allow applications to mmap() at address 0, but surely there
must be a better way to make your case than to sow FUD?

DES
--
Dag-Erling Smørgrav - des@...
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Pieter de Boer]

Dag-Erling Smørgrav wrote:

>> 'amount' => 2, 'of late' is more figure of speech than anything
>> else. For me, amount was high enough to get interested and 'of late'
>> may be because I've not been looking long enough.
>
> A search of FreeBSD security advisories shows two in the last four
> years, plus the current unreleased issue.  I agree that there is no
> reason to allow applications to mmap() at address 0, but surely there
> must be a better way to make your case than to sow FUD?
I have no intention to sow FUD. Three such advisories is not much, but
if there is a simple/inexpensive way to ensure that such bugs are not
exploitable to gain root, I think 'we' should consider it.

--
Pieter
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Jon Passki-3]

2009/9/15 Dag-Erling Smørgrav <des@...>

>
> Pieter de Boer <pieter@...> writes:
> > Given the amount of NULL-pointer dereference vulnerabilities in the
> > FreeBSD kernel that have been discovered of late,
>
> Specify "amount" and define "of late".
>
> > By disallowing userland to map pages at address 0x0 (and a bit beyond),
> > it is possible to make such NULL-pointer deref bugs mere DoS'es instead
> > of code execution bugs. Linux has implemented such a protection for a
> > long while now, by disallowing page mappings on 0x0 - 0xffff.
>
> Yes, that really worked out great for them:
>
> http://isc.sans.org/diary.html?storyid=6820

As I assume you know, one reason (not the only reason) the exploit
works is because the SELinux default policy allowed (allows?) users to
map at NULL, regardless of the protections offered by the OS (e.g.
Redhat w/ mmap_min_addr).  His later exploit framework abuses SELinux
another way by downgrading protection by going into libselinux and
uses a context such as wine_t to execute at NULL [1].  It's not that
mmap_min_addr failed (which it doesn't on some distros of Linux); it's
that other mechanisms exist that can undo the control put into place.

Cheers,

Jon Passki

[1] http://grsecurity.net/~spender/enlightenment.tgz, exploit.c, pa__init()
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[István-3]

hehe this is the "install another security layer to introduce less security"
model




2009/9/15 Jon Passki <jon@...>

> 2009/9/15 Dag-Erling Smørgrav <des@...>
> >
> > Pieter de Boer <pieter@...> writes:
> > > Given the amount of NULL-pointer dereference vulnerabilities in the
> > > FreeBSD kernel that have been discovered of late,
> >
> > Specify "amount" and define "of late".
> >
> > > By disallowing userland to map pages at address 0x0 (and a bit beyond),
> > > it is possible to make such NULL-pointer deref bugs mere DoS'es instead
> > > of code execution bugs. Linux has implemented such a protection for a
> > > long while now, by disallowing page mappings on 0x0 - 0xffff.
> >
> > Yes, that really worked out great for them:
> >
> > http://isc.sans.org/diary.html?storyid=6820
>
> As I assume you know, one reason (not the only reason) the exploit
> works is because the SELinux default policy allowed (allows?) users to
> map at NULL, regardless of the protections offered by the OS (e.g.
> Redhat w/ mmap_min_addr).  His later exploit framework abuses SELinux
> another way by downgrading protection by going into libselinux and
> uses a context such as wine_t to execute at NULL [1].  It's not that
> mmap_min_addr failed (which it doesn't on some distros of Linux); it's
> that other mechanisms exist that can undo the control put into place.
>
> Cheers,
>
> Jon Passki
>
> [1] http://grsecurity.net/~spender/enlightenment.tgz<http://grsecurity.net/%7Espender/enlightenment.tgz>,
> exploit.c, pa__init()
> _______________________________________________
> freebsd-security@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-security
> To unsubscribe, send any mail to "freebsd-security-unsubscribe@...
> "
>

--
the sun shines for all
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

On Tuesday 15 Sep 2009 13:42:28 Dag-Erling Smørgrav wrote:

> there must be a better way to make your case than to sow FUD?

Where? To paraphrase yourself: Please define "sowing FUD." There's an issue;
there have been two previously. Nobody is blaming anyone, blowing it out of
proportion, leaving FBSD in droves or pointing fingers. We know it's local and
we're all well aware of the axiom "if someone else has physical access to your
box, it isn't your box any more." I don't see or feel any fear, uncertainty or
doubt. I just see a concerned but dedicated FBSD user discussing an issue
sensibly with the information currently to hand.

Providing it does not seriously affect anything else (Pieter has already asked
for information and opinions before the thread went off on this tangent), if
setting this #define downgrades arbitrary code execution vulnerabilities and
privilege escalations to crashes where system and, more importantly IMHO, host
integrity is preserved then I am all for it. I'd certainly rather have a DoS
and the risk of cached data loss than a potential information leak or a
reputation-destroying spamming session run. That we don't have multiple places
that this could be overridden/reset similar to the SELinux issue also inspires
confidence in Pieter's method. As simple as it seems, it would appear to be
(sorry, buzzword-that-fits coming up) proactive in its approach, addressing
and mitigating any future issues of this type and limiting the possible
damage.

Also worth thinking about: Do we need to consider -fno-delete-null-pointer-
checks or a downgrade to -O for kernel/world optimisation level for now until
we're sure there are no more of these lurking? Linux found out the hard way
that code order matters when compiling at >-O and that perfectly acceptable
code coupled with assumptions made by the compiler can bite you in the
backside.
--
Matt Dawson
MTD15-RIPE
matt@...
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Przemyslaw Frasunek-2]

Dag-Erling Smørgrav:
> A search of FreeBSD security advisories shows two in the last four
> years, plus the current unreleased issue.

There are three NULL pointer dereference issues, that I found in last
month, but probably more to come, so implementing some kind of zero page
protection should be considered.

The first one affects 6.1 and it was made public in August:

http://www.frasunek.com/kqueue.txt

Another one affects 6.4 and is currently handled by secteam. Advisory
will be released on Wednesday.

The last one, as demonstrated on http://www.vimeo.com/6580991 affects
7.x up to 7.2 and 6.x up to 6.4. I'm not going to disclose any details
before official security advisory.


_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Dag-Erling Smørgrav]

Matt Dawson <matt@...> writes:
> Dag-Erling Smørgrav <des@...> writes:
> > there must be a better way to make your case than to sow FUD?
> Where? To paraphrase yourself: Please define "sowing FUD." There's an issue;
> there have been two previously.

Pieter strongly implied that there had been *numerous* such cases, when
in fact there hasn't.

DES
--
Dag-Erling Smørgrav - des@...
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Dag-Erling Smørgrav]

Przemyslaw Frasunek <przemyslaw@...> writes:
> Dag-Erling Smørgrav <des@...> writes:
> > A search of FreeBSD security advisories shows two in the last four
> > years, plus the current unreleased issue.
> There are three NULL pointer dereference issues, that I found in last
> month, but probably more to come, so implementing some kind of zero page
> protection should be considered.

Feel free to *actually read what Pieter wrote and what I wrote in reply*

EOD

DES
--
Dag-Erling Smørgrav - des@...
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Chris Palmer-3]

Pieter's approach to the problem seems reasonable. If it provides some safety
without breaking any/too many applications, why not adopt it?

I wonder how many of these kinds of issues could also be caught with unit
tests/regression tests. See also: the CanSecWest 2009 FreeBSD bugs by
Christer Oberg and Neil Kettle.


--
http://www.noncombatant.org/
http://hemiolesque.blogspot.com/

_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

On Tuesday 15 Sep 2009 17:07:35 Dag-Erling Smørgrav wrote:
> Pieter strongly implied that there had been numerous such cases, when
> in fact there hasn't.

Yes, DES, it could be read that way and I apologise. Without trying to
wiggle out of that apology, it just seemed a bit harsh when I doubt what
was written was meant as "the code is riddled with these things! RIDDLED!"
given the fact that Pieter proposed a possible mitigation instead of the
expected "El Reg says it's broken! EL REG! Fix it now, goddammit!" ;o)

@All:
Having put both feet in my mouth and had to publicly apologise, we now have
a little more information from Przemyslaw on what is potentially broken and
what isn't (7.2, the current production release). That "probably more to
come," while still vague and very much unverified, makes me wonder if
Pieter's interim mitigation or something very much like it isn't needed
Right Now [TM] as he says. So, is there any technically sound reason why
raising VM_MIN_ADDRESS to 65536 would not be a good trade-off (or even just
a good idea) in security terms until we're sure there are no more of these
lurking? A few of us paranoid security types might want to do so manually
in the interim if there are no objections.
--
Matt Dawson
MTD15-RIPE
matt@...
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Hideki EIRAKU]

From: Pieter de Boer <pieter@...>
Subject: Protecting against kernel NULL-pointer derefs
Date: Tue, 15 Sep 2009 10:03:48 +0200

> - Are there unwanted side-effects of raising VM_MIN_ADDRESS?

Mapping at address 0x0 is needed by some softwares using vm86 mode.
For example, emulators/doscmd uses vm86 mode to emulate an old DOS
environment. In this case, the address 0x0 - 0x3ff is used for an
interrupt vector of emulated DOS world. If VM_MIN_ADDRESS is not zero,
doscmd doesn't work. vm86 mode is in 32bit i386 architecture only, not
in amd64 or other architectures.

--
Hideki EIRAKU
--------------------------------------
Thanks 10 years!  Yahoo! Shopping and Yahoo! Auctions
http://pr.mail.yahoo.co.jp/ec10years/
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[István-3]

the question is how much percent of the user are using wine and dosbox which
are going to break with this setting, i guess 10% or less. So those guys
could use _NO_VM_MIN kernel or something while the rest of the world would
fly high with secured kernel.

to quote a security friend:

the freebsd was the last target platform for this bugclass :))

regards,
Istvan


On Fri, Sep 18, 2009 at 4:13 PM, Hideki EIRAKU <hdk_2@...> wrote:

> From: Pieter de Boer <pieter@...>
> Subject: Protecting against kernel NULL-pointer derefs
> Date: Tue, 15 Sep 2009 10:03:48 +0200
>
> > - Are there unwanted side-effects of raising VM_MIN_ADDRESS?
>
> Mapping at address 0x0 is needed by some softwares using vm86 mode.
> For example, emulators/doscmd uses vm86 mode to emulate an old DOS
> environment. In this case, the address 0x0 - 0x3ff is used for an
> interrupt vector of emulated DOS world. If VM_MIN_ADDRESS is not zero,
> doscmd doesn't work. vm86 mode is in 32bit i386 architecture only, not
> in amd64 or other architectures.
>
> --
> Hideki EIRAKU
> --------------------------------------
> Thanks 10 years!  Yahoo! Shopping and Yahoo! Auctions
> http://pr.mail.yahoo.co.jp/ec10years/
> _______________________________________________
> freebsd-security@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-security
> To unsubscribe, send any mail to "freebsd-security-unsubscribe@...
> "
>

--
the sun shines for all
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Julian Elischer]

István wrote:
> the question is how much percent of the user are using wine and dosbox which
> are going to break with this setting, i guess 10% or less. So those guys
> could use _NO_VM_MIN kernel or something while the rest of the world would
> fly high with secured kernel.

The assumption is that the userland and kernel share a memory map.
While we do implement it this way, it is not necessarily needed.
We do it for performance reasons (each user memory map includes an
identical top section that is the kernel space, so that we do not need
to switch memory page arenas (change CR3) when entering the kernel.
However it might be possible to not do this, and in fact on some
hardware it is mandatory to not do this).

It would require a page table arena switch with each syscall which
would require flushing the TLBs which would be expensive..
Hmm I guess I've talked myself out of this as a solution..  :-)

Julian

>
> to quote a security friend:
>
> the freebsd was the last target platform for this bugclass :))
>
> regards,
> Istvan
>
>
> On Fri, Sep 18, 2009 at 4:13 PM, Hideki EIRAKU <hdk_2@...> wrote:
>
>> From: Pieter de Boer <pieter@...>
>> Subject: Protecting against kernel NULL-pointer derefs
>> Date: Tue, 15 Sep 2009 10:03:48 +0200
>>
>>> - Are there unwanted side-effects of raising VM_MIN_ADDRESS?
>> Mapping at address 0x0 is needed by some softwares using vm86 mode.
>> For example, emulators/doscmd uses vm86 mode to emulate an old DOS
>> environment. In this case, the address 0x0 - 0x3ff is used for an
>> interrupt vector of emulated DOS world. If VM_MIN_ADDRESS is not zero,
>> doscmd doesn't work. vm86 mode is in 32bit i386 architecture only, not
>> in amd64 or other architectures.
>>
>> --
>> Hideki EIRAKU
>> --------------------------------------
>> Thanks 10 years!  Yahoo! Shopping and Yahoo! Auctions
>> http://pr.mail.yahoo.co.jp/ec10years/
>> _______________________________________________
>> freebsd-security@... mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-security
>> To unsubscribe, send any mail to "freebsd-security-unsubscribe@...
>> "
>>
>
>
>

_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Garance A Drosihn]

At 12:13 AM +0900 9/19/09, Hideki EIRAKU wrote:

>From: Pieter de Boer <pieter@...>
>Subject: Protecting against kernel NULL-pointer derefs
>Date: Tue, 15 Sep 2009 10:03:48 +0200
>
>>  - Are there unwanted side-effects of raising VM_MIN_ADDRESS?
>
>Mapping at address 0x0 is needed by some softwares using vm86 mode.
>For example, emulators/doscmd uses vm86 mode to emulate an old DOS
>environment. In this case, the address 0x0 - 0x3ff is used for an
>interrupt vector of emulated DOS world. If VM_MIN_ADDRESS is not
>zero, doscmd doesn't work. vm86 mode is in 32bit i386 architecture
>only, not in amd64 or other architectures.

Could we:

a) alter those programs so they didn't need that vector in page 0?

or

b) provide some system call or other facility which would allow
    *that* process to use page 0?

--
Garance Alistair Drosehn            =   gad@...
Senior Systems Programmer           or  gad@...
Rensselaer Polytechnic Institute    or  drosih@...
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Pieter de Boer]

Julian wrote:

> The assumption is that the userland and kernel share a memory map.
> While we do implement it this way, it is not necessarily needed.
> We do it for performance reasons (each user memory map includes an
> identical top section that is the kernel space, so that we do not need
> to switch memory page arenas (change CR3) when entering the kernel.
> However it might be possible to not do this, and in fact on some
> hardware it is mandatory to not do this).
>
> It would require a page table arena switch with each syscall which
> would require flushing the TLBs which would be expensive..
> Hmm I guess I've talked myself out of this as a solution..  :-)

So, to be able to run VM86 mode or Wine we could make the NULL mapping
protection a configurable kernel option, (defaulting to 'on'?), which
doscmd/wine users could turn off. A nicer way would be to be able to map
0x0 in userland while having the kernel use its own 0x0 mapping.
Possibly there is a way to do that without making context switches very
expensive? Partial TLB flushes??

I also wonder how Linux and (possibly) other OS'es handle this; I can
imagine it can easily become quite messy resulting in added security
issues or insufficient protection. Anyone have pointers to that regard?

--
Pieter
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Bruce Evans-4]

On Fri, 18 Sep 2009, Pieter de Boer wrote:

> Julian wrote:
>> The assumption is that the userland and kernel share a memory map.
>> While we do implement it this way, it is not necessarily needed.
>> We do it for performance reasons (each user memory map includes an
>> identical top section that is the kernel space, so that we do not need
>> to switch memory page arenas (change CR3) when entering the kernel.
>> However it might be possible to not do this, and in fact on some
>> hardware it is mandatory to not do this).
>>
>> It would require a page table arena switch with each syscall which
>> would require flushing the TLBs which would be expensive..
>> Hmm I guess I've talked myself out of this as a solution..  :-)
>
> So, to be able to run VM86 mode or Wine we could make the NULL mapping
> protection a configurable kernel option, (defaulting to 'on'?), which
> doscmd/wine users could turn off.

Does VM86 mode really require or use mapping to kernel address 0?  I think
it doesn't and shouldn't, since  VM86 mode gets a special %cs which can
have a nonzero base address.   Hmm, the user %cs is always different from
the kernel %cs, so I think it can alway have a nonzero base, but then
user addresses would be different from kernel address, which would require
large changes and small extra runtime to convert the addresses.  VM86
mode would hopefully require only small or null changes since it is already
weird.

> A nicer way would be to be able to map
> 0x0 in userland while having the kernel use its own 0x0 mapping.
> Possibly there is a way to do that without making context switches very
> expensive? Partial TLB flushes??

Not just context switches, but all kernel entries and exits are relevant.
I think the cost of switching the map would be small if you only do
it when necessary (on every kernel entry/exit from/to a user context
that has pages mapped near address 0).  Most switches should be null
since most processes shouldn't do that.  This can be optimized a bit
more by delaying the switch back to the unsafe user map until userland
actually accesses a low address.

Bruce
_______________________________________________
freebsd-security@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-security
To unsubscribe, send any mail to "freebsd-security-unsubscribe@..."

[Kostik Belousov]

On Sun, Sep 20, 2009 at 12:44:25AM +1000, Bruce Evans wrote:

> On Fri, 18 Sep 2009, Pieter de Boer wrote:
>
> >Julian wrote:
> >>The assumption is that the userland and kernel share a memory map.
> >>While we do implement it this way, it is not necessarily needed.
> >>We do it for performance reasons (each user memory map includes an
> >>identical top section that is the kernel space, so that we do not need
> >>to switch memory page arenas (change CR3) when entering the kernel.
> >>However it might be possible to not do this, and in fact on some
> >>hardware it is mandatory to not do this).
> >>
> >>It would require a page table arena switch with each syscall which
> >>would require flushing the TLBs which would be expensive..
> >>Hmm I guess I've talked myself out of this as a solution..  :-)
> >
> >So, to be able to run VM86 mode or Wine we could make the NULL mapping
> >protection a configurable kernel option, (defaulting to 'on'?), which
> >doscmd/wine users could turn off.
>
> Does VM86 mode really require or use mapping to kernel address 0?  I think
> it doesn't and shouldn't, since  VM86 mode gets a special %cs which can
> have a nonzero base address.   Hmm, the user %cs is always different from
> the kernel %cs, so I think it can alway have a nonzero base, but then
> user addresses would be different from kernel address, which would require
> large changes and small extra runtime to convert the addresses.  VM86
> mode would hopefully require only small or null changes since it is already
> weird.

In vm86 mode, %cs works exactly the same as in real mode, as well as

all other segment registers. vm86-mode code is free to load any 16bit
value into any segment register, and virtual address is calculated as
(seg << 4) + offset.

>
> >A nicer way would be to be able to map
> >0x0 in userland while having the kernel use its own 0x0 mapping.
> >Possibly there is a way to do that without making context switches very
> >expensive? Partial TLB flushes??
>
> Not just context switches, but all kernel entries and exits are relevant.
> I think the cost of switching the map would be small if you only do
> it when necessary (on every kernel entry/exit from/to a user context
> that has pages mapped near address 0).  Most switches should be null
> since most processes shouldn't do that.  This can be optimized a bit
> more by delaying the switch back to the unsafe user map until userland
> actually accesses a low address.

Redhat did that some time ago, but do not any more.