size_t changes and 32/64-bit, kernel uio_resid type changed.

View: New views
10 Messages — Rating Filter:   Alert me  

size_t changes and 32/64-bit, kernel uio_resid type changed.

by Matthew Dillon :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

    size_t has been changed to unsigned long (and ssize_t to long).  I may
    revert this prior to the release if we hit pkgsrc problems.  The change
    will be kept for 64-bit machines.

    The change on 32 bit machines seems to do a pretty good job generating
    compiler warnings on 32-bit machines for code that will break on
    64-bit machines.  Because of this I think it is worth seeing how badly
    pkgsrc breaks with the change.  If it breaks too much we will revert
    it on 32 bit machines.

    --

    The kernel uio structure's uio_resid field has been changed from a
    signed integer to a size_t (aka unsigned long).  On 32 bit machines
    the 2G limit on VM functions like mmap() has been removed.  I/O
    functions which return ssize_t (which is signed) still impose a 2G
    limit but all the code inside the kernel has been refactored to use
    the whole 4G space.

    On 64 bit systems all prior 2G size limitations for both I/O and
    VM system calls have been removed.  size_t's full range may be used.

    Since it is possible to mmap() multiple terrabytes and issue a single
    write() system call covering the entire space, the HAMMER filesystem
    has been adjusted to allow read() and write() calls to be interrupted
    for I/O sizes greater then 100MB.  I don't think we have a choice here,
    it is too easy to DOS a machine if they aren't interruptable for the
    large-I/O-size case.

    I have only done a little testing with hugely-sized reads and writes
    to files, pipes, and socket pairs.  I have not tested e.g. sendfile().

                                        -Matt
                                        Matthew Dillon
                                        <dillon@...>

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Garance A Drosihn :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

At 9:49 PM -0700 8/18/09, Matthew Dillon wrote:
>     size_t has been changed to unsigned long (and ssize_t to long).  I may
>     revert this prior to the release if we hit pkgsrc problems.  The change
>     will be kept for 64-bit machines.
>
>     The change on 32 bit machines seems to do a pretty good job generating
>     compiler warnings on 32-bit machines for code that will break on
>     64-bit machines.  Because of this I think it is worth seeing how badly
>     pkgsrc breaks with the change.  If it breaks too much we will revert
>     it on 32 bit machines.

Isn't this explicitly wrong?  Isn't ssize_t defined as holding all
valid values of size_t, *plus* some extra values which can be used
for error indications?

To quote SUSv3:

    "The type ssize_t shall be capable of storing values at least
     in the range [-1, {SSIZE_MAX}]"

So are you defining SSIZE_MAX to be smaller than maximum-long?

--
Garance Alistair Drosehn            =   gad@...
Senior Systems Programmer           or  gad@...
Rensselaer Polytechnic Institute    or  drosih@...

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Jordan Gordeev :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Garance A Drosihn wrote:

> At 9:49 PM -0700 8/18/09, Matthew Dillon wrote:
>>     size_t has been changed to unsigned long (and ssize_t to long).  I
>> may
>>     revert this prior to the release if we hit pkgsrc problems.  The
>> change
>>     will be kept for 64-bit machines.
>>
>>     The change on 32 bit machines seems to do a pretty good job
>> generating
>>     compiler warnings on 32-bit machines for code that will break on
>>     64-bit machines.  Because of this I think it is worth seeing how
>> badly
>>     pkgsrc breaks with the change.  If it breaks too much we will revert
>>     it on 32 bit machines.
>
> Isn't this explicitly wrong?  Isn't ssize_t defined as holding all
> valid values of size_t, *plus* some extra values which can be used
> for error indications?
>
> To quote SUSv3:
>
>    "The type ssize_t shall be capable of storing values at least
>     in the range [-1, {SSIZE_MAX}]"
>
> So are you defining SSIZE_MAX to be smaller than maximum-long?
>
You are mistaken.
ssize_t is expected to be the same size as size_t.
SSIZE_MAX in our case will equal LONG_MAX. What do you see as being
wrong with that?

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Oliver Fromme-45 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Garance A Drosihn wrote:
 > Matthew Dillon wrote:
 > > size_t has been changed to unsigned long (and ssize_t to long).  I may
 > > revert this prior to the release if we hit pkgsrc problems.  The change
 > > will be kept for 64-bit machines.
 >
 > Isn't this explicitly wrong?  Isn't ssize_t defined as holding all
 > valid values of size_t, *plus* some extra values which can be used
 > for error indications?

No, it isn't.

 > To quote SUSv3:
 >
 >    "The type ssize_t shall be capable of storing values at least
 >     in the range [-1, {SSIZE_MAX}]"
 >
 > So are you defining SSIZE_MAX to be smaller than maximum-long?

SSIZE_MAX != SIZE_MAX

Best regards
   Oliver

--
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758,  Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Garance A Drosihn :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

At 8:27 PM +0300 8/19/09, Jordan Gordeev wrote:

>Garance A Drosihn wrote:
>>At 9:49 PM -0700 8/18/09, Matthew Dillon wrote:
>>>     size_t has been changed to unsigned long (and ssize_t to long).  I may
>>>     revert this prior to the release if we hit pkgsrc problems.  The change
>>>     will be kept for 64-bit machines.
>>>
>>>     The change on 32 bit machines seems to do a pretty good job generating
>>>     compiler warnings on 32-bit machines for code that will break on
>>>     64-bit machines.  Because of this I think it is worth seeing how badly
>>>     pkgsrc breaks with the change.  If it breaks too much we will revert
>>>     it on 32 bit machines.
>>
>>Isn't this explicitly wrong?  Isn't ssize_t defined as holding all
>>valid values of size_t, *plus* some extra values which can be used
>>for error indications?
>>
>>To quote SUSv3:
>>
>>    "The type ssize_t shall be capable of storing values at least
>>     in the range [-1, {SSIZE_MAX}]"
>>
>>So are you defining SSIZE_MAX to be smaller than maximum-long?
>>
>You are mistaken.
>ssize_t is expected to be the same size as size_t.
>
>SSIZE_MAX in our case will equal LONG_MAX. What do you see as being
>wrong with that?

Ah, yeah, I see where I'm reading it wrong.  Sorry for the noise.

--
Garance Alistair Drosehn            =   gad@...
Senior Systems Programmer           or  gad@...
Rensselaer Polytechnic Institute    or  drosih@...

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Matthew Dillon :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

    There are plenty of things that can go wrong, we'll have to see what
    our pkgsrc experts have to say once they are able to run some bulk
    builds.

                                                -Matt

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Hasso Tepper-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Matthew Dillon wrote:
> size_t has been changed to unsigned long (and ssize_t to long).  I may
> revert this prior to the release if we hit pkgsrc problems.  The change
> will be kept for 64-bit machines.
>
> The change on 32 bit machines seems to do a pretty good job generating
> compiler warnings on 32-bit machines for code that will break on
> 64-bit machines.  Because of this I think it is worth seeing how badly
> pkgsrc breaks with the change.  If it breaks too much we will revert
> it on 32 bit machines.

As it turns out, it breaks C++ ABI. IMHO it's too high cost for just to
get better warnings. And massive breakage of software on 64-bit machines
is thing of past. In fact I see opposite more nowadays - software written
on 64-bit platforms breaks on 32-bit machines ("Who uses 32-bit nowadays?
Embedded platforms?").


--
Hasso Tepper

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Matthew Dillon :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

   All right.  I'll revert the size_t stuff on thursday.  It was a
   single commit in that deluge.

                                        -Matt

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Garance A Drosihn :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

At 1:36 AM -0700 8/20/09, Matthew Dillon wrote:
>    All right.  I'll revert the size_t stuff on thursday.  It was a
>    single commit in that deluge.
>
> -Matt

Maybe some #ifdef magic, so people could easily compile with the
alternate definitions if they wanted to see what those would do?

--
Garance Alistair Drosehn            =   gad@...
Senior Systems Programmer           or  gad@...
Rensselaer Polytechnic Institute    or  drosih@...

Re: size_t changes and 32/64-bit, kernel uio_resid type changed.

by Simon 'corecode' Schubert :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Garance A Drosihn wrote:
> At 1:36 AM -0700 8/20/09, Matthew Dillon wrote:
>>    All right.  I'll revert the size_t stuff on thursday.  It was a
>>    single commit in that deluge.
>>
>>                     -Matt
>
> Maybe some #ifdef magic, so people could easily compile with the
> alternate definitions if they wanted to see what those would do?

Better not.  Just compile an amd64 world and you should get these warnings.

cheers
  simon