Abstraction leak

[Jon Fairbairn]

Andrew Coppin <andrewcoppin@...> writes:

> While we're on the subject... am I the first person to
> notice that Haskell doesn't appear to have much support for
> fiddling with streams of bits?

No. Presumably the author of Data.Bits noticed some
lack. (Note that Integer is an instance of Num and hence
Bits)

--
Jón Fairbairn                                 Jon.Fairbairn@...

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Josef Svenningsson]

On 6/30/07, Jon Cast <jcast@...> wrote:
> On Friday 29 June 2007, Jon Cast wrote:
> > Here's my solution (drawn from a library I'll be posting Real Soon Now):
> <snip solution>
>
> I forgot to point out that this is 75-90% drawn from a library called
> Fudgets[1], which is probably the most extended practical meditation to date
> on programming with lazy streams in Haskell.  Embedding that approach in a
> monadic interface seems to be my own idea, though.
>
Koen Claessen had the same idea. He used it for designing parsers. See:
http://www.cs.chalmers.se/~koen/pubs/entry-jfp04-parser.html

> Jonathan Cast
> http://sourceforge.net/projects/fid-core
> http://sourceforge.net/projects/fid-emacs
>
> [1] http://www.md.chalmers.se/Cs/Research/Functional/Fudgets/

Cheers,

Josef
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Andrew Coppin]

Jon Fairbairn wrote:

> Andrew Coppin <andrewcoppin@...> writes:
>
>  
>> While we're on the subject... am I the first person to
>> notice that Haskell doesn't appear to have much support for
>> fiddling with streams of bits?
>>    
>
> No. Presumably the author of Data.Bits noticed some
> lack. (Note that Integer is an instance of Num and hence
> Bits)
>  

Right. But (for instance) there is no library function anywhere to
convert a Word16 into a [Bool] (never mind handling endian issues), or
back again. There is no standard construct for reading from a [Word8] as
if it's a [Bool], or for writing to a [Bool] and ending up with a [Word8]...

...Then again, there is no library function for reading from a file and
getting a [Word8]. The only way I have found to do this is to open a
file, put the handle into "binary mode", use hGetContents to read a
[Char] from it, and then do a map (fromIntegral . fromEnum) on that.
Writing binary data back to a file requires opening a handle, putting it
into binary mode, taking your [Word8] and doing a map (toEnum .
fromIntegral) over it, and then using putStr. All of which works
because, apparently, in binary mode the file is interpreted as 8-bit
ASCII. God forbit that Haskell ever decides to transparently support
other encodings...

I haven't actually tried, but presumably a TCP connection is represented
in the same way as a file, and so has the same problems.

Basically doing binary I/O seems to be one of those things that in
Haskell falls into the class of "it's possibly but annoyingly messy"...

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Philip Armstrong]

On Sun, Jul 01, 2007 at 06:07:13PM +0100, Andrew Coppin wrote:
> I haven't actually tried, but presumably a TCP connection is represented in
> the same way as a file, and so has the same problems.
>
> Basically doing binary I/O seems to be one of those things that in Haskell
> falls into the class of "it's possibly but annoyingly messy"...

In an ideal world there would be a 'deriving Serializable[1]' you
could do on datatypes which would get this right. In a really ideal
world, you could specify the data layout somehow[2][2a], which would
make integrating Haskell code into a wider distributed network of
processes exchanging binary data a cinch. In a super really ideal
world, you could operate on the packets in place in Haskell where
possible and save the deserialization overhead...

Anyone trying to do any of this?

Phil

[1] deriving picklable?
[2] DFDL does this for XML / binary data translation.
[2a] Or even dump to arbitrary formats: XML, JSON for concrete
      datatypes[3], mabe use the approach from
      http://www.ps.uni-sb.de/Papers/abstracts/hotPickles2007.html
      (link stolen shamelessly from Lambda the Ultimate) for higher
      order data?
[3] Maybe UBL from http://www.erlang.se/workshop/2002/Armstrong.pdf ?

--
http://www.kantaka.co.uk/ .oOo. public key: http://www.kantaka.co.uk/gpg.txt
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Donald Bruce Stewart]

phil:
> On Sun, Jul 01, 2007 at 06:07:13PM +0100, Andrew Coppin wrote:
> >I haven't actually tried, but presumably a TCP connection is represented
> >in the same way as a file, and so has the same problems.
> >
> >Basically doing binary I/O seems to be one of those things that in Haskell
> >falls into the class of "it's possibly but annoyingly messy"...
>
> In an ideal world there would be a 'deriving Serializable[1]' you

derive Binary
    (use an external tool for this)

> could do on datatypes which would get this right. In a really ideal
> world, you could specify the data layout somehow[2][2a], which would

Directly in Haskell data type decls -- see the ICFP 05 paper on the
House OS, which added packing annotations, and bit syntax. In current
Haskell, you specify the layout in instances Storable or Binary.

> make integrating Haskell code into a wider distributed network of
> processes exchanging binary data a cinch. In a super really ideal

Definitely. See things like the zlib or iconv
Data.Binary/Data.ByteString bindings, for prototypes. The 'tar'
reader/writer on hackage.haskell.org is also a good example.

> world, you could operate on the packets in place in Haskell where
> possible and save the deserialization overhead...

Data.ByteString.* for this.
 
> Anyone trying to do any of this?

Yeah, its a bit of a hot topic currently. :)
Gotta chase Erlang for hacking network data.

-- Don
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Thomas Conway]

> Anyone trying to do any of this?

I've done some work in this area. I'm particularly interested in
manipulating ASN.1 in haskell. Actually, my first use of Parsec was an
ASN.1 parser. I'd done one previously in Spirit (the Boost C++ rip-off
of parsec), but semantic actions were horrible in the extreme. Mmmm
Parsec.

In the indexing system I'm currently building in Haskell for my day
job, I'm serializing several data structures, and using Data.Bits and
Data.ByteString heavily.

I was using HaXml, but I found it was very slow. So instead, I'm using
an internal (within the indexing system) representation that is more
akin to WBXML:

import Data.ByteString as ByteString
import Data.List as List
import Data.Sequence as Seq

data DocTree
    = DocElem ByteString [(ByteString,ByteString)] [DocTree]
    | DocText ByteString

serialize tree = ByteString.concat $ Seq.toList $ execState
(serialize' tree) Seq.empty
serialize' (DocText txt) = do
    stuff <- get
    put (stuff |> pack [0])
    putStr txt
serialize' (DocElem name attrs kids) = do
    stuff <- get
    put (stuff |> pack [1])
    putStr name
    putNum (List.length attrs)
    mapM_ (putPair putStr putStr) attrs
    putNum (List.length kids)
    mapM_ serialize' kids

putStr ....

You get the idea. Actually, the *real* code is trickier - it grovels
first to find all the element names and numbers them. Likewise with
attribute names (per element). The extra grovel is well worth it - it
takes a little longer to serialize, but is more compact and
deserializes quicker.

Also worth noting - whether you compile a dictionary of element names
or not, the result is much much much more space efficient than using
HaXml, since it can all be decoded out of a single ByteString
containing the document tree, with no actual string copying at all.
That's the kind of [de]serialization I like. :-) Mind you, I still
have to use HaXml when I first read documents into the system, and a
very nice job it does too.

T.
--
Dr Thomas Conway
drtomc@...

Silence is the perfectest herald of joy:
I were but little happy, if I could say how much.
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Donald Bruce Stewart]

drtomc:

> >Anyone trying to do any of this?
>
> I've done some work in this area. I'm particularly interested in
> manipulating ASN.1 in haskell. Actually, my first use of Parsec was an
> ASN.1 parser. I'd done one previously in Spirit (the Boost C++ rip-off
> of parsec), but semantic actions were horrible in the extreme. Mmmm
> Parsec.
>
> In the indexing system I'm currently building in Haskell for my day
> job, I'm serializing several data structures, and using Data.Bits and
> Data.ByteString heavily.
>
> I was using HaXml, but I found it was very slow. So instead, I'm using
> an internal (within the indexing system) representation that is more
> akin to WBXML:
>
> import Data.ByteString as ByteString
> import Data.List as List
> import Data.Sequence as Seq
>
> data DocTree
>    = DocElem ByteString [(ByteString,ByteString)] [DocTree]
>    | DocText ByteString
>
> serialize tree = ByteString.concat $ Seq.toList $ execState
> (serialize' tree) Seq.empty
> serialize' (DocText txt) = do
>    stuff <- get
>    put (stuff |> pack [0])
>    putStr txt
> serialize' (DocElem name attrs kids) = do
>    stuff <- get
>    put (stuff |> pack [1])
>    putStr name
>    putNum (List.length attrs)
>    mapM_ (putPair putStr putStr) attrs
>    putNum (List.length kids)
>    mapM_ serialize' kids
>
> putStr ....
>
> You get the idea. Actually, the *real* code is trickier - it grovels
> first to find all the element names and numbers them. Likewise with
> attribute names (per element). The extra grovel is well worth it - it
> takes a little longer to serialize, but is more compact and
> deserializes quicker.
>
> Also worth noting - whether you compile a dictionary of element names
> or not, the result is much much much more space efficient than using
> HaXml, since it can all be decoded out of a single ByteString
> containing the document tree, with no actual string copying at all.
> That's the kind of [de]serialization I like. :-) Mind you, I still
> have to use HaXml when I first read documents into the system, and a
> very nice job it does too.

Can we do a cheap bytestring binding to libxml, to avoid any initial
String processing?

-- Don
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Philip Armstrong]

On Wed, Jul 04, 2007 at 09:02:15PM +1000, Donald Bruce Stewart wrote:

>phil:
>> On Sun, Jul 01, 2007 at 06:07:13PM +0100, Andrew Coppin wrote:
>> >I haven't actually tried, but presumably a TCP connection is represented
>> >in the same way as a file, and so has the same problems.
>> >
>> >Basically doing binary I/O seems to be one of those things that in Haskell
>> >falls into the class of "it's possibly but annoyingly messy"...
>>
>> In an ideal world there would be a 'deriving Serializable[1]' you
>
>derive Binary
>    (use an external tool for this)

such as?

>> could do on datatypes which would get this right. In a really ideal
>> world, you could specify the data layout somehow[2][2a], which would
>
>Directly in Haskell data type decls -- see the ICFP 05 paper on the
>House OS, which added packing annotations, and bit syntax. In current
>Haskell, you specify the layout in instances Storable or Binary.

I'll have a look.

>> make integrating Haskell code into a wider distributed network of
>> processes exchanging binary data a cinch. In a super really ideal
>
>Definitely. See things like the zlib or iconv
>Data.Binary/Data.ByteString bindings, for prototypes. The 'tar'
>reader/writer on hackage.haskell.org is also a good example.

OK. Maybe this is the sort of stuff which ought to go into the new
Haskell book? 'Integrating Haskell with external data sources' or
something...

>> world, you could operate on the packets in place in Haskell where
>> possible and save the deserialization overhead...
>
>Data.ByteString.* for this.
>
>> Anyone trying to do any of this?
>
>Yeah, its a bit of a hot topic currently. :)

>Gotta chase Erlang for hacking network data.

Absolutely. Ta for the pointers anyway.

Phil

--
http://www.kantaka.co.uk/ .oOo. public key: http://www.kantaka.co.uk/gpg.txt
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Donald Bruce Stewart]

phil:

> On Wed, Jul 04, 2007 at 09:02:15PM +1000, Donald Bruce Stewart wrote:
> >phil:
> >>On Sun, Jul 01, 2007 at 06:07:13PM +0100, Andrew Coppin wrote:
> >>>I haven't actually tried, but presumably a TCP connection is represented
> >>>in the same way as a file, and so has the same problems.
> >>>
> >>>Basically doing binary I/O seems to be one of those things that in
> >>Haskell >falls into the class of "it's possibly but annoyingly messy"...
> >>
> >>In an ideal world there would be a 'deriving Serializable[1]' you
> >
> >derive Binary
> >   (use an external tool for this)
>
> such as?

Binary instances are pretty easy to write. For a simple data type:

       > instance Binary Exp where
       >       put (IntE i)          = do put (0 :: Word8)
       >                                  put i
       >       put (OpE s e1 e2)     = do put (1 :: Word8)
       >                                  put s
       >                                  put e1
       >                                  put e2

       >       get = do tag <- getWord8
       >                case tag of
       >                    0 -> liftM  IntE get
       >                    1 -> liftM3 OpE  get get get

The Data.Binary comes with one tool to derive these. The DrIFT preprocessor
also can, as can Stefan O'Rear's SYB deriver.

I just write them by hand, or use the tool that comes with the lib.

More docs here,
    http://hackage.haskell.org/packages/archive/binary/0.3/doc/html/Data-Binary.html

>
> >>could do on datatypes which would get this right. In a really ideal
> >>world, you could specify the data layout somehow[2][2a], which would
> >
> >Directly in Haskell data type decls -- see the ICFP 05 paper on the
> >House OS, which added packing annotations, and bit syntax. In current
> >Haskell, you specify the layout in instances Storable or Binary.
>
> I'll have a look.
>
> >>make integrating Haskell code into a wider distributed network of
> >>processes exchanging binary data a cinch. In a super really ideal
> >
> >Definitely. See things like the zlib or iconv
> >Data.Binary/Data.ByteString bindings, for prototypes. The 'tar'
> >reader/writer on hackage.haskell.org is also a good example.
>
> OK. Maybe this is the sort of stuff which ought to go into the new
> Haskell book? 'Integrating Haskell with external data sources' or
> something...

Indeed :-)
 

> >>world, you could operate on the packets in place in Haskell where
> >>possible and save the deserialization overhead...
> >
> >Data.ByteString.* for this.
> >
> >>Anyone trying to do any of this?
> >
> >Yeah, its a bit of a hot topic currently. :)
>
> >Gotta chase Erlang for hacking network data.
>
> Absolutely. Ta for the pointers anyway.

Yeah, so:

    Data.ByteString
    Data.Bits
    Data.Binary

Hack those bytes! Quickly! :-)

-- Don
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Philip Armstrong]

On Wed, Jul 04, 2007 at 09:44:13PM +1000, Donald Bruce Stewart wrote:

>Binary instances are pretty easy to write. For a simple data type:
>
>       > instance Binary Exp where
>       >       put (IntE i)          = do put (0 :: Word8)
>       >                                  put i
>       >       put (OpE s e1 e2)     = do put (1 :: Word8)
>       >                                  put s
>       >                                  put e1
>       >                                  put e2
>
>       >       get = do tag <- getWord8
>       >                case tag of
>       >                    0 -> liftM  IntE get
>       >                    1 -> liftM3 OpE  get get get

That's quite verbose! Plus I'm a bit concerned by the boxing implied
by those IntE / OpE constructors in get. If you were using those
values in a pattern match on the result of get, would the compiler be
able to eliminate them and refer directly to the values in the source
data?

>The Data.Binary comes with one tool to derive these. The DrIFT preprocessor
>also can, as can Stefan O'Rear's SYB deriver.
>
>I just write them by hand, or use the tool that comes with the lib.
>
>More docs here,
>    http://hackage.haskell.org/packages/archive/binary/0.3/doc/html/Data-Binary.html

This doesn't seem to deal with endianness. Am I missing something?

>> >>world, you could operate on the packets in place in Haskell where
>> >>possible and save the deserialization overhead...
>> >
>> >Data.ByteString.* for this.

Ah, does Data.Binary fuse with ByteString.* then?

>Hack those bytes! Quickly! :-)

:)

It's a shame the layout definition is so verbose. Erlang's is quite
compact. I wonder if something could be done with template haskell to
translate an Erlang-style data layout definition to the Data.Binary
form?

(Bonus points for being able to parse ASN.1 and generate appropriate
Haskell datatypes & serialization primitives automatically :-) )

Phil

--
http://www.kantaka.co.uk/ .oOo. public key: http://www.kantaka.co.uk/gpg.txt
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Bulat Ziganshin-2]

Hello Philip,

Wednesday, July 4, 2007, 5:50:42 PM, you wrote:
> This doesn't seem to deal with endianness. Am I missing something?

alternative:
http://haskell.org/haskellwiki/Library/AltBinary
http://haskell.org/haskellwiki/Library/Streams

--
Best regards,
 Bulat                            mailto:Bulat.Ziganshin@...

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Philip Armstrong]

On Wed, Jul 04, 2007 at 06:52:08PM +0400, Bulat Ziganshin wrote:
>Hello Philip,
>
>Wednesday, July 4, 2007, 5:50:42 PM, you wrote:
>> This doesn't seem to deal with endianness. Am I missing something?
>
>alternative:
>http://haskell.org/haskellwiki/Library/AltBinary
>http://haskell.org/haskellwiki/Library/Streams

Nice: bit aligning if you want it, little or big endian IO. Intermixed
endianness in the same datastream even[1]. However:

   3.2.10 Defining Binary instances for custom serialization formats (unwritten)

Does that mean that the code is unwritten or that the documentation is
unwritten. IAMFI :)

There seems to be some overlap between Streams and ByteStrings: Could
a Stream built on a ByteString backend benefit from all the fusion
work that's been put into ByteStrings recently? Oh wait, I see you
list that as 'future work' on the wiki page...

Phil

[1] Which sick application *needs* intermixed endianness?

--
http://www.kantaka.co.uk/ .oOo. public key: http://www.kantaka.co.uk/gpg.txt
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Malcolm Wallace]

Andrew Coppin <andrewcoppin@...> wrote:

> While we're on the subject... am I the first person to notice that
> Haskell doesn't appear to have much support for fiddling with streams
> of  bits?

See
  "The Bits Between The Lambdas: Binary Data in a Lazy Functional Language"
  Malcolm Wallace and Colin Runciman,
  International Symposium on Memory Management, Vancouver, Canada, Oct 1998
  ftp://ftp.cs.york.ac.uk/pub/malcolm/ismm98.html

Regards,
    Malcolm
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Bulat Ziganshin-2]

Hello Philip,

Wednesday, July 4, 2007, 7:31:56 PM, you wrote:

> On Wed, Jul 04, 2007 at 06:52:08PM +0400, Bulat Ziganshin wrote:
>>Hello Philip,
>>
>>Wednesday, July 4, 2007, 5:50:42 PM, you wrote:
>>> This doesn't seem to deal with endianness. Am I missing something?
>>
>>alternative:
>>http://haskell.org/haskellwiki/Library/AltBinary
>>http://haskell.org/haskellwiki/Library/Streams

> Nice: bit aligning if you want it, little or big endian IO. Intermixed
> endianness in the same datastream even[1]. However:

>    3.2.10 Defining Binary instances for custom serialization formats (unwritten)

> Does that mean that the code is unwritten or that the documentation is
> unwritten. IAMFI :)

of course all "unwritten" notes means unfinished docs. library
contains more than 100 functions so it was not easy to document them
all. you can browse sources, although probably it will not help too
much

in particular interest for you should be the following:
- Template Haskell can used to automatically derive new Binary instances

for specifying width of fields you may use int/word with specific
width, for example:

data Header = H Int32 Word16 Word8
$(deriveBinary ``Header)

(see DeriveBinary.hs example)

if you need to define instances manually, ask me

> There seems to be some overlap between Streams and ByteStrings: Could
> a Stream built on a ByteString backend benefit from all the fusion
> work that's been put into ByteStrings recently? Oh wait, I see you
> list that as 'future work' on the wiki page...

if you will write all popular words together, this probably will be
just a set of popular words, not something working :)  how fusion
should work together with serialization?

> [1] Which sick application *needs* intermixed endianness?

i just tried to implement everything possible :)


--
Best regards,
 Bulat                            mailto:Bulat.Ziganshin@...

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Stefan O'Rear]

On Wed, Jul 04, 2007 at 02:50:42PM +0100, Philip Armstrong wrote:

>> The Data.Binary comes with one tool to derive these. The DrIFT
>> preprocessor
>> also can, as can Stefan O'Rear's SYB deriver.
>>
>> I just write them by hand, or use the tool that comes with the lib.
>>
>> More docs here,
>>    
>> http://hackage.haskell.org/packages/archive/binary/0.3/doc/html/Data-Binary.html
>
> This doesn't seem to deal with endianness. Am I missing something?

The Data.Binary high level interface standardizes on 64-bit big endian.
The low level interface allows you to choose it yourself.

Stefan
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Philip Armstrong]

On Wed, Jul 04, 2007 at 09:15:59PM +0400, Bulat Ziganshin wrote:
>> Does that mean that the code is unwritten or that the documentation is
>> unwritten. IAMFI :)
>
>of course all "unwritten" notes means unfinished docs. library
>contains more than 100 functions so it was not easy to document them
>all. you can browse sources, although probably it will not help too
>much

OK.

>> There seems to be some overlap between Streams and ByteStrings: Could
>> a Stream built on a ByteString backend benefit from all the fusion
>> work that's been put into ByteStrings recently? Oh wait, I see you
>> list that as 'future work' on the wiki page...
>
>if you will write all popular words together, this probably will be
>just a set of popular words, not something working :)  how fusion
>should work together with serialization?

I'm thinking of the elimination of the boxing of values drawn out of
the input stream where possible, eg if I was writing a stream
processor that folded across the values in the input stream, it would
(presumably) be more efficient if the compiler noticed that the
function in question was (say) just reading Int values at offsets
within the stream, and could pass those as unboxed references in the
compiled code rather than freshly constructed values.

Fusion might be the wrong term: I was thinking by analogy with loop
fusion, with one of the loops was the 'data reading' loop. Does that
make sense?

>> [1] Which sick application *needs* intermixed endianness?
>
>i just tried to implement everything possible :)

Completeness is always good!

Thanks for the pointers,

Phil

--
http://www.kantaka.co.uk/ .oOo. public key: http://www.kantaka.co.uk/gpg.txt
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Andrew Coppin]

Philip Armstrong wrote:
> [1] Which sick application *needs* intermixed endianness?

*Clearly* you've never been to Singapore...



...er, I mean, "Ever tried playing with networking protocol stacks?"

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Bulat Ziganshin-2]

Hello Philip,

Wednesday, July 4, 2007, 9:41:27 PM, you wrote:

> I'm thinking of the elimination of the boxing of values drawn out of
> the input stream where possible, eg if I was writing a stream
> processor that folded across the values in the input stream, it would
> (presumably) be more efficient if the compiler noticed that the
> function in question was (say) just reading Int values at offsets
> within the stream, and could pass those as unboxed references in the
> compiled code rather than freshly constructed values.

it will depend on your code. the library doesn't make unnecessary
boxing, but (unlike Data.Binary?) it supports only monadic
(de)serialization. so there is no room for ByteString-like fusion
which pass unboxed data through several transformations. with my lib,
you can only read whole unboxed structure and then process it:

data T = C !Int32 !Word16

do x <- get h :: IO T
   processT $! x

--
Best regards,
 Bulat                            mailto:Bulat.Ziganshin@...

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Philip Armstrong]

On Wed, Jul 04, 2007 at 07:36:11PM +0100, Andrew Coppin wrote:
> Philip Armstrong wrote:
>> [1] Which sick application *needs* intermixed endianness?
>
> *Clearly* you've never been to Singapore...
>
> ...er, I mean, "Ever tried playing with networking protocol stacks?"

No (thankfully?).

Phil

--
http://www.kantaka.co.uk/ .oOo. public key: http://www.kantaka.co.uk/gpg.txt
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Donald Bruce Stewart]

phil:

> On Wed, Jul 04, 2007 at 09:44:13PM +1000, Donald Bruce Stewart wrote:
> >Binary instances are pretty easy to write. For a simple data type:
> >
> >      > instance Binary Exp where
> >      >       put (IntE i)          = do put (0 :: Word8)
> >      >                                  put i
> >      >       put (OpE s e1 e2)     = do put (1 :: Word8)
> >      >                                  put s
> >      >                                  put e1
> >      >                                  put e2
> >
> >      >       get = do tag <- getWord8
> >      >                case tag of
> >      >                    0 -> liftM  IntE get
> >      >                    1 -> liftM3 OpE  get get get
>
> That's quite verbose! Plus I'm a bit concerned by the boxing implied
> by those IntE / OpE constructors in get. If you were using those
> values in a pattern match on the result of get, would the compiler be
> able to eliminate them and refer directly to the values in the source
> data?

Well, here's you're flattening a Haskell structure, so it has to get
reboxed. If it was bytestring chunks, or Ints, then you can avoid any
serious copying. The 'get' just tags a value.

>
> >The Data.Binary comes with one tool to derive these. The DrIFT preprocessor
> >also can, as can Stefan O'Rear's SYB deriver.
> >
> >I just write them by hand, or use the tool that comes with the lib.
> >
> >More docs here,
> >   http://hackage.haskell.org/packages/archive/binary/0.3/doc/html/Data-Binary.html
>
> This doesn't seem to deal with endianness. Am I missing something?

That's the Haskell serialisation layer. Look at Data.Binary.Get/Put for
endian-primitives, to be used instead of 'get'. i.e. getWord16be

>
> >>>>world, you could operate on the packets in place in Haskell where
> >>>>possible and save the deserialization overhead...
> >>>
> >>>Data.ByteString.* for this.
>
> Ah, does Data.Binary fuse with ByteString.* then?

They know about each other, and Binary avoids copying if you're reading
ByteStrings.

>
> >Hack those bytes! Quickly! :-)
>
> :)
>
> It's a shame the layout definition is so verbose. Erlang's is quite
> compact. I wonder if something could be done with template haskell to
> translate an Erlang-style data layout definition to the Data.Binary
> form?

Right, simple but a bit verbose. The Erlang bit syntax is a nice pattern
matching/layout syntax for bit/byte data. There's a couple of ports of
this to Haskell -- one using pattern guards, another using Template
Haskell. Look on hackage.haskell.org for bitsyntax if you're interested.

> (Bonus points for being able to parse ASN.1 and generate appropriate
> Haskell datatypes & serialization primitives automatically :-) )

I think there's at least an ASN.1 definition in the crypto library.
Dominic might be able to enlighten us on that.

-- Don
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@...
http://www.haskell.org/mailman/listinfo/haskell-cafe