[Fwd: Re: Clean versus Haskell]

View: New views
7 Messages — Rating Filter:   Alert me  

[Fwd: Re: Clean versus Haskell]

by Adrian Hey :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


-------- Original Message --------
Subject: Re: [clean-list] Clean versus Haskell
Date: Thu, 15 Oct 2009 06:41:20 +0100
From: Adrian Hey <ahey@...>
To: Philippos Apolinarius <phi500ac@...>
References: <8400.82204.qm@...>

Hello Philippos,

GHC has a long standing performance bug for garbage collection and
mutable arrays:

http://hackage.haskell.org/trac/ghc/ticket/650

For some reason they haven't (can't?) fixed it. I guess the
authors of Haskell/ghc shootout entries are aware of this so don't
use native mutable arrays in their entries (at least not those that
show haskell/ghc to be "fast" :-)

Regards
--
Adrian Hey

Philippos Apolinarius wrote:

> I wrote a very simple program to check whether Haskell improved its array processing libraries or not. Here is how to compile and run the program arr.hs in Haskell (I have used the GHC compiler):
>
>> ghc -O arr.hs -o arr.exe
>
> $ time arr.exe +RTS -K32000000
> 2.8e8
>
> real    0m3.938s
> user    0m0.031s
> sys     0m0.000s
>
> The same program in Clean:
> C:\Clean 2.2\exemplos\console>arraytest.exe
> 280000000
> Execution: 0.01  Garbage collection: 0.01  Total: 0.03
>
> C:\Clean 2.2\exemplos\console>arraytest.exe
> 280000000
> Execution: 0.01  Garbage collection: 0.01  Total: 0.03
>
> This means that Clean is 390 times faster than Haskell in this particular problem. These results makes me worder whether Haskell is safer than Clean. It turns out that Haskell checks index out of range at runtime, exactly like Clean. Larger problems make the difference between Clean and Haskell even worse. For instance, neural networks like the one described in Schmidtt et al. run 400 times faster in Clean.
>
> Haskell seems to be slow, and not safe. For instance, GHC compiler does not at a program trying to write into a closed handle.
>
> module Main where
>  import System( getArgs )
>  import IO
>
>  main = do
>           args <- getArgs
>           if (length args /= 2)
>         then putStr "Usage: f1a f2a <n>"
>             else (do
>               fromHandle <- openFile (head args)  ReadMode
>               contents   <- hGetContents fromHandle
>               toHandle <- openFile (head (tail args)) WriteMode
>               hClose toHandle  -- Comment this line
>               hPutStr toHandle contents
>               hClose toHandle
>               putStr "Done.")
>
> The Clean equivalent program is somewhat smaller. In my opinion it is easier to understand. What is more important, Clean compiler balks at closed handles.
>
> module cleancopy
> import StdEnv, ArgEnv
>
> Start w
>   # argv= getCommandLine
>   | size argv < 3 = abort "Usage, etc."
>   # (ok, f, w)= fopen argv.[1] FReadText w
>       (contents, f)= freads f 64000
>       (ok, f, w)= fopen argv.[2] FWriteText w
>       f= fwrites contents f
>   = fclose f w
>
> Below you will find the array examples. You can check that Clean is really much faster than Haskell. I wonder why the Benchmarks Game site does not report such a large difference between Haskell and Clean performances. I believe that people who wrote Haskell benchmarks for the Benchmarks Game site cheated in using foreign pointers to access arrays.
>
> -- arr.hs
> import Control.Monad.ST
> import Data.Array.ST
> main = print $ runST
>           (do arr <- newArray (1,2000000)
>                         137.0 :: ST s
>                                   (STArray s
>                                     Int Double)
>               a <- readArray arr 1
>               b <- readArray arr 1
>               fn 2000000 arr 0.0 )
>
>
> fn i a acc | i < 1 = do (return acc)
> fn i a acc= do
>              b <- readArray a i
>              writeArray a i (b+3.0)
>              c <- readArray a i
>              fn (i-1) a (acc+c)
>
> //Clean version
> module arraytest
> import StdEnv
> fn i a acc | i<1 = acc
> fn i a=:{[i]=b} acc
>   # a= {a&[i]= b+3.0}
>   # (c, a)= a![i]
>   = fn (i-1) a (c+acc)
>  
> Start= fn 2000000 vt 0.0
> where
>    vt:: .{#Real}
>    vt = createArray 2000001 137.0
>
>
>
> --- On Mon, 10/12/09, rinus plasmeijer <rinus@...> wrote:
>
> From: rinus plasmeijer <rinus@...>
> Subject: [clean-list] Re: Clean
> To: "Henrique" <henrique_gusmao_@...>
> Cc: clean-list@...
> Received: Monday, October 12, 2009, 1:22 AM
>
>
>
>  
>  
>
>
> Hi Henrique
>
>  
>> Is this e-mail still working? http://clean.cs.ru.nl/
>  
>  
> Yes.
>
>  
>> I want to know more about Clean: news?
> updates?
>> What is the future of Clean?
>  
> See: http://wiki.clean.cs.ru.nl/Latest_developments
> We are working on a new version which allows you to
> mix Clean and Haskell 98 code.
> It is a lot of work, it will still take some time.
>  
>> Is Haskell killing Clean?
> Haskell is certainly much more used, which is also
> the raison for adding a Haskell front end.
>
>> I would like to
> learn and test Clean, and maybe then use it commercially.
>> Where do I
> can download it? I send e-mail to clean@... , asked for
> Clean at the site above, but I got no answer.
>
> We did not got your
> email.
> To download browse to: http://clean.cs.ru.nl/Download/main/main.htm
>  
>> Obs: I am using windows platform.
> Thats fine, it should work on any windows platform.
>  
>
> Greetings,
>  
> Rinus
>  
> -----Inline Attachment Follows-----
>
> _______________________________________________
> clean-list mailing list
> clean-list@...
> http://mailman.science.ru.nl/mailman/listinfo/clean-list
>
>
>
>       __________________________________________________________________
> Make your browsing faster, safer, and easier with the new Internet Explorer® 8. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email 
> ______________________________________________________________________
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> clean-list mailing list
> clean-list@...
> http://mailman.science.ru.nl/mailman/listinfo/clean-list


_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list

Re: [Fwd: Re: Clean versus Haskell]

by Adrian Hey :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello Philippos,

Adrian Hey wrote:
> GHC has a long standing performance bug for garbage collection and
> mutable arrays:
>
> http://hackage.haskell.org/trac/ghc/ticket/650

BTW, I forgot to mention that this bug does not affect unboxed arrays.
I see you've used a boxed STArray. You might get a fairer idea of ghc
performance using STUArray (at least for types that are unboxable).

Regards
--
Adrian Hey
_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list

RE: [Fwd: Re: Clean versus Haskell]

by Simon Peyton-Jones :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I submitted Philippos as a performance bug to GHC's trac, here:
        http://hackage.haskell.org/trac/ghc/ticket/3586

If you follow the link you'll see lots of commentary, including a version that generates code twice as fast as Clean's, and is purely functional.

That said, I think it's v bad that a straightforward program runs so slowly, and it's certainly true that this is an area we could pay more attention to.  (Trouble is, there are so many such areas!)

Meanwhile, I'm curious: are the arrays in Philippos's program strict?  Or lazy?  If strict, that's a pretty big difference.  (The "STU" arrays mentioned in the above link are strict and unboxed; that's what the "U" means.)

Simon

| -----Original Message-----
| From: clean-list-bounces@... [mailto:clean-list-bounces@...] On
| Behalf Of Adrian Hey
| Sent: 15 October 2009 07:33
| To: clean-list@...
| Subject: [Fwd: Re: [clean-list] Clean versus Haskell]
|
|
| -------- Original Message --------
| Subject: Re: [clean-list] Clean versus Haskell
| Date: Thu, 15 Oct 2009 06:41:20 +0100
| From: Adrian Hey <ahey@...>
| To: Philippos Apolinarius <phi500ac@...>
| References: <8400.82204.qm@...>
|
| Hello Philippos,
|
| GHC has a long standing performance bug for garbage collection and
| mutable arrays:
|
| http://hackage.haskell.org/trac/ghc/ticket/650
|
| For some reason they haven't (can't?) fixed it. I guess the
| authors of Haskell/ghc shootout entries are aware of this so don't
| use native mutable arrays in their entries (at least not those that
| show haskell/ghc to be "fast" :-)
|
| Regards
| --
| Adrian Hey
|
| Philippos Apolinarius wrote:
| > I wrote a very simple program to check whether Haskell improved its array
| processing libraries or not. Here is how to compile and run the program arr.hs in
| Haskell (I have used the GHC compiler):
| >
| >> ghc -O arr.hs -o arr.exe
| >
| > $ time arr.exe +RTS -K32000000
| > 2.8e8
| >
| > real    0m3.938s
| > user    0m0.031s
| > sys     0m0.000s
| >
| > The same program in Clean:
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01  Garbage collection: 0.01  Total: 0.03
| >
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01  Garbage collection: 0.01  Total: 0.03
| >
| > This means that Clean is 390 times faster than Haskell in this particular problem.
| These results makes me worder whether Haskell is safer than Clean. It turns out that
| Haskell checks index out of range at runtime, exactly like Clean. Larger problems
| make the difference between Clean and Haskell even worse. For instance, neural
| networks like the one described in Schmidtt et al. run 400 times faster in Clean.
| >
| > Haskell seems to be slow, and not safe. For instance, GHC compiler does not at a
| program trying to write into a closed handle.
| >
| > module Main where
| >  import System( getArgs )
| >  import IO
| >
| >  main = do
| >           args <- getArgs
| >           if (length args /= 2)
| >         then putStr "Usage: f1a f2a <n>"
| >             else (do
| >               fromHandle <- openFile (head args)  ReadMode
| >               contents   <- hGetContents fromHandle
| >               toHandle <- openFile (head (tail args)) WriteMode
| >               hClose toHandle  -- Comment this line
| >               hPutStr toHandle contents
| >               hClose toHandle
| >               putStr "Done.")
| >
| > The Clean equivalent program is somewhat smaller. In my opinion it is easier to
| understand. What is more important, Clean compiler balks at closed handles.
| >
| > module cleancopy
| > import StdEnv, ArgEnv
| >
| > Start w
| >   # argv= getCommandLine
| >   | size argv < 3 = abort "Usage, etc."
| >   # (ok, f, w)= fopen argv.[1] FReadText w
| >       (contents, f)= freads f 64000
| >       (ok, f, w)= fopen argv.[2] FWriteText w
| >       f= fwrites contents f
| >   = fclose f w
| >
| > Below you will find the array examples. You can check that Clean is really much
| faster than Haskell. I wonder why the Benchmarks Game site does not report such a
| large difference between Haskell and Clean performances. I believe that people who
| wrote Haskell benchmarks for the Benchmarks Game site cheated in using foreign
| pointers to access arrays.
| >
| > -- arr.hs
| > import Control.Monad.ST
| > import Data.Array.ST
| > main = print $ runST
| >           (do arr <- newArray (1,2000000)
| >                         137.0 :: ST s
| >                                   (STArray s
| >                                     Int Double)
| >               a <- readArray arr 1
| >               b <- readArray arr 1
| >               fn 2000000 arr 0.0 )
| >
| >
| > fn i a acc | i < 1 = do (return acc)
| > fn i a acc= do
| >              b <- readArray a i
| >              writeArray a i (b+3.0)
| >              c <- readArray a i
| >              fn (i-1) a (acc+c)
| >
| > //Clean version
| > module arraytest
| > import StdEnv
| > fn i a acc | i<1 = acc
| > fn i a=:{[i]=b} acc
| >   # a= {a&[i]= b+3.0}
| >   # (c, a)= a![i]
| >   = fn (i-1) a (c+acc)
| >
| > Start= fn 2000000 vt 0.0
| > where
| >    vt:: .{#Real}
| >    vt = createArray 2000001 137.0

_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list

RE: [Fwd: Re: Clean versus Haskell]

by Isaac Gouy :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message



--- On Thu, 10/15/09, Simon Peyton-Jones <simonpj@...> wrote:

> From: Simon Peyton-Jones <simonpj@...>
> Subject: RE: [Fwd: Re: [clean-list] Clean versus Haskell]
> To: "Adrian Hey" <ahey@...>, "clean-list@..." <clean-list@...>
> Date: Thursday, October 15, 2009, 12:22 AM
> I submitted Philippos as a
> performance bug to GHC's trac, here:
>     http://hackage.haskell.org/trac/ghc/ticket/3586
>
> If you follow the link you'll see lots of commentary,
> including a version that generates code twice as fast as
> Clean's, and is purely functional.
>
> That said, I think it's v bad that a straightforward
> program runs so slowly, and it's certainly true that this is
> an area we could pay more attention to.  (Trouble is,
> there are so many such areas!)
>
> Meanwhile, I'm curious: are the arrays in Philippos's
> program strict?  Or lazy?  If strict, that's a
> pretty big difference.  (The "STU" arrays mentioned in
> the above link are strict and unboxed; that's what the "U"
> means.)
>
> Simon


I'm curious too - does the answer depend on whether the strictness analyzer is enabled (strict context?) or is an unboxed array always strict?

strictness analyzer - off

"arraytest"
fn :: Int *(a Real) Real -> Real | Array a Real
vt :: .{#Real}
fn :: Int *{#Real} Real -> Real
Start :: Real

280000000
Execution: 1.81  Garbage collection: 0.01  Total: 1.82


strictness analyzer - on

"arraytest"
fn :: !Int *(a Real) !Real -> Real | Array a Real
vt :: .{#Real}
fn :: !Int *{#Real} !Real -> Real
Start :: Real

280000000
Execution: 0.15  Garbage collection: 0.00  Total: 0.15


     

_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list

RE: [Fwd: Re: Clean versus Haskell]

by Philippos Apolinarius :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi, Simon.

I ported the array benchmark to a slower machine, in order to detect smaller variations in speed. Clean compiler does not show huge differences in speed when one switches from boxed to unboxed arrays, or from lazy to strict arrays. As you can see below, adding unboxed annotation, or strictness annotation does not change performance noticeably. This is a feature that I like in Clean, and I would like to see in Haskell. Correct programs are almost equally fast. Therefore, the programmer does not need to worry about boxed, unboxed, strict or lazy applications. What I miss in Clean is an interpreter, like GHCi. I am writing a somewhat large genetic programming system to  diagnostic electric installations, and electronic circuits.  I mean, I am working for people who are writing the programs and designing the circuits. The programs are working in Lisp and Scheme (SBCL and Gambit). The Clean version is almost functional. I am struggling with the Haskell version. I will write a private letter to you explaining how the Clean program works. I wonder whether you could give me some hints on how to port it to Haskell. Since Haskell has an interpreter, I think that it could be used to test the generated applications, and calculate the fitness function.

/* BOXED LAZY
D:\Clean 2.2\geneticprog\bench>boxedlazy
280000000
Execution: 0.04  Garbage collection: 0.17  Total: 0.21

D:\Clean 2.2\geneticprog\bench>boxedlazy
280000000
Execution: 0.06  Garbage collection: 0.17  Total: 0.23

D:\Clean 2.2\geneticprog\bench>boxedlazy
280000000
Execution: 0.04  Garbage collection: 0.17  Total: 0.21
*/
module boxedlazy
import StdEnv

fn i a acc | i<1 = acc
fn i a=:{[i]=b} acc
   # a= {a&[i]= b+3.0}
   # (c, a)= a![i]
   = fn (i-1) a (c+acc)

Start= fn 2000000 vt 0.0
 where
    vt:: .{Real}
    vt = createArray 2000001 137.0
   

module arrays
import StdEnv

fn i a acc | i<1 = acc
fn i a=:{[i]=b} acc
   # a= {a&[i]= b+3.0}
   # (c, a)= a![i]
   = fn (i-1) a (c+acc)

Start= fn 2000000 vt 0.0
 where
    vt:: .{#Real}
    vt = createArray 2000001 137.0

/* unboxedlazy
D:\Clean 2.2\geneticprog\bench>unboxedlazy.exe
280000000
Execution: 0.03  Garbage collection: 0.00  Total: 0.03

D:\Clean 2.2\geneticprog\bench>unboxedlazy.exe
280000000
Execution: 0.04  Garbage collection: 0.00  Total: 0.04

D:\Clean 2.2\geneticprog\bench>unboxedlazy.exe
280000000
Execution: 0.03  Garbage collection: 0.00  Total: 0.03
*/
module unboxedlazy
import StdEnv

fn i a acc | i<1 = acc
fn i a=:{[i]=b} acc
   # a= {a&[i]= b+3.0}
   # (c, a)= a![i]
   = fn (i-1) a (c+acc)

Start= fn 2000000 vt 0.0
 where
    vt:: .{#Real}
    vt = createArray 2000001 137.0

/* unboxedstrict
D:\Clean 2.2\geneticprog\bench>unboxedstrict.exe
280000000
Execution: 0.03  Garbage collection: 0.00  Total: 0.03

D:\Clean 2.2\geneticprog\bench>unboxedstrict.exe
280000000
Execution: 0.04  Garbage collection: 0.00  Total: 0.04

D:\Clean 2.2\geneticprog\bench>unboxedstrict.exe
280000000
Execution: 0.04  Garbage collection: 0.00  Total: 0.04

*/
module unboxedstrict
import StdEnv

fn i a acc | i<1 = acc
fn i a=:{[i]=b} acc
   # a= {a&[i]= b+3.0}
   # (c, a)= a![i]
   = fn (i-1) a (c+acc)

Start= fn 2000000 vt 0.0
 where
    vt:: !.{#!Real}
    vt = createArray 2000001 137.0



--- On Thu, 10/15/09, Simon Peyton-Jones <simonpj@...> wrote:

From: Simon Peyton-Jones <simonpj@...>
Subject: RE: [Fwd: Re: [clean-list] Clean versus Haskell]
To: "Adrian Hey" <ahey@...>, "clean-list@..." <clean-list@...>
Received: Thursday, October 15, 2009, 1:22 AM

I submitted Philippos as a performance bug to GHC's trac, here:
    http://hackage.haskell.org/trac/ghc/ticket/3586

If you follow the link you'll see lots of commentary, including a version that generates code twice as fast as Clean's, and is purely functional.

That said, I think it's v bad that a straightforward program runs so slowly, and it's certainly true that this is an area we could pay more attention to.  (Trouble is, there are so many such areas!)

Meanwhile, I'm curious: are the arrays in Philippos's program strict?  Or lazy?  If strict, that's a pretty big difference.  (The "STU" arrays mentioned in the above link are strict and unboxed; that's what the "U" means.)

Simon

| -----Original Message-----
| From: clean-list-bounces@... [mailto:clean-list-bounces@...] On
| Behalf Of Adrian Hey
| Sent: 15 October 2009 07:33
| To: clean-list@...
| Subject: [Fwd: Re: [clean-list] Clean versus Haskell]
|
|
| -------- Original Message --------
| Subject: Re: [clean-list] Clean versus Haskell
| Date: Thu, 15 Oct 2009 06:41:20 +0100
| From: Adrian Hey <ahey@...>
| To: Philippos Apolinarius <phi500ac@...>
| References: <8400.82204.qm@...>
|
| Hello Philippos,
|
| GHC has a long standing performance bug for garbage collection and
| mutable arrays:
|
| http://hackage.haskell.org/trac/ghc/ticket/650
|
| For some reason they haven't (can't?) fixed it. I guess the
| authors of Haskell/ghc shootout entries are aware of this so don't
| use native mutable arrays in their entries (at least not those that
| show haskell/ghc to be "fast" :-)
|
| Regards
| --
| Adrian Hey
|
| Philippos Apolinarius wrote:
| > I wrote a very simple program to check whether Haskell improved its array
| processing libraries or not. Here is how to compile and run the program arr.hs in
| Haskell (I have used the GHC compiler):
| >
| >> ghc -O arr.hs -o arr.exe
| >
| > $ time arr.exe +RTS -K32000000
| > 2.8e8
| >
| > real    0m3.938s
| > user    0m0.031s
| > sys     0m0.000s
| >
| > The same program in Clean:
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01  Garbage collection: 0.01  Total: 0.03
| >
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01  Garbage collection: 0.01  Total: 0.03
| >
| > This means that Clean is 390 times faster than Haskell in this particular problem.
| These results makes me worder whether Haskell is safer than Clean. It turns out that
| Haskell checks index out of range at runtime, exactly like Clean. Larger problems
| make the difference between Clean and Haskell even worse. For instance, neural
| networks like the one described in Schmidtt et al. run 400 times faster in Clean.
| >
| > Haskell seems to be slow, and not safe. For instance, GHC compiler does not at a
| program trying to write into a closed handle.
| >
| > module Main where
| >  import System( getArgs )
| >  import IO
| >
| >  main = do
| >           args <- getArgs
| >           if (length args /= 2)
| >         then putStr "Usage: f1a f2a <n>"
| >             else (do
| >               fromHandle <- openFile (head args)  ReadMode
| >               contents   <- hGetContents fromHandle
| >               toHandle <- openFile (head (tail args)) WriteMode
| >               hClose toHandle  -- Comment this line
| >               hPutStr toHandle contents
| >               hClose toHandle
| >               putStr "Done.")
| >
| > The Clean equivalent program is somewhat smaller. In my opinion it is easier to
| understand. What is more important, Clean compiler balks at closed handles.
| >
| > module cleancopy
| > import StdEnv, ArgEnv
| >
| > Start w
| >   # argv= getCommandLine
| >   | size argv < 3 = abort "Usage, etc."
| >   # (ok, f, w)= fopen argv.[1] FReadText w
| >       (contents, f)= freads f 64000
| >       (ok, f, w)= fopen argv.[2] FWriteText w
| >       f= fwrites contents f
| >   = fclose f w
| >
| > Below you will find the array examples. You can check that Clean is really much
| faster than Haskell. I wonder why the Benchmarks Game site does not report such a
| large difference between Haskell and Clean performances. I believe that people who
| wrote Haskell benchmarks for the Benchmarks Game site cheated in using foreign
| pointers to access arrays.
| >
| > -- arr.hs
| > import Control.Monad.ST
| > import Data.Array.ST
| > main = print $ runST
| >           (do arr <- newArray (1,2000000)
| >                         137.0 :: ST s
| >                                   (STArray s
| >                                     Int Double)
| >               a <- readArray arr 1
| >               b <- readArray arr 1
| >               fn 2000000 arr 0.0 )
| >
| >
| > fn i a acc | i < 1 = do (return acc)
| > fn i a acc= do
| >              b <- readArray a i
| >              writeArray a i (b+3.0)
| >              c <- readArray a i
| >              fn (i-1) a (acc+c)
| >
| > //Clean version
| > module arraytest
| > import StdEnv
| > fn i a acc | i<1 = acc
| > fn i a=:{[i]=b} acc
| >   # a= {a&[i]= b+3.0}
| >   # (c, a)= a![i]
| >   = fn (i-1) a (c+acc)
| >
| > Start= fn 2000000 vt 0.0
| > where
| >    vt:: .{#Real}
| >    vt = createArray 2000001 137.0

_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list


Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail
_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list

More realistic benchmark (I)

by Philippos Apolinarius :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Dear Dr. Simon.
I believe that I oversimplified the benchmark. However, since the suggestions improved my program a lot, I am sending you a more realistic version. Here are my comments on the link you have sent me:

1 --- The functional approach does not work for the real stuff, because I use random processes to access the array. I am sending you a more realistic program in the next email.

2 --- Although my previous example works with unboxed arrays, what I am really storing are Trees, that represent electronic circuits.

3 --- I would rather not use unsafeRead and unsafeWrite. I don't use them in Clean, and I think I should not use them in Haskell.

Here is the first program, where I replaced Double by a Tree (Windows). I will explain why I did this in a separate email, to prevent confusion. Anyway, now Haskell is 23 times slower than Clean, as you can check from my numbers, or repeating the benchmark. BTW, I tried to use unsafeRead, and unsafeWrite, but I got a segmentation fault when I tried to run the program.

/* Clean version
C:\Clean 2.2\geneticprog\bench>population.exe
280000000
Execution: 0.07  Garbage collection: 0.18  Total: 0.26

C:\Clean 2.2\geneticprog\bench>population.exe
280000000
Execution: 0.07  Garbage collection: 0.17  Total: 0.25
*/
module population
import StdEnv

:: Op = AND | OR | NOT;
:: Tree= L Real | T Op [Tree];

fn i a acc | i<1 = acc
fn i a=:{[i]=b} acc
  # (L n)= b
  # a= {a&[i]= L (n+3.0)}
  # (c, a)= a![i]
  # (L x)= c
  = fn (i-1) a (x+acc)

Start= fn 2000000 vt 0.0
where
   vt:: .{Tree}
   vt = createArray 2000001 (L 137.0)


{-# Haskell version #-}
{-# OPTIONS -fvia-C -optc-O3 -fexcess-precision -optc-msse3 #-}

import Control.Monad.ST
import Data.Array.ST
import Data.Array.Base

data Op = AND | OR | NOT;
data Tree= L Double | T Op [Tree]

main = print $ runST
          (do arr <- newArray (1,2000000)
                        (L 137.0) :: ST s
                                     (STArray s
                                        Int Tree)
              go  arr 2000000 0.0 )

go ::  STArray s Int Tree -> Int -> Double -> ST s Double
go !a i !acc
  | i < 1 = return acc
  | otherwise=do
               b <- readArray a i
               writeArray a i (setDouble ((getDouble b)+3.0))
               c <- readArray a i
               go  a (i-1) (acc+ (getDouble c))

getDouble (L r)= r
getDouble _ = 0.0

setDouble r= L r

{-# D:\Programs\ghc-programs\bench>ghc -O2 --make \
  bingo.hs -fvia-C -optc-O3 -optc-msse3 -o bingo.exe

D:\ghc-programs\bench>bingo.exe +RTS -sstderr -K32000000
bingo.exe +RTS -sstderr -K32000000
2.8e8
      72,392,664 bytes allocated in the heap
     118,966,296 bytes copied during GC
      37,490,648 bytes maximum residency (4 sample(s))
         416,176 bytes maximum slop
              66 MB total memory in use (1 MB lost due to fragmentation)

  Generation 0:   119 collections,     0 parallel,  1.42s,  1.42s elapsed
  Generation 1:     4 collections,     0 parallel,  0.14s,  0.16s elapsed

  INIT  time    0.03s  (  0.00s elapsed)
  MUT   time    0.06s  (  0.06s elapsed)
  GC    time    1.56s  (  1.58s elapsed)
  EXIT  time    0.00s  (  0.02s elapsed)
  Total time    1.66s  (  1.64s elapsed)

  %GC time      94.3%  (96.2% elapsed)

  Alloc rate    772,188,416 bytes per MUT second

  Productivity   3.8% of total user, 3.8% of total elapsed
#-}


--- On Thu, 10/15/09, Simon Peyton-Jones <simonpj@...> wrote:

From: Simon Peyton-Jones <simonpj@...>
Subject: RE: [Fwd: Re: [clean-list] Clean versus Haskell]
To: "Adrian Hey" <ahey@...>, "clean-list@..." <clean-list@...>
Received: Thursday, October 15, 2009, 1:22 AM

I submitted Philippos as a performance bug to GHC's trac, here:
    http://hackage.haskell.org/trac/ghc/ticket/3586

If you follow the link you'll see lots of commentary, including a version that generates code twice as fast as Clean's, and is purely functional.

That said, I think it's v bad that a straightforward program runs so slowly, and it's certainly true that this is an area we could pay more attention to.  (Trouble is, there are so many such areas!)

Meanwhile, I'm curious: are the arrays in Philippos's program strict?  Or lazy?  If strict, that's a pretty big difference.  (The "STU" arrays mentioned in the above link are strict and unboxed; that's what the "U" means.)

Simon

| -----Original Message-----
| From: clean-list-bounces@... [mailto:clean-list-bounces@...] On
| Behalf Of Adrian Hey
| Sent: 15 October 2009 07:33
| To: clean-list@...
| Subject: [Fwd: Re: [clean-list] Clean versus Haskell]
|
|
| -------- Original Message --------
| Subject: Re: [clean-list] Clean versus Haskell
| Date: Thu, 15 Oct 2009 06:41:20 +0100
| From: Adrian Hey <ahey@...>
| To: Philippos Apolinarius <phi500ac@...>
| References: <8400.82204.qm@...>
|
| Hello Philippos,
|
| GHC has a long standing performance bug for garbage collection and
| mutable arrays:
|
| http://hackage.haskell.org/trac/ghc/ticket/650
|
| For some reason they haven't (can't?) fixed it. I guess the
| authors of Haskell/ghc shootout entries are aware of this so don't
| use native mutable arrays in their entries (at least not those that
| show haskell/ghc to be "fast" :-)
|
| Regards
| --
| Adrian Hey
|
| Philippos Apolinarius wrote:
| > I wrote a very simple program to check whether Haskell improved its array
| processing libraries or not. Here is how to compile and run the program arr.hs in
| Haskell (I have used the GHC compiler):
| >
| >> ghc -O arr.hs -o arr.exe
| >
| > $ time arr.exe +RTS -K32000000
| > 2.8e8
| >
| > real    0m3.938s
| > user    0m0.031s
| > sys     0m0.000s
| >
| > The same program in Clean:
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01  Garbage collection: 0.01  Total: 0.03
| >
| > C:\Clean 2.2\exemplos\console>arraytest.exe
| > 280000000
| > Execution: 0.01  Garbage collection: 0.01  Total: 0.03
| >
| > This means that Clean is 390 times faster than Haskell in this particular problem.
| These results makes me worder whether Haskell is safer than Clean. It turns out that
| Haskell checks index out of range at runtime, exactly like Clean. Larger problems
| make the difference between Clean and Haskell even worse. For instance, neural
| networks like the one described in Schmidtt et al. run 400 times faster in Clean.
| >
| > Haskell seems to be slow, and not safe. For instance, GHC compiler does not at a
| program trying to write into a closed handle.
| >
| > module Main where
| >  import System( getArgs )
| >  import IO
| >
| >  main = do
| >           args <- getArgs
| >           if (length args /= 2)
| >         then putStr "Usage: f1a f2a <n>"
| >             else (do
| >               fromHandle <- openFile (head args)  ReadMode
| >               contents   <- hGetContents fromHandle
| >               toHandle <- openFile (head (tail args)) WriteMode
| >               hClose toHandle  -- Comment this line
| >               hPutStr toHandle contents
| >               hClose toHandle
| >               putStr "Done.")
| >
| > The Clean equivalent program is somewhat smaller. In my opinion it is easier to
| understand. What is more important, Clean compiler balks at closed handles.
| >
| > module cleancopy
| > import StdEnv, ArgEnv
| >
| > Start w
| >   # argv= getCommandLine
| >   | size argv < 3 = abort "Usage, etc."
| >   # (ok, f, w)= fopen argv.[1] FReadText w
| >       (contents, f)= freads f 64000
| >       (ok, f, w)= fopen argv.[2] FWriteText w
| >       f= fwrites contents f
| >   = fclose f w
| >
| > Below you will find the array examples. You can check that Clean is really much
| faster than Haskell. I wonder why the Benchmarks Game site does not report such a
| large difference between Haskell and Clean performances. I believe that people who
| wrote Haskell benchmarks for the Benchmarks Game site cheated in using foreign
| pointers to access arrays.
| >
| > -- arr.hs
| > import Control.Monad.ST
| > import Data.Array.ST
| > main = print $ runST
| >           (do arr <- newArray (1,2000000)
| >                         137.0 :: ST s
| >                                   (STArray s
| >                                     Int Double)
| >               a <- readArray arr 1
| >               b <- readArray arr 1
| >               fn 2000000 arr 0.0 )
| >
| >
| > fn i a acc | i < 1 = do (return acc)
| > fn i a acc= do
| >              b <- readArray a i
| >              writeArray a i (b+3.0)
| >              c <- readArray a i
| >              fn (i-1) a (acc+c)
| >
| > //Clean version
| > module arraytest
| > import StdEnv
| > fn i a acc | i<1 = acc
| > fn i a=:{[i]=b} acc
| >   # a= {a&[i]= b+3.0}
| >   # (c, a)= a![i]
| >   = fn (i-1) a (c+acc)
| >
| > Start= fn 2000000 vt 0.0
| > where
| >    vt:: .{#Real}
| >    vt = createArray 2000001 137.0

_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list


Yahoo! Canada Toolbar : Search from anywhere on the web and bookmark your favourite sites. Download it now!


_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list

RE: [Fwd: Re: Clean versus Haskell]

by John van Groningen :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Isaac Gouy wrote:
>..
>I'm curious too - does the answer depend on whether the strictness analyzer is enabled (strict context?) or is an unboxed array always strict?

An unboxed array is always strict (but not hyperstrict), the array element
is evaluated to root normal form (and unboxed) before it is stored in
the array.

>..

Kind regards,

John van Groningen
_______________________________________________
clean-list mailing list
clean-list@...
http://mailman.science.ru.nl/mailman/listinfo/clean-list