Comparison of FreeBSD/Linux TCP Throughput performance

[Hongtao Yin]

Hi,

 

I compared TCP performance between FreeBSD and Linux by running test tools
Netperf and Iperf with Intel NIC.

The kernels are full version and default values are used in the testing
except TCP Congestion Control algorithm set to Reno.

>From the test results we can see Linux TCP performance in throughput is
better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
throughput is only 43% of Linux's.

 

I like to get some feedback if anyone did similar comparison test, or knows
any issues with kernels or drivers. Thanks lot.

 

FreeBSD and Linux Sysctl captures are attached for reference.

 

Regards,

Hongtao

 

 

 

Test Environments:

PC: Dell Precision T3400 (same 4 PCs)

CPU: Intel Core 2 Duo CPU E4600@...

FreeBSD: V7.1 (full version) (TCP CC: newReno)

Linux: V2.6.31.1 (full version) (TCP CC: Reno)

Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter (Gigabit)
chip 82546EB (only one port used for each PC)

Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)

Iperf: V2.0.4

Netperf: V2.4.4

 

Setup:

 

 

                                  ----------

                                  | switch |

                                  ----------

               ---------------------| | | |--------------------

               |                      | |                     |

               |              --------| |--------             |

               |              |                 |             |

               |              |                 |             |

        --------------  --------------  --------------  --------------

        |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |

        |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |

        |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|

        --------------  --------------  --------------  --------------

 

 

 

 

 

================================

     Netperf Test Results  

================================

 

TCP Throughput Test

-------------------

 

PC2/4: #netserver -p 22113

PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10

 

          Recv             Send            Send           Elapsed
Throughput

          Socket           Socket          Message        Time
10^6 bits/sec

          Size             Size            Size           Sec.

          bytes            bytes           bytes

FreeBSD:  65536            32768           32768          10.34
598.11

Linux:    87380            16384           16384          10.04
779.02

 

 

PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
64/128/256/512/1024/2048/4096

 

          Recv             Send            Send           Elapsed
Throughput

          Socket           Socket          Message        Time
10^6 bits/sec

          Size             Size            Size           Sec.

          bytes            bytes           bytes

FreeBSD:  65536            32768           64              10.19
417.10

          65536            32768           128             10.35
336.63

          65536            32768           256             10.36
576.99

          65536            32768           512             10.35
569.79

          65536            32768           1024            10.35
553.70

          65536            32768           2048            10.35
584.20

          65536            32768           4096            10.35
602.45

 

Linux:    87380            16384           64              10.03
778.21

          87380            16384           128             10.03
779.72

          87380            16384           256             10.04
780.16

          87380            16384           512             10.03
776.85

          87380            16384           1024            10.04
777.52

          87380            16384           2048            10.04
777.83

          87380            16384           4096            10.03
780.17

 

 

 

 

===============================

      Iperf Test Results

===============================

 

Bandwidth Test

--------------

 

PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20

 

           Interval        Transfer        Bandwidth

           sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        740             600

Linux:     0.0-10.0        972             815

 

 

PC1/3: #iperf -c 192.168.1.20 -d

 

           Interval        Transfer        Bandwidth

           sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.0        402             337

           0.0-10.0        404             338

 

Linux:     0.0-10.0        926             776

           0.0-10.0        44.1            36.9

 

 

Parallel Test

-------------

 

PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20 -P 2

 

           Interval        Transfer        Bandwidth

           sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        370             300

           0.0-10.3        370             300

      SUM: 0.0-10.3        739             600

 

Linux:     0.0-10.0        479             402

           0.0-10.0        473             396

      SUM: 0.0-10.0        952             797

 

 

 




_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[O. Hartmann-4]

Hongtao Yin wrote:

> Hi,
>
>  
>
> I compared TCP performance between FreeBSD and Linux by running test tools
> Netperf and Iperf with Intel NIC.
>
> The kernels are full version and default values are used in the testing
> except TCP Congestion Control algorithm set to Reno.
>
>>From the test results we can see Linux TCP performance in throughput is
> better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
> throughput is only 43% of Linux's.
>
>  
>
> I like to get some feedback if anyone did similar comparison test, or knows
> any issues with kernels or drivers. Thanks lot.
>
>  
>
> FreeBSD and Linux Sysctl captures are attached for reference.
>
>  
>
> Regards,
>
> Hongtao
>
>  
>
>  
>
>  
>
> Test Environments:
>
> PC: Dell Precision T3400 (same 4 PCs)
>
> CPU: Intel Core 2 Duo CPU E4600@...
>
> FreeBSD: V7.1 (full version) (TCP CC: newReno)
>
> Linux: V2.6.31.1 (full version) (TCP CC: Reno)
>
> Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter (Gigabit)
> chip 82546EB (only one port used for each PC)
>
> Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)
>
> Iperf: V2.0.4
>
> Netperf: V2.4.4
>
>  
>
> Setup:
>
>  
>
>  
>
>                                   ----------
>
>                                   | switch |
>
>                                   ----------
>
>                ---------------------| | | |--------------------
>
>                |                      | |                     |
>
>                |              --------| |--------             |
>
>                |              |                 |             |
>
>                |              |                 |             |
>
>         --------------  --------------  --------------  --------------
>
>         |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |
>
>         |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |
>
>         |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|
>
>         --------------  --------------  --------------  --------------
>
>  
>
>  
>
>  
>
>  
>
>  
>
> ================================
>
>      Netperf Test Results  
>
> ================================
>
>  
>
> TCP Throughput Test
>
> -------------------
>
>  
>
> PC2/4: #netserver -p 22113
>
> PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10
>
>  
>
>           Recv             Send            Send           Elapsed
> Throughput
>
>           Socket           Socket          Message        Time
> 10^6 bits/sec
>
>           Size             Size            Size           Sec.
>
>           bytes            bytes           bytes
>
> FreeBSD:  65536            32768           32768          10.34
> 598.11
>
> Linux:    87380            16384           16384          10.04
> 779.02
>
>  
>
>  
>
> PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
> 64/128/256/512/1024/2048/4096
>
>  
>
>           Recv             Send            Send           Elapsed
> Throughput
>
>           Socket           Socket          Message        Time
> 10^6 bits/sec
>
>           Size             Size            Size           Sec.
>
>           bytes            bytes           bytes
>
> FreeBSD:  65536            32768           64              10.19
> 417.10
>
>           65536            32768           128             10.35
> 336.63
>
>           65536            32768           256             10.36
> 576.99
>
>           65536            32768           512             10.35
> 569.79
>
>           65536            32768           1024            10.35
> 553.70
>
>           65536            32768           2048            10.35
> 584.20
>
>           65536            32768           4096            10.35
> 602.45
>
>  
>
> Linux:    87380            16384           64              10.03
> 778.21
>
>           87380            16384           128             10.03
> 779.72
>
>           87380            16384           256             10.04
> 780.16
>
>           87380            16384           512             10.03
> 776.85
>
>           87380            16384           1024            10.04
> 777.52
>
>           87380            16384           2048            10.04
> 777.83
>
>           87380            16384           4096            10.03
> 780.17
>
>  
>
>  
>
>  
>
>  
>
> ===============================
>
>       Iperf Test Results
>
> ===============================
>
>  
>
> Bandwidth Test
>
> --------------
>
>  
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20
>
>  
>
>            Interval        Transfer        Bandwidth
>
>            sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        740             600
>
> Linux:     0.0-10.0        972             815
>
>  
>
>  
>
> PC1/3: #iperf -c 192.168.1.20 -d
>
>  
>
>            Interval        Transfer        Bandwidth
>
>            sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.0        402             337
>
>            0.0-10.0        404             338
>
>  
>
> Linux:     0.0-10.0        926             776
>
>            0.0-10.0        44.1            36.9
>
>  
>
>  
>
> Parallel Test
>
> -------------
>
>  
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20 -P 2
>
>  
>
>            Interval        Transfer        Bandwidth
>
>            sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        370             300
>
>            0.0-10.3        370             300
>
>       SUM: 0.0-10.3        739             600
>
>  
>
> Linux:     0.0-10.0        479             402
>
>            0.0-10.0        473             396
>
>       SUM: 0.0-10.0        952             797
>
>  
>
>  
>
>  
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> freebsd-performance@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

FreeBSD 7.1 is quite old compared to Linux 2.6.31 - I'd like to see at
least FreeBSD 7.2 compared, if not 8.0-RC1. Maybe also the most recent
FreeBSD 4 should be taken into this test.

_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[István-3]

use netpipe TCP pls.
And graph it.

Thanks in advance,
Istvan

On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin <htyin@...> wrote:

> Hi,
>
>
>
> I compared TCP performance between FreeBSD and Linux by running test tools
> Netperf and Iperf with Intel NIC.
>
> The kernels are full version and default values are used in the testing
> except TCP Congestion Control algorithm set to Reno.
>
> >From the test results we can see Linux TCP performance in throughput is
> better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
> throughput is only 43% of Linux's.
>
>
>
> I like to get some feedback if anyone did similar comparison test, or knows
> any issues with kernels or drivers. Thanks lot.
>
>
>
> FreeBSD and Linux Sysctl captures are attached for reference.
>
>
>
> Regards,
>
> Hongtao
>
>
>
>
>
>
>
> Test Environments:
>
> PC: Dell Precision T3400 (same 4 PCs)
>
> CPU: Intel Core 2 Duo CPU E4600@...
>
> FreeBSD: V7.1 (full version) (TCP CC: newReno)
>
> Linux: V2.6.31.1 (full version) (TCP CC: Reno)
>
> Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter
> (Gigabit)
> chip 82546EB (only one port used for each PC)
>
> Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)
>
> Iperf: V2.0.4
>
> Netperf: V2.4.4
>
>
>
> Setup:
>
>
>
>
>
>                                  ----------
>
>                                  | switch |
>
>                                  ----------
>
>               ---------------------| | | |--------------------
>
>               |                      | |                     |
>
>               |              --------| |--------             |
>
>               |              |                 |             |
>
>               |              |                 |             |
>
>        --------------  --------------  --------------  --------------
>
>        |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |
>
>        |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |
>
>        |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|
>
>        --------------  --------------  --------------  --------------
>
>
>
>
>
>
>
>
>
>
>
> ================================
>
>     Netperf Test Results
>
> ================================
>
>
>
> TCP Throughput Test
>
> -------------------
>
>
>
> PC2/4: #netserver -p 22113
>
> PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10
>
>
>
>          Recv             Send            Send           Elapsed
> Throughput
>
>          Socket           Socket          Message        Time
> 10^6 bits/sec
>
>          Size             Size            Size           Sec.
>
>          bytes            bytes           bytes
>
> FreeBSD:  65536            32768           32768          10.34
> 598.11
>
> Linux:    87380            16384           16384          10.04
> 779.02
>
>
>
>
>
> PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
> 64/128/256/512/1024/2048/4096
>
>
>
>          Recv             Send            Send           Elapsed
> Throughput
>
>          Socket           Socket          Message        Time
> 10^6 bits/sec
>
>          Size             Size            Size           Sec.
>
>          bytes            bytes           bytes
>
> FreeBSD:  65536            32768           64              10.19
> 417.10
>
>          65536            32768           128             10.35
> 336.63
>
>          65536            32768           256             10.36
> 576.99
>
>          65536            32768           512             10.35
> 569.79
>
>          65536            32768           1024            10.35
> 553.70
>
>          65536            32768           2048            10.35
> 584.20
>
>          65536            32768           4096            10.35
> 602.45
>
>
>
> Linux:    87380            16384           64              10.03
> 778.21
>
>          87380            16384           128             10.03
> 779.72
>
>          87380            16384           256             10.04
> 780.16
>
>          87380            16384           512             10.03
> 776.85
>
>          87380            16384           1024            10.04
> 777.52
>
>          87380            16384           2048            10.04
> 777.83
>
>          87380            16384           4096            10.03
> 780.17
>
>
>
>
>
>
>
>
>
> ===============================
>
>      Iperf Test Results
>
> ===============================
>
>
>
> Bandwidth Test
>
> --------------
>
>
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20
>
>
>
>           Interval        Transfer        Bandwidth
>
>           sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        740             600
>
> Linux:     0.0-10.0        972             815
>
>
>
>
>
> PC1/3: #iperf -c 192.168.1.20 -d
>
>
>
>           Interval        Transfer        Bandwidth
>
>           sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.0        402             337
>
>           0.0-10.0        404             338
>
>
>
> Linux:     0.0-10.0        926             776
>
>           0.0-10.0        44.1            36.9
>
>
>
>
>
> Parallel Test
>
> -------------
>
>
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20 -P 2
>
>
>
>           Interval        Transfer        Bandwidth
>
>           sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        370             300
>
>           0.0-10.3        370             300
>
>      SUM: 0.0-10.3        739             600
>
>
>
> Linux:     0.0-10.0        479             402
>
>           0.0-10.0        473             396
>
>      SUM: 0.0-10.0        952             797
>
>
>
>
>
>
>
>
> _______________________________________________
> freebsd-performance@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "
> freebsd-performance-unsubscribe@..."
>

--
the sun shines for all
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Adrian Chadd-2]

2009/10/15 Hongtao Yin <htyin@...>:
> Hi,
>
>
>
> I compared TCP performance between FreeBSD and Linux by running test tools
> Netperf and Iperf with Intel NIC.

Did you compare syscalls made and time taken?

For example, do either/both of them do a lot of gettimeofday() calls?
FreeBSD and Linux have (had?) different behaviours and performance
with those.

I'd suggest digging a bit deeper? :)


adrian
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Steve Dong]

 
There are two gettimeofday calls, one at the beginning of the test and the
other at the end. netperf-2.4.4 is used. I believe gettimeofday is a
vsyscall in linux, but a real system call in freebsd.

Thanks,
Steve

 

-----Original Message-----
From: owner-freebsd-performance@...
[mailto:owner-freebsd-performance@...] On Behalf Of Adrian Chadd
Sent: Thursday, October 15, 2009 1:55 PM
To: Hongtao Yin
Cc: freebsd-performance@...
Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance

2009/10/15 Hongtao Yin <htyin@...>:
> Hi,
>
>
>
> I compared TCP performance between FreeBSD and Linux by running test
> tools Netperf and Iperf with Intel NIC.

Did you compare syscalls made and time taken?

For example, do either/both of them do a lot of gettimeofday() calls?
FreeBSD and Linux have (had?) different behaviours and performance with
those.

I'd suggest digging a bit deeper? :)


adrian
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribe@..."

_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Hongtao Yin]

Istvan,

 

I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we collected
test data and the result shows that FreeBSD TCP performance is worse than
Linux. I got troubles to plot using gnuplot, so here I am attaching the raw
data files.

The netpipe commands are as follows:

PC2: #NPtcp

PC1: #NPtcp �Ch 192.168.1.20

 

Regards,

Hongtao

 

 

 

  _____  

From: István [mailto:leccine@...]
Sent: 2009年10月15日 5:13
To: Hongtao Yin
Cc: freebsd-performance@...
Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance

 

use netpipe TCP pls.

 

And graph it.

 

Thanks in advance,

Istvan

On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin <htyin@...> wrote:

Hi,



I compared TCP performance between FreeBSD and Linux by running test tools
Netperf and Iperf with Intel NIC.

The kernels are full version and default values are used in the testing
except TCP Congestion Control algorithm set to Reno.

>From the test results we can see Linux TCP performance in throughput is
better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
throughput is only 43% of Linux's.



I like to get some feedback if anyone did similar comparison test, or knows
any issues with kernels or drivers. Thanks lot.



FreeBSD and Linux Sysctl captures are attached for reference.



Regards,

Hongtao







Test Environments:

PC: Dell Precision T3400 (same 4 PCs)

CPU: Intel Core 2 Duo CPU E4600@...

FreeBSD: V7.1 (full version) (TCP CC: newReno)

Linux: V2.6.31.1 (full version) (TCP CC: Reno)

Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter (Gigabit)
chip 82546EB (only one port used for each PC)

Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)

Iperf: V2.0.4

Netperf: V2.4.4



Setup:





                                 ----------

                                 | switch |

                                 ----------

              ---------------------| | | |--------------------

              |                      | |                     |

              |              --------| |--------             |

              |              |                 |             |

              |              |                 |             |

       --------------  --------------  --------------  --------------

       |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |

       |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |

       |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|

       --------------  --------------  --------------  --------------











================================

    Netperf Test Results

================================



TCP Throughput Test

-------------------



PC2/4: #netserver -p 22113

PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           32768          10.34
598.11

Linux:    87380            16384           16384          10.04
779.02





PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
64/128/256/512/1024/2048/4096



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           64              10.19
417.10

         65536            32768           128             10.35
336.63

         65536            32768           256             10.36
576.99

         65536            32768           512             10.35
569.79

         65536            32768           1024            10.35
553.70

         65536            32768           2048            10.35
584.20

         65536            32768           4096            10.35
602.45



Linux:    87380            16384           64              10.03
778.21

         87380            16384           128             10.03
779.72

         87380            16384           256             10.04
780.16

         87380            16384           512             10.03
776.85

         87380            16384           1024            10.04
777.52

         87380            16384           2048            10.04
777.83

         87380            16384           4096            10.03
780.17









===============================

     Iperf Test Results

===============================



Bandwidth Test

--------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        740             600

Linux:     0.0-10.0        972             815





PC1/3: #iperf -c 192.168.1.20 -d



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.0        402             337

          0.0-10.0        404             338



Linux:     0.0-10.0        926             776

          0.0-10.0        44.1            36.9





Parallel Test

-------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20 -P 2



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        370             300

          0.0-10.3        370             300

     SUM: 0.0-10.3        739             600



Linux:     0.0-10.0        479             402

          0.0-10.0        473             396

     SUM: 0.0-10.0        952             797








_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribe@..."




--
the sun shines for all




_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Steve Dong]

Here are graphs from the netpipe test results with 8.0 RC1
 

Thanks,
Steve

 

-----Original Message-----
From: owner-freebsd-performance@...
[mailto:owner-freebsd-performance@...] On Behalf Of Hongtao Yin
Sent: Thursday, October 15, 2009 8:52 PM
To: 'István'
Cc: freebsd-performance@...
Subject: RE: Comparison of FreeBSD/Linux TCP Throughput performance

Istvan,

 

I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we collected
test data and the result shows that FreeBSD TCP performance is worse than
Linux. I got troubles to plot using gnuplot, so here I am attaching the raw
data files.

The netpipe commands are as follows:

PC2: #NPtcp

PC1: #NPtcp �Ch 192.168.1.20

 

Regards,

Hongtao

 

 

 

  _____  

From: István [mailto:leccine@...]
Sent: 2009年10月15日 5:13
To: Hongtao Yin
Cc: freebsd-performance@...
Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance

 

use netpipe TCP pls.

 

And graph it.

 

Thanks in advance,

Istvan

On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin <htyin@...> wrote:

Hi,



I compared TCP performance between FreeBSD and Linux by running test tools
Netperf and Iperf with Intel NIC.

The kernels are full version and default values are used in the testing
except TCP Congestion Control algorithm set to Reno.

>From the test results we can see Linux TCP performance in throughput is
better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
throughput is only 43% of Linux's.



I like to get some feedback if anyone did similar comparison test, or knows
any issues with kernels or drivers. Thanks lot.



FreeBSD and Linux Sysctl captures are attached for reference.



Regards,

Hongtao







Test Environments:

PC: Dell Precision T3400 (same 4 PCs)

CPU: Intel Core 2 Duo CPU E4600@...

FreeBSD: V7.1 (full version) (TCP CC: newReno)

Linux: V2.6.31.1 (full version) (TCP CC: Reno)

Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter (Gigabit)
chip 82546EB (only one port used for each PC)

Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)

Iperf: V2.0.4

Netperf: V2.4.4



Setup:





                                 ----------

                                 | switch |

                                 ----------

              ---------------------| | | |--------------------

              |                      | |                     |

              |              --------| |--------             |

              |              |                 |             |

              |              |                 |             |

       --------------  --------------  --------------  --------------

       |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |

       |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |

       |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|

       --------------  --------------  --------------  --------------











================================

    Netperf Test Results

================================



TCP Throughput Test

-------------------



PC2/4: #netserver -p 22113

PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           32768          10.34
598.11

Linux:    87380            16384           16384          10.04
779.02





PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
64/128/256/512/1024/2048/4096



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           64              10.19
417.10

         65536            32768           128             10.35
336.63

         65536            32768           256             10.36
576.99

         65536            32768           512             10.35
569.79

         65536            32768           1024            10.35
553.70

         65536            32768           2048            10.35
584.20

         65536            32768           4096            10.35
602.45



Linux:    87380            16384           64              10.03
778.21

         87380            16384           128             10.03
779.72

         87380            16384           256             10.04
780.16

         87380            16384           512             10.03
776.85

         87380            16384           1024            10.04
777.52

         87380            16384           2048            10.04
777.83

         87380            16384           4096            10.03
780.17









===============================

     Iperf Test Results

===============================



Bandwidth Test

--------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        740             600

Linux:     0.0-10.0        972             815





PC1/3: #iperf -c 192.168.1.20 -d



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.0        402             337

          0.0-10.0        404             338



Linux:     0.0-10.0        926             776

          0.0-10.0        44.1            36.9





Parallel Test

-------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20 -P 2



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        370             300

          0.0-10.3        370             300

     SUM: 0.0-10.3        739             600



Linux:     0.0-10.0        479             402

          0.0-10.0        473             396

     SUM: 0.0-10.0        952             797








_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribe@..."




--
the sun shines for all


_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Steve Dong]

It looks the jpeg attachments were somehow dropped. Trying again with pdf
attachment. Hopefully it works this time.


Thanks,
Steve

 

-----Original Message-----
From: owner-freebsd-performance@...
[mailto:owner-freebsd-performance@...] On Behalf Of Steve Dong
Sent: Thursday, October 15, 2009 10:44 PM
To: 'Hongtao Yin'; 'István'
Cc: freebsd-performance@...
Subject: RE: Comparison of FreeBSD/Linux TCP Throughput performance

Here are graphs from the netpipe test results with 8.0 RC1
 

Thanks,
Steve

 

-----Original Message-----
From: owner-freebsd-performance@...
[mailto:owner-freebsd-performance@...] On Behalf Of Hongtao Yin
Sent: Thursday, October 15, 2009 8:52 PM
To: 'István'
Cc: freebsd-performance@...
Subject: RE: Comparison of FreeBSD/Linux TCP Throughput performance

Istvan,

 

I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we collected
test data and the result shows that FreeBSD TCP performance is worse than
Linux. I got troubles to plot using gnuplot, so here I am attaching the raw
data files.

The netpipe commands are as follows:

PC2: #NPtcp

PC1: #NPtcp �Ch 192.168.1.20

 

Regards,

Hongtao

 

 

 

  _____  

From: István [mailto:leccine@...]
Sent: 2009年10月15日 5:13
To: Hongtao Yin
Cc: freebsd-performance@...
Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance

 

use netpipe TCP pls.

 

And graph it.

 

Thanks in advance,

Istvan

On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin <htyin@...> wrote:

Hi,



I compared TCP performance between FreeBSD and Linux by running test tools
Netperf and Iperf with Intel NIC.

The kernels are full version and default values are used in the testing
except TCP Congestion Control algorithm set to Reno.

>From the test results we can see Linux TCP performance in throughput is
better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
throughput is only 43% of Linux's.



I like to get some feedback if anyone did similar comparison test, or knows
any issues with kernels or drivers. Thanks lot.



FreeBSD and Linux Sysctl captures are attached for reference.



Regards,

Hongtao







Test Environments:

PC: Dell Precision T3400 (same 4 PCs)

CPU: Intel Core 2 Duo CPU E4600@...

FreeBSD: V7.1 (full version) (TCP CC: newReno)

Linux: V2.6.31.1 (full version) (TCP CC: Reno)

Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter (Gigabit)
chip 82546EB (only one port used for each PC)

Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)

Iperf: V2.0.4

Netperf: V2.4.4



Setup:





                                 ----------

                                 | switch |

                                 ----------

              ---------------------| | | |--------------------

              |                      | |                     |

              |              --------| |--------             |

              |              |                 |             |

              |              |                 |             |

       --------------  --------------  --------------  --------------

       |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |

       |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |

       |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|

       --------------  --------------  --------------  --------------











================================

    Netperf Test Results

================================



TCP Throughput Test

-------------------



PC2/4: #netserver -p 22113

PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           32768          10.34
598.11

Linux:    87380            16384           16384          10.04
779.02





PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
64/128/256/512/1024/2048/4096



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           64              10.19
417.10

         65536            32768           128             10.35
336.63

         65536            32768           256             10.36
576.99

         65536            32768           512             10.35
569.79

         65536            32768           1024            10.35
553.70

         65536            32768           2048            10.35
584.20

         65536            32768           4096            10.35
602.45



Linux:    87380            16384           64              10.03
778.21

         87380            16384           128             10.03
779.72

         87380            16384           256             10.04
780.16

         87380            16384           512             10.03
776.85

         87380            16384           1024            10.04
777.52

         87380            16384           2048            10.04
777.83

         87380            16384           4096            10.03
780.17









===============================

     Iperf Test Results

===============================



Bandwidth Test

--------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        740             600

Linux:     0.0-10.0        972             815





PC1/3: #iperf -c 192.168.1.20 -d



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.0        402             337

          0.0-10.0        404             338



Linux:     0.0-10.0        926             776

          0.0-10.0        44.1            36.9





Parallel Test

-------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20 -P 2



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        370             300

          0.0-10.3        370             300

     SUM: 0.0-10.3        739             600



Linux:     0.0-10.0        479             402

          0.0-10.0        473             396

     SUM: 0.0-10.0        952             797








_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribe@..."




--
the sun shines for all



_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[David Kelly]

On Oct 16, 2009, at 12:53 AM, Steve Dong wrote:

> It looks the jpeg attachments were somehow dropped. Trying again  
> with pdf
> attachment. Hopefully it works this time.


*I* saw two jpeg charts at the very bottom of Steve's top-posted  
reply. This demonstrates just one of many evils of top-posting: the  
top-poster doesn't read everything he re-sends. If its not worth  
reading then why send it again?




--
David Kelly N4HHE, dkelly@...
========================================================================
Whom computers would destroy, they must first drive mad.




_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Bill Moran-2]

Steve Dong <sdong@...> wrote:
>
> It looks the jpeg attachments were somehow dropped. Trying again with pdf
> attachment. Hopefully it works this time.

Really.  Don't post attachments to mailing lists.  It's just a bad idea,
a lot of people will be upset with the bandwidth it consumes.  Keep in
mind that not everyone on the list is interested in every conversation.

--
Bill Moran
Collaborative Fusion Inc.

wmoran@...
Phone: [phone]x4023

****************************************************************
IMPORTANT: This message contains confidential information
and is intended only for the individual named. If the reader of
this message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.
****************************************************************
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[kometen]

> Really.  Don't post attachments to mailing lists.  It's just a bad idea,
> a lot of people will be upset with the bandwidth it consumes.  Keep in
> mind that not everyone on the list is interested in every conversation.

Disclaimers should go the same way too! :)

echo "
****************************************************************
IMPORTANT: This message contains confidential information
and is intended only for the individual named. If the reader of
this message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.
****************************************************************" | wc
      16     129     958

--
regards
Claus

When lenity and cruelty play for a kingdom,
the gentler gamester is the soonest winner.

Shakespeare

twitter.com/kometen
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[István-3]

I see.
It shows that linux default setup is better.

Have you seen any FreeBSD performance tuning guides?

Regards,
Istvan

2009/10/16 Hongtao Yin <htyin@...>

>  Istvan,
>
>
>
> I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we collected
> test data and the result shows that FreeBSD TCP performance is worse than
> Linux. I got troubles to plot using gnuplot, so here I am attaching the raw
> data files.
>
> The netpipe commands are as follows:
>
> PC2: #NPtcp
>
> PC1: #NPtcp �Ch 192.168.1.20
>
>
>
> Regards,
>
> Hongtao
>
>
>
>
>
>
>  ------------------------------
>
> *From:* István [mailto:leccine@...]
> *Sent:* 2009年10月15日 5:13
>
> *To:* Hongtao Yin
> *Cc:* freebsd-performance@...
> *Subject:* Re: Comparison of FreeBSD/Linux TCP Throughput performance
>
>
>
> use netpipe TCP pls.
>
>
>
> And graph it.
>
>
>
> Thanks in advance,
>
> Istvan
>
> On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin <htyin@...> wrote:
>
> Hi,
>
>
>
> I compared TCP performance between FreeBSD and Linux by running test tools
> Netperf and Iperf with Intel NIC.
>
> The kernels are full version and default values are used in the testing
> except TCP Congestion Control algorithm set to Reno.
>
> >From the test results we can see Linux TCP performance in throughput is
> better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
> throughput is only 43% of Linux's.
>
>
>
> I like to get some feedback if anyone did similar comparison test, or knows
> any issues with kernels or drivers. Thanks lot.
>
>
>
> FreeBSD and Linux Sysctl captures are attached for reference.
>
>
>
> Regards,
>
> Hongtao
>
>
>
>
>
>
>
> Test Environments:
>
> PC: Dell Precision T3400 (same 4 PCs)
>
> CPU: Intel Core 2 Duo CPU E4600@...
>
> FreeBSD: V7.1 (full version) (TCP CC: newReno)
>
> Linux: V2.6.31.1 (full version) (TCP CC: Reno)
>
> Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter
> (Gigabit)
> chip 82546EB (only one port used for each PC)
>
> Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)
>
> Iperf: V2.0.4
>
> Netperf: V2.4.4
>
>
>
> Setup:
>
>
>
>
>
>                                  ----------
>
>                                  | switch |
>
>                                  ----------
>
>               ---------------------| | | |--------------------
>
>               |                      | |                     |
>
>               |              --------| |--------             |
>
>               |              |                 |             |
>
>               |              |                 |             |
>
>        --------------  --------------  --------------  --------------
>
>        |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |
>
>        |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |
>
>        |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|
>
>        --------------  --------------  --------------  --------------
>
>
>
>
>
>
>
>
>
>
>
> ================================
>
>     Netperf Test Results
>
> ================================
>
>
>
> TCP Throughput Test
>
> -------------------
>
>
>
> PC2/4: #netserver -p 22113
>
> PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10
>
>
>
>          Recv             Send            Send           Elapsed
> Throughput
>
>          Socket           Socket          Message        Time
> 10^6 bits/sec
>
>          Size             Size            Size           Sec.
>
>          bytes            bytes           bytes
>
> FreeBSD:  65536            32768           32768          10.34
> 598.11
>
> Linux:    87380            16384           16384          10.04
> 779.02
>
>
>
>
>
> PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
> 64/128/256/512/1024/2048/4096
>
>
>
>          Recv             Send            Send           Elapsed
> Throughput
>
>          Socket           Socket          Message        Time
> 10^6 bits/sec
>
>          Size             Size            Size           Sec.
>
>          bytes            bytes           bytes
>
> FreeBSD:  65536            32768           64              10.19
> 417.10
>
>          65536            32768           128             10.35
> 336.63
>
>          65536            32768           256             10.36
> 576.99
>
>          65536            32768           512             10.35
> 569.79
>
>          65536            32768           1024            10.35
> 553.70
>
>          65536            32768           2048            10.35
> 584.20
>
>          65536            32768           4096            10.35
> 602.45
>
>
>
> Linux:    87380            16384           64              10.03
> 778.21
>
>          87380            16384           128             10.03
> 779.72
>
>          87380            16384           256             10.04
> 780.16
>
>          87380            16384           512             10.03
> 776.85
>
>          87380            16384           1024            10.04
> 777.52
>
>          87380            16384           2048            10.04
> 777.83
>
>          87380            16384           4096            10.03
> 780.17
>
>
>
>
>
>
>
>
>
> ===============================
>
>      Iperf Test Results
>
> ===============================
>
>
>
> Bandwidth Test
>
> --------------
>
>
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20
>
>
>
>           Interval        Transfer        Bandwidth
>
>           sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        740             600
>
> Linux:     0.0-10.0        972             815
>
>
>
>
>
> PC1/3: #iperf -c 192.168.1.20 -d
>
>
>
>           Interval        Transfer        Bandwidth
>
>           sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.0        402             337
>
>           0.0-10.0        404             338
>
>
>
> Linux:     0.0-10.0        926             776
>
>           0.0-10.0        44.1            36.9
>
>
>
>
>
> Parallel Test
>
> -------------
>
>
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20 -P 2
>
>
>
>           Interval        Transfer        Bandwidth
>
>           sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        370             300
>
>           0.0-10.3        370             300
>
>      SUM: 0.0-10.3        739             600
>
>
>
> Linux:     0.0-10.0        479             402
>
>           0.0-10.0        473             396
>
>      SUM: 0.0-10.0        952             797
>
>
>
>
>
>
>
>
> _______________________________________________
> freebsd-performance@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to "
> freebsd-performance-unsubscribe@..."
>
>
>
>
> --
> the sun shines for all
>

--
the sun shines for all
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[István-3]

Thanks!
Regards,

Istvan

2009/10/16 Steve Dong <sdong@...>

> Here are graphs from the netpipe test results with 8.0 RC1
>
>
> Thanks,
> Steve
>
>
>
> -----Original Message-----
> From: owner-freebsd-performance@...
> [mailto:owner-freebsd-performance@...] On Behalf Of Hongtao Yin
> Sent: Thursday, October 15, 2009 8:52 PM
> To: 'István'
> Cc: freebsd-performance@...
> Subject: RE: Comparison of FreeBSD/Linux TCP Throughput performance
>
> Istvan,
>
>
>
> I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we collected
> test data and the result shows that FreeBSD TCP performance is worse than
> Linux. I got troubles to plot using gnuplot, so here I am attaching the raw
> data files.
>
> The netpipe commands are as follows:
>
> PC2: #NPtcp
>
> PC1: #NPtcp �Ch 192.168.1.20
>
>
>
> Regards,
>
> Hongtao
>
>
>
>
>
>
>
>  _____
>
> From: István [mailto:leccine@...]
> Sent: 2009年10月15日 5:13
> To: Hongtao Yin
> Cc: freebsd-performance@...
> Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance
>
>
>
> use netpipe TCP pls.
>
>
>
> And graph it.
>
>
>
> Thanks in advance,
>
> Istvan
>
> On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin <htyin@...> wrote:
>
> Hi,
>
>
>
> I compared TCP performance between FreeBSD and Linux by running test tools
> Netperf and Iperf with Intel NIC.
>
> The kernels are full version and default values are used in the testing
> except TCP Congestion Control algorithm set to Reno.
>
> >From the test results we can see Linux TCP performance in throughput is
> better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
> throughput is only 43% of Linux's.
>
>
>
> I like to get some feedback if anyone did similar comparison test, or knows
> any issues with kernels or drivers. Thanks lot.
>
>
>
> FreeBSD and Linux Sysctl captures are attached for reference.
>
>
>
> Regards,
>
> Hongtao
>
>
>
>
>
>
>
> Test Environments:
>
> PC: Dell Precision T3400 (same 4 PCs)
>
> CPU: Intel Core 2 Duo CPU E4600@...
>
> FreeBSD: V7.1 (full version) (TCP CC: newReno)
>
> Linux: V2.6.31.1 (full version) (TCP CC: Reno)
>
> Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter
> (Gigabit)
> chip 82546EB (only one port used for each PC)
>
> Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)
>
> Iperf: V2.0.4
>
> Netperf: V2.4.4
>
>
>
> Setup:
>
>
>
>
>
>                                 ----------
>
>                                 | switch |
>
>                                 ----------
>
>              ---------------------| | | |--------------------
>
>              |                      | |                     |
>
>              |              --------| |--------             |
>
>              |              |                 |             |
>
>              |              |                 |             |
>
>       --------------  --------------  --------------  --------------
>
>       |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |
>
>       |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |
>
>       |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|
>
>       --------------  --------------  --------------  --------------
>
>
>
>
>
>
>
>
>
>
>
> ================================
>
>    Netperf Test Results
>
> ================================
>
>
>
> TCP Throughput Test
>
> -------------------
>
>
>
> PC2/4: #netserver -p 22113
>
> PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10
>
>
>
>         Recv             Send            Send           Elapsed
> Throughput
>
>         Socket           Socket          Message        Time
> 10^6 bits/sec
>
>         Size             Size            Size           Sec.
>
>         bytes            bytes           bytes
>
> FreeBSD:  65536            32768           32768          10.34
> 598.11
>
> Linux:    87380            16384           16384          10.04
> 779.02
>
>
>
>
>
> PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
> 64/128/256/512/1024/2048/4096
>
>
>
>         Recv             Send            Send           Elapsed
> Throughput
>
>         Socket           Socket          Message        Time
> 10^6 bits/sec
>
>         Size             Size            Size           Sec.
>
>         bytes            bytes           bytes
>
> FreeBSD:  65536            32768           64              10.19
> 417.10
>
>         65536            32768           128             10.35
> 336.63
>
>         65536            32768           256             10.36
> 576.99
>
>         65536            32768           512             10.35
> 569.79
>
>         65536            32768           1024            10.35
> 553.70
>
>         65536            32768           2048            10.35
> 584.20
>
>         65536            32768           4096            10.35
> 602.45
>
>
>
> Linux:    87380            16384           64              10.03
> 778.21
>
>         87380            16384           128             10.03
> 779.72
>
>         87380            16384           256             10.04
> 780.16
>
>         87380            16384           512             10.03
> 776.85
>
>         87380            16384           1024            10.04
> 777.52
>
>         87380            16384           2048            10.04
> 777.83
>
>         87380            16384           4096            10.03
> 780.17
>
>
>
>
>
>
>
>
>
> ===============================
>
>     Iperf Test Results
>
> ===============================
>
>
>
> Bandwidth Test
>
> --------------
>
>
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20
>
>
>
>          Interval        Transfer        Bandwidth
>
>          sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        740             600
>
> Linux:     0.0-10.0        972             815
>
>
>
>
>
> PC1/3: #iperf -c 192.168.1.20 -d
>
>
>
>          Interval        Transfer        Bandwidth
>
>          sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.0        402             337
>
>          0.0-10.0        404             338
>
>
>
> Linux:     0.0-10.0        926             776
>
>          0.0-10.0        44.1            36.9
>
>
>
>
>
> Parallel Test
>
> -------------
>
>
>
> PC2/4: #iperf -s
>
> PC1/3: #iperf -c 192.168.1.20 -P 2
>
>
>
>          Interval        Transfer        Bandwidth
>
>          sec             MBytes          Mbits/sec
>
> FreeBSD:   0.0-10.3        370             300
>
>          0.0-10.3        370             300
>
>     SUM: 0.0-10.3        739             600
>
>
>
> Linux:     0.0-10.0        479             402
>
>          0.0-10.0        473             396
>
>     SUM: 0.0-10.0        952             797
>
>
>
>
>
>
>
>
> _______________________________________________
> freebsd-performance@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to
> "freebsd-performance-unsubscribe@..."
>
>
>
>
> --
> the sun shines for all
>
>

--
the sun shines for all
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Francisco Reyes-2]

Hongtao Yin writes:

> I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we collected

Check
man tuning

There are a few parameters there worth exploring.
For example check the section on net.inet.tcp.sendspace and
net.inet.tcp.recvspace
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Hongtao Yin]

This is one of the first things we tried.

 

Does anyone have the tuned parameters?

 

Actually we are looking for info like

 

1. Any bugs in Freebsd driver which have been fixed in linux kernel

2. Any TCP features/RFCs implemented in Linux, but not in Freebsd

3. Any other discrepancies between Linux and FreeBSD TCP implementation that
could potentially have caused this

 

 

Thanks.

Hongtao

 

 

  _____  

From: István [mailto:leccine@...]
Sent: 2009年10月16日 5:29
To: Hongtao Yin
Cc: freebsd-performance@...
Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance

 

I see.

 

It shows that linux default setup is better.

 

Have you seen any FreeBSD performance tuning guides?

 

Regards,

Istvan

2009/10/16 Hongtao Yin <htyin@...>

Istvan,

 

I ran newer FreeBSD code 8.0 RC1 this time. By using NetPIPE, we collected
test data and the result shows that FreeBSD TCP performance is worse than
Linux. I got troubles to plot using gnuplot, so here I am attaching the raw
data files.

The netpipe commands are as follows:

PC2: #NPtcp

PC1: #NPtcp �Ch 192.168.1.20

 

Regards,

Hongtao

 

 

 

  _____  

From: István [mailto:leccine@...]
Sent: 2009年10月15日 5:13


To: Hongtao Yin
Cc: freebsd-performance@...
Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance

 

use netpipe TCP pls.

 

And graph it.

 

Thanks in advance,

Istvan

On Thu, Oct 15, 2009 at 2:42 AM, Hongtao Yin <htyin@...> wrote:

Hi,



I compared TCP performance between FreeBSD and Linux by running test tools
Netperf and Iperf with Intel NIC.

The kernels are full version and default values are used in the testing
except TCP Congestion Control algorithm set to Reno.

>From the test results we can see Linux TCP performance in throughput is
better than FreeBSD. The worst case (send msg size 128) shows that FreeBSD
throughput is only 43% of Linux's.



I like to get some feedback if anyone did similar comparison test, or knows
any issues with kernels or drivers. Thanks lot.



FreeBSD and Linux Sysctl captures are attached for reference.



Regards,

Hongtao







Test Environments:

PC: Dell Precision T3400 (same 4 PCs)

CPU: Intel Core 2 Duo CPU E4600@...

FreeBSD: V7.1 (full version) (TCP CC: newReno)

Linux: V2.6.31.1 (full version) (TCP CC: Reno)

Ethernet card: Intel Pro/1000 PWLA8492 MT Dual Port Server Adapter (Gigabit)
chip 82546EB (only one port used for each PC)

Switch: Netgear ProSafe 8 port Gigabit Switch (model GS108)

Iperf: V2.0.4

Netperf: V2.4.4



Setup:





                                 ----------

                                 | switch |

                                 ----------

              ---------------------| | | |--------------------

              |                      | |                     |

              |              --------| |--------             |

              |              |                 |             |

              |              |                 |             |

       --------------  --------------  --------------  --------------

       |     PC1    |  |    PC2     |  |    PC3     |  |    PC4     |

       |   FreeBSD  |  |   FreeBSD  |  |   Linux    |  |   Linux    |

       |192.168.1.10|  |192.168.1.20|  |192.168.1.30|  |192.168.1.40|

       --------------  --------------  --------------  --------------











================================

    Netperf Test Results

================================



TCP Throughput Test

-------------------



PC2/4: #netserver -p 22113

PC1/3: #netperf -H 192.168.1.20 -p 22113 -l 10



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           32768          10.34
598.11

Linux:    87380            16384           16384          10.04
779.02





PC1/3: #netperf -t TCP_STREAM -H 192.168.1.20 -p 22113 -- -m
64/128/256/512/1024/2048/4096



         Recv             Send            Send           Elapsed
Throughput

         Socket           Socket          Message        Time
10^6 bits/sec

         Size             Size            Size           Sec.

         bytes            bytes           bytes

FreeBSD:  65536            32768           64              10.19
417.10

         65536            32768           128             10.35
336.63

         65536            32768           256             10.36
576.99

         65536            32768           512             10.35
569.79

         65536            32768           1024            10.35
553.70

         65536            32768           2048            10.35
584.20

         65536            32768           4096            10.35
602.45



Linux:    87380            16384           64              10.03
778.21

         87380            16384           128             10.03
779.72

         87380            16384           256             10.04
780.16

         87380            16384           512             10.03
776.85

         87380            16384           1024            10.04
777.52

         87380            16384           2048            10.04
777.83

         87380            16384           4096            10.03
780.17









===============================

     Iperf Test Results

===============================



Bandwidth Test

--------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        740             600

Linux:     0.0-10.0        972             815





PC1/3: #iperf -c 192.168.1.20 -d



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.0        402             337

          0.0-10.0        404             338



Linux:     0.0-10.0        926             776

          0.0-10.0        44.1            36.9





Parallel Test

-------------



PC2/4: #iperf -s

PC1/3: #iperf -c 192.168.1.20 -P 2



          Interval        Transfer        Bandwidth

          sec             MBytes          Mbits/sec

FreeBSD:   0.0-10.3        370             300

          0.0-10.3        370             300

     SUM: 0.0-10.3        739             600



Linux:     0.0-10.0        479             402

          0.0-10.0        473             396

     SUM: 0.0-10.0        952             797








_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribe@..."




--
the sun shines for all




--
the sun shines for all

_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Adrian Chadd-2]

2009/10/16 István <leccine@...>:
> I see.
> It shows that linux default setup is better.

.. being completely correct, it shows the linux default setup _for
netpipe_ is better on that particular hardware.

That identifies a few other variables which may need addressing. :)


Adrian
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[István-3]

I guess it is not only for netpipe, it is doing a pretty decent job changing
the packet size checking the performance so finally you have an overview
about the size, lag, bw

I like! :)

On Sat, Oct 17, 2009 at 11:15 AM, Adrian Chadd <adrian@...> wrote:

> 2009/10/16 István <leccine@...>:
> > I see.
> > It shows that linux default setup is better.
>
> .. being completely correct, it shows the linux default setup _for
> netpipe_ is better on that particular hardware.
>
> That identifies a few other variables which may need addressing. :)
>
>
> Adrian
>

--
the sun shines for all
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[O. Hartmann-5]

István wrote:

> I guess it is not only for netpipe, it is doing a pretty decent job changing
> the packet size checking the performance so finally you have an overview
> about the size, lag, bw
>
> I like! :)
>
> On Sat, Oct 17, 2009 at 11:15 AM, Adrian Chadd <adrian@...> wrote:
>
>> 2009/10/16 István <leccine@...>:
>>> I see.
>>> It shows that linux default setup is better.
>> .. being completely correct, it shows the linux default setup _for
>> netpipe_ is better on that particular hardware.
>>
>> That identifies a few other variables which may need addressing. :)
>>
>>
>> Adrian
>>
>
>
>

And maybe the wise can send out word of doing well on FreeBSD, so we can
choose a better setup with looking forward repeating the test under
'tuned' conditions? I'm willing to perform some tests within the next 4
weeks when our server hardware (Dell PowerEdge 1950-III with two
if_bge() NICs and 16GB RAM) changes OS from FreeBSD 8.0 to RedHat Linux.
In a time-window of about a week I might be capable of testing FreeBSD
8.0 as it would be that time by the mid of November with a setup of
Linux (distro doesn't matter as I can choose). I need to know WHAT,
WHERE and HOW. Thanks.

oh

_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Steve Dong]

If there's a better/lighter way to show these graphics, I'd like to know.  

Thanks,
Steve

 

-----Original Message-----
From: owner-freebsd-performance@...
[mailto:owner-freebsd-performance@...] On Behalf Of Bill Moran
Sent: Friday, October 16, 2009 4:54 AM
To: Steve Dong
Cc: freebsd-performance@...
Subject: Re: Comparison of FreeBSD/Linux TCP Throughput performance

Steve Dong <sdong@...> wrote:
>
> It looks the jpeg attachments were somehow dropped. Trying again with
> pdf attachment. Hopefully it works this time.

Really.  Don't post attachments to mailing lists.  It's just a bad idea, a
lot of people will be upset with the bandwidth it consumes.  Keep in mind
that not everyone on the list is interested in every conversation.

--
Bill Moran
Collaborative Fusion Inc.

wmoran@...
Phone: [phone]x4023

****************************************************************
IMPORTANT: This message contains confidential information and is intended
only for the individual named. If the reader of this message is not an
intended recipient (or the individual responsible for the delivery of this
message to an intended recipient), please be advised that any re-use,
dissemination, distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received this e-mail by
mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or error-free as
information could be intercepted, corrupted, lost, destroyed, arrive late or
incomplete, or contain viruses. The sender therefore does not accept
liability for any errors or omissions in the contents of this message, which
arise as a result of e-mail transmission.
****************************************************************
_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribe@..."

_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."

[Chuck Swiger-2]

Hi, Steve--

On Oct 17, 2009, at 8:14 AM, Steve Dong wrote:
> If there's a better/lighter way to show these graphics, I'd like to  
> know.

Sure-- put 'em on a webserver somewhere, and put links to them in your  
email to this mailing list.

If you wanted to do even better than that, set up a simple webpage  
describing what you are doing in your comparison, have a link to the  
dmesg/boot output for each platform as a .txt file and a description  
of any system tweaks & tuning, have a link that points to a  
description of the test setup (ie, your ASCII diagram of the switch  
and 4 machines), then your graphs, then the raw data (or links to it,  
depending).  You can then throw in netstat -s output, or NIC driver  
stats from sysctl, or switch stats, etc-- anything else that adds  
useful context.

There are a fair number of posts in the list archives which describe  
how to benchmark reliably, and the people who are most likely to be  
making code changes to FreeBSD also tend to like to know whether  
you've collected enough data, in a controlled fashion, to have an idea  
as to whether your measurements are reproducible.  I'm not a purist,  
and I believe you can get useful estimations without rigorous testing,  
but there are others who make the point that if you haven't provided  
at least a standard deviation, then you haven't collected enough  
data-- done enough trials-- to determine whether the results are  
meaningful.   (See /usr/src/tools/tools/ministat/README)

Of course, you're not obligated to do any of the above, but if you  
want the effort you've put in to be more useful, consider these a  
suggestion.  Finally, the next step beyond that would be to try  
tweaking some things, and see what kind of changes you get from that  
versus the original performance.  It might be the case that making a  
simple tuning change would have a significant difference in  
performance; if you can identify that, then FreeBSD or Linux  
developers can use that information to better tune the OS defaults.

Regards,
--
-Chuck

_______________________________________________
freebsd-performance@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "freebsd-performance-unsubscribe@..."