PPPoE server (high traffic in WDM network)

[Dmitriy-6]

Hello, Michelle

>The main problem with the bandwidth is, that even the VOD/IPTV and  VoIP
>traffic goes throug the PPPoE server which is very bad.
I don't think so, yes it is bad and not very bad. It depends on BRAS
hardware.

>I do not want to count the traffic to a  specific  /25  which  hold  the
>storage servers,If you want this, you may use the MPD-specific RADIUS
attributes described in:
http://mpd.sourceforge.net/doc5/mpd30.html#30
see last paragraph.
> mean, the VOD/IPTV and VoIP  traffic  must  bypass  the
>PPPoE server.
If VOD/IPTV is uses multicast - yes else you must set the pipe for this kind
of traffic
>  and this reduce the traffic enormous...
hmm, What better way for you:
change all active network devices (e.g. switches) to use the multicast
groups
or buy additional servers for PPPoE purpose

>My idea is/was, to put the PPPoE server diretly byside the FTTH DSLAM's,
>which mean, each 96port DSLAM has an upstrem of 1 GE and even if  I  put
>10 of them in a 42RU, it would normaly not fill the  10 GE  ports  of  a
>professionel Server.  And of corse,  I  can  put  always  two  or  three
>together parallel.
Yes, it is good idea.

>The problem is only, that I can not install 10 (or 20 redunant) 1U Sun
>Fire X4100M2, even if I can get up to 60% rebat of the listprice.
>I have not the place to put 20 additiona servers into, nor  I  like  the
>power consumation ~70 Watt with the smalles CPU and only 4 GByte of RAM.
I have't Sun servers and i can't compare Sun with i386 usage with this task
I use i386 servers. See below.

>> instead of bigger one
>> gives better results performance-wise. You also have to test if SMP
>> helps and how much. A beast with 16 cores is more powerful from a
>> regular computer with 2 cores, but does it help in your setup?
> If I go with 1 U Sun Fire X4100M2 the Opteron has 4 Cores
> and 4  threads per core (AFAIK there is a 8 threads version too)
the SMP with many NIC helps, and so much

>> Can you recomment it for an ISP setup?
>> It's FreeBSD running from a read-only mounted medium.
>> No more, no less. Yes, it's fine for an ISP setup.
>If I have 4 GByte of memory, I could run entirely from RAMDISK...
>Memory is cheaper then the harddrives
I use usb flash drive in read-write mode as full replace of hard drive



The *BIG* problem is to handle rx packets which goes in to pppoe server.

The goal of which is using hardware interrupt moderation of NIC and good
motherboard.
Maybe second cache of CPU to be more and more is better (but i have't tests)


I use that configuration:
The hardware:
S3000AHV motherboard with Core2Quad 6600 processor (4 cores).
motherboard has 82571 chip NIC, the two additional cars: EXPI9402
(PCI-E x4 two 10/100/1000BaseTX ports) and EXPI9400 (PCI-E x1
one ethernet port) is installed to the system.
The goal of which is to load all cores of CPU
or use yandex em(4) driver who uses different rx kernel threads on every
CPU/core (if you have one NIC)
for the driver see: people. yandex . ru/~wawa
The software:
1. FreeBSD 7.0, mpd5.3 + ipfw + dummynet (shaping on this machine - bad
    idea! for performance.  but it is first my setup and works very stable)
2. FreeBSD 7.2, mpd5.3 + ng_car + ng_bpf (rate-limiting
    on this machine - the perfomance of traffic processing is very good,
    because ng_car & ng_bpf is in kernel space, but have trouble
    described in PR kern/137881, i think on FBSD 7.0 it may be works fine,
    but no test are made, i'm busy)
3. Use PPPoE concentrator and traffic shaper/rate_limiting on different
    machines - it is best but is very older practice.
I think the best choise today is #2



Dmitriy Zamuraev, system administrator
The Netline NSP company
Russia, Tyumen.

_______________________________________________
freebsd-isp@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to "freebsd-isp-unsubscribe@..."

[Michelle Konzack-4]

Am 2009-08-20 15:25:50, schrieb Дмитрий Замураев:
> Hello, Michelle
>
>> The main problem with the bandwidth is, that even the VOD/IPTV and  VoIP
>> traffic goes throug the PPPoE server which is very bad.
> I don't think so, yes it is bad and not very bad. It depends on BRAS
> hardware.

Hmmm...

>> I do not want to count the traffic to a  specific  /25  which  hold  the
>> storage servers,If you want this, you may use the MPD-specific RADIUS
> attributes described in:
> http://mpd.sourceforge.net/doc5/mpd30.html#30
> see last paragraph.

I have tried this but it does not work.

>> mean, the VOD/IPTV and VoIP  traffic  must  bypass  the
>> PPPoE server.
> If VOD/IPTV is uses multicast - yes else you must set the pipe for this
> kind of traffic

Ehm, noy, VOD is unicast, because the  user  can  view  videos  whenever
she/he want.  Currently we do not know whether we  support  TV  channels
over IPTV.

>> My idea is/was, to put the PPPoE server diretly byside the FTTH DSLAM's,
>> which mean, each 96port DSLAM has an upstrem of 1 GE and even if  I  put
>> 10 of them in a 42RU, it would normaly not fill the  10 GE  ports  of  a
>> professionel Server.  And of corse,  I  can  put  always  two  or  three
>> together parallel.
> Yes, it is good idea.

I will see...  It seems, that even 96 FTTH ports with 100 MBit  each  do
not hit the limit of a 1GE  upstream,  so  the  now  I  could  test  the
"Sun Fire X4100M2" becaus it has four 1GE ports, which mean, I  can  use
one as upstream and then connect the three others to up  to  three  FTTH
DSLAMS...

Transforming a 1400 US$ Server into a pppoe server/router which has  not
more then 2 GByte of memory and the smallest CPU available.

This could work.

>> The problem is only, that I can not install 10 (or 20 redunant) 1U Sun
>> Fire X4100M2, even if I can get up to 60% rebat of the listprice.
>> I have not the place to put 20 additiona servers into, nor  I  like  the
>> power consumation ~70 Watt with the smalles CPU and only 4 GByte of RAM.
> I have't Sun servers and i can't compare Sun with i386 usage with this task
> I use i386 servers. See below.

The X4100M2 is a Opteron

> the SMP with many NIC helps, and so much

I know, the smallest Quad-Core CPU with 4 threads/core could be enough

> I use usb flash drive in read-write mode as full replace of hard drive

I do not know whether the X4100 can boot from USB-Flash, but  it  seems,
a SATA/CF adaptor is working or even a very cheap SATA  HDD.  The  X4100
is not bound to SAS drives.

> for the driver see: people. yandex . ru/~wawa

OK

> 2. FreeBSD 7.2, mpd5.3 + ng_car + ng_bpf (rate-limiting
>    on this machine - the perfomance of traffic processing is very good,
>    because ng_car & ng_bpf is in kernel space, but have trouble
>    described in PR kern/137881, i think on FBSD 7.0 it may be works fine,
>    but no test are made, i'm busy)
> I think the best choise today is #2

I will give it a try

Thanks, Greetings and nice Day/Evening
    Michelle Konzack
    Systemadministrator
    Tamay Dogan Network
    Debian GNU/Linux Consultant

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
<http://www.tamay-dogan.net/>                 Michelle Konzack
<http://www.can4linux.org/>                   Apt. 917
<http://www.flexray4linux.org/>               50, rue de Soultz
Jabber linux4michelle@...           67100 Strabourg/France
IRC    #Debian (irc.icq.com)                  Tel. DE: [phone]
ICQ    #328449886                             Tel. FR: [phone]
_______________________________________________
freebsd-isp@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to "freebsd-isp-unsubscribe@..."

[Dmitriy-6]

>>> I do not want to count the traffic to a  specific  /25  which  hold  the
>>> storage servers,If you want this, you may use the MPD-specific RADIUS
>> attributes described in:
>> http://mpd.sourceforge.net/doc5/mpd30.html#30
>> see last paragraph.
>
> I have tried this but it does not work.
I think, you not understand me. You need to learn the radius server to
handle and process this attributes.

>>> My idea is/was, to put the PPPoE server diretly byside the FTTH DSLAM's,
>>> which mean, each 96port DSLAM has an upstrem of 1 GE and even if  I  put
>>> 10 of them in a 42RU, it would normaly not fill the  10 GE  ports  of  a
>>> professionel Server.  And of corse,  I  can  put  always  two  or  three
>>> together parallel.
>> Yes, it is good idea.
>
> I will see...  It seems, that even 96 FTTH ports with 100 MBit  each  do
> not hit the limit of a 1GE  upstream,  so  the  now  I  could  test  the
> "Sun Fire X4100M2" becaus it has four 1GE ports, which mean, I  can  use
> one as upstream and then connect the three others to up  to  three  FTTH
> DSLAMS...

Yes all users can't utialize full bandwidth of his channel at the time.

> Transforming a 1400 US$ Server into a pppoe server/router which has  not
> more then 2 GByte of memory and the smallest CPU available.
>
> This could work.
That i don't understand you.


>> for the driver see: people. yandex . ru/~wawa
>
> OK
i'm sorry, people. yandex-team. ru/~wawa is right URL

_______________________________________________
freebsd-isp@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to "freebsd-isp-unsubscribe@..."