out xmit (demux) pipe bw accounting

[Michael Spratt-2]

Could not find answer to following question.

Given :
# pipe 1 config bw 1Mbit/sec
# ipfw 1000 add pipe 1 ip from any to any out xmit em0

Will the bandwidth limit include layer 2 Ethernet headers or will only
the IP datagram itself be included in the accounting mechanism?

Total output on the em0 at layer 2 is limited to 1Mbit/s or IP going out
em0 layer 3 will be limited to 1Mbit/s

--
--------------------------------------------------

Mike Spratt - Iraq (GMT+3)

Office: [phone]

DSN: [phone]

Cell: [phone]

mike@...

---------------------------------------------------
_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."

[Chuck Swiger-2]

Hi--

On Aug 17, 2009, at 7:32 AM, Michael Spratt wrote:
> Could not find answer to following question.
>
> Given :
> # pipe 1 config bw 1Mbit/sec
> # ipfw 1000 add pipe 1 ip from any to any out xmit em0
>
> Will the bandwidth limit include layer 2 Ethernet headers or will  
> only the IP datagram itself be included in the accounting mechanism?

IPFW normally processes traffic at layer-3, but you can toggle  
net.link.ether.ipfw sysctl to on, in which case the above rule would  
also be run against layer-2 packets including the 802.3 ethernet  
header.  See the diagram in "PACKET FLOW" of "man ipfw"....

Regards,
--
-Chuck

_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."

[Michael Spratt-2]

Chuck Swiger wrote:

> Hi--
>
> On Aug 17, 2009, at 7:32 AM, Michael Spratt wrote:
>> Could not find answer to following question.
>>
>> Given :
>> # pipe 1 config bw 1Mbit/sec
>> # ipfw 1000 add pipe 1 ip from any to any out xmit em0
>>
>> Will the bandwidth limit include layer 2 Ethernet headers or will only
>> the IP datagram itself be included in the accounting mechanism?
>
> IPFW normally processes traffic at layer-3, but you can toggle
> net.link.ether.ipfw sysctl to on, in which case the above rule would
> also be run against layer-2 packets including the 802.3 ethernet
> header.  See the diagram in "PACKET FLOW" of "man ipfw"....
>
> Regards,

Dear Chuck thanks for your response. I am not sure though that my
question is answered. Yes switching net.link.ether.ipfw->1 does cause
processing against layer 2 as indicated in the man ipfw diagram and the
command
# sysctl -d net.link.ether.ipfw Yields the description
net.link.ether.ipfw: Pass ether pkts through firewall

Setting this sysctl variable to 0 does disengage the processing of
Ethernet frames and corresponding IP payloads according to the rule set
specified and visa-versa.

I do not see anything indicating this sysctl variable affects the way
dummynet does bit counting as it enforces bw limits on a pipe or queue.

Will the ruleset I provided above impliment the 1Mbit/s limit at layer 2
including the Ethernet frames in the 1Mbit/s budget or (VS.) Only layer
3 IP datagrams (and their coresponding layer 4 payloads) are
incrementing the counters responsible for counting against the 1Mbit/s limit

Simplification of question:
  Given:
   # pipe 1 config bw 1Mbit/sec
   # ipfw 1000 add pipe 1 ip from any to any out xmit em0
  Dichotomy:
   layer 2 is limit to 1Mbit/s out em0 (Vs.) layer 3 is limit to 1Mbit/s
out em0

Only one of these can be correct and I do not see documentation anywhere
indicating to me which one is indeed correct. I am not educated enough
to look in the source code to determine this for myself.

I did not state before the dummynet is on a freebsd transparent bridge.
Here are my sysctl variables

net.link.ether.ipfw: 0
net.link.bridge.ipfw: 1
net.link.bridge.ipfw_arp: 0
# sysctl -a | grep dummynet
net.inet.ip.dummynet.debug: 0
net.inet.ip.dummynet.pipe_byte_limit: 7000000
net.inet.ip.dummynet.pipe_slot_limit: 100
net.inet.ip.dummynet.io_pkt_drop: 33250421
net.inet.ip.dummynet.io_pkt_fast: 0
net.inet.ip.dummynet.io_pkt: 3512830485
net.inet.ip.dummynet.io_fast: 0
net.inet.ip.dummynet.tick_lost: 0
net.inet.ip.dummynet.tick_diff: 1662798
net.inet.ip.dummynet.tick_adjustment: 1679911
net.inet.ip.dummynet.tick_delta_sum: 721
net.inet.ip.dummynet.tick_delta: 2
net.inet.ip.dummynet.red_max_pkt_size: 1500
net.inet.ip.dummynet.red_avg_pkt_size: 512
net.inet.ip.dummynet.red_lookup_depth: 256
net.inet.ip.dummynet.max_chain_len: 16
net.inet.ip.dummynet.expire: 1
net.inet.ip.dummynet.search_steps: 0
net.inet.ip.dummynet.searches: 0
net.inet.ip.dummynet.extract_heap: 0
net.inet.ip.dummynet.ready_heap: 16
net.inet.ip.dummynet.curr_time: 1804818806
net.inet.ip.dummynet.hash_size: 64
# sysctl -a | grep fw
net.inet.ip.fw.dyn_keepalive: 1
net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.static_count: 26
net.inet.ip.fw.dyn_max: 4096
net.inet.ip.fw.dyn_count: 0
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_buckets: 256
net.inet.ip.fw.default_rule: 65535
net.inet.ip.fw.verbose_limit: 0
net.inet.ip.fw.verbose: 0
net.inet.ip.fw.debug: 1
net.inet.ip.fw.one_pass: 1
net.inet.ip.fw.autoinc_step: 100
net.inet.ip.fw.enable: 1

rule set

00900    15945359     1264958184 allow ip from 10.18.0.6 to any out xmit
bce0
00901    44378882     8041056830 pipe 1 ip from 10.18.0.0/24 to any out
xmit bce0
00910      225165       18913860 pipe 1 ip from 10.18.2.1 to any out
xmit bce0
00915           0              0 pipe 2 ip from 10.18.2.2 to any out
xmit bce0
00920           0              0 pipe 3 ip from 10.18.2.3 to any out
xmit bce0
01000      265959       40261978 pipe 1 ip from 10.21.192.8/29 to any
out xmit bce0
01001    19627389     3444748305 pipe 1 ip from 10.20.0.0/20 to any out
xmit bce0
01002   180922838    31277916333 pipe 1 ip from 10.21.32.0/20 to any out
xmit bce0
01003   376697028    75171259985 pipe 1 ip from 10.20.64.0/20 to any out
xmit bce0
01004    68629348    12603497407 pipe 1 ip from 10.20.192.0/20 to any
out xmit bce0
01005   185005607    37413644435 pipe 1 ip from 10.20.176.0/20 to any
out xmit bce0
01006    59179395    10051795154 pipe 1 ip from 10.20.144.0/20 to any
out xmit bce0
01007   235863676    41845371053 pipe 1 ip from 10.20.128.0/20 to any
out xmit bce0
01008    51518999    10847386956 pipe 1 ip from 10.20.96.0/20 to any out
xmit bce0
01009    79734709    13723937026 pipe 1 ip from 10.20.112.0/20 to any
out xmit bce0
01010    30062283     4078276506 pipe 1 ip from 10.20.80.0/20 to any out
xmit bce0
01011    36318511     6974450810 pipe 1 ip from 10.20.208.0/20 to any
out xmit bce0
01012    19642496     3098922560 pipe 1 ip from 10.20.160.0/20 to any
out xmit bce0
01013    18209570     2964820411 pipe 1 ip from 10.20.224.0/20 to any
out xmit bce0
01014    13707931     3915527156 pipe 1 ip from 10.22.160.0/20 to any
out xmit bce0
01015     9893051     1175739042 pipe 1 ip from 10.21.112.0/20 to any
out xmit bce0
01016    39876049     7905051976 pipe 1 ip from 10.22.144.0/20 to any
out xmit bce0
01017           0              0 pipe 1 ip from 10.22.147.0/24 to any
out xmit bce0
01018   128251743    24972920481 pipe 1 ip from 10.21.0.0/20 to any out
xmit bce0
01019   102099411    27172050432 pipe 1 ip from 10.21.16.0/20 to any out
xmit bce0
65535 53581341116 24295389614128 allow ip from any to any

and pipe spec

# ipfw pipe list
00001:  11.500 Mbit/s    0 ms  75 KB 1 queues (1 buckets) droptail
     mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes
Pkt/Byte Drp
   0 tcp     10.21.44.163/47934   65.55.131.121/80    1735939204
358391598184  1  191 10152696


You can see that the bw has been configured to 11.5Mbit/s which should
be the base 10 expression for bit/s meaning exactly 11,500,000 bit/s

And here lies the question again does this limit include layer 2
Ethernet frame headers in the bit/s counting mechanism


-Mike







--
--------------------------------------------------

Mike Spratt - Iraq (GMT+3)

Office: [phone]

DSN: [phone]

Cell: [phone]

mike@...

---------------------------------------------------
_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."

[Ian Smith-12]

On Wed, 19 Aug 2009, Michael Spratt wrote:
 > Chuck Swiger wrote:
 > > Hi--
 > >
 > > On Aug 17, 2009, at 7:32 AM, Michael Spratt wrote:
 > > > Could not find answer to following question.
 > > >
 > > > Given :
 > > > # pipe 1 config bw 1Mbit/sec
 > > > # ipfw 1000 add pipe 1 ip from any to any out xmit em0
 > > >
 > > > Will the bandwidth limit include layer 2 Ethernet headers or will only
 > > > the IP datagram itself be included in the accounting mechanism?
 > >
 > > IPFW normally processes traffic at layer-3, but you can toggle
 > > net.link.ether.ipfw sysctl to on, in which case the above rule would also
 > > be run against layer-2 packets including the 802.3 ethernet header.  See
 > > the diagram in "PACKET FLOW" of "man ipfw"....
 > >
 > > Regards,
 >
 > Dear Chuck thanks for your response. I am not sure though that my question is
 > answered. Yes switching net.link.ether.ipfw->1 does cause processing against
 > layer 2 as indicated in the man ipfw diagram and the command
 > # sysctl -d net.link.ether.ipfw Yields the description net.link.ether.ipfw:
 > Pass ether pkts through firewall
 >
 > Setting this sysctl variable to 0 does disengage the processing of Ethernet
 > frames and corresponding IP payloads according to the rule set specified and
 > visa-versa.
 >
 > I do not see anything indicating this sysctl variable affects the way
 > dummynet does bit counting as it enforces bw limits on a pipe or queue.

That's right, dummynet pipes and queues work at the IP packet level.

 > Will the ruleset I provided above impliment the 1Mbit/s limit at layer 2
 > including the Ethernet frames in the 1Mbit/s budget or (VS.) Only layer 3 IP
 > datagrams (and their coresponding layer 4 payloads) are incrementing the
 > counters responsible for counting against the 1Mbit/s limit

I think only IP packets are queued in dummynet pipes, and it's IP packet
throughput rates being managed.

Check out Luigi's site at http://info.iet.unipi.it/~luigi/dummynet/ 
There's a couple of really good .pdfs listed at the bottom describing
the porting of ipfw/dummynet to Linux, with lots of detail background.

It'd seem unusual if the 1Mbit/s limit you've presumably been allocated
from 'higher up' would include ethernet headers as well as IP traffic.  
And even if it did, it's not so hard to calculate what factor to use to
keep the rate below 1Mbit/s at ethernet layer, if they count that way.

 > Simplification of question:
 >  Given:
 >   # pipe 1 config bw 1Mbit/sec
 >   # ipfw 1000 add pipe 1 ip from any to any out xmit em0
 >  Dichotomy:
 >   layer 2 is limit to 1Mbit/s out em0 (Vs.) layer 3 is limit to 1Mbit/s out
 > em0
 >
 > Only one of these can be correct and I do not see documentation anywhere
 > indicating to me which one is indeed correct. I am not educated enough to
 > look in the source code to determine this for myself.

man ipfw read a few times forwards and backwards pretty well covers it,
though the code's not bad bedtime reading :)

 > I did not state before the dummynet is on a freebsd transparent bridge.
 > Here are my sysctl variables
 >
 > net.link.ether.ipfw: 0
 > net.link.bridge.ipfw: 1
 > net.link.bridge.ipfw_arp: 0
 > # sysctl -a | grep dummynet
 > net.inet.ip.dummynet.debug: 0
 > net.inet.ip.dummynet.pipe_byte_limit: 7000000
 > net.inet.ip.dummynet.pipe_slot_limit: 100
 > net.inet.ip.dummynet.io_pkt_drop: 33250421
 > net.inet.ip.dummynet.io_pkt_fast: 0
 > net.inet.ip.dummynet.io_pkt: 3512830485
 > net.inet.ip.dummynet.io_fast: 0
 > net.inet.ip.dummynet.tick_lost: 0
 > net.inet.ip.dummynet.tick_diff: 1662798
 > net.inet.ip.dummynet.tick_adjustment: 1679911
 > net.inet.ip.dummynet.tick_delta_sum: 721
 > net.inet.ip.dummynet.tick_delta: 2
 > net.inet.ip.dummynet.red_max_pkt_size: 1500
 > net.inet.ip.dummynet.red_avg_pkt_size: 512
 > net.inet.ip.dummynet.red_lookup_depth: 256
 > net.inet.ip.dummynet.max_chain_len: 16
 > net.inet.ip.dummynet.expire: 1
 > net.inet.ip.dummynet.search_steps: 0
 > net.inet.ip.dummynet.searches: 0
 > net.inet.ip.dummynet.extract_heap: 0
 > net.inet.ip.dummynet.ready_heap: 16
 > net.inet.ip.dummynet.curr_time: 1804818806
 > net.inet.ip.dummynet.hash_size: 64
 > # sysctl -a | grep fw
 > net.inet.ip.fw.dyn_keepalive: 1
 > net.inet.ip.fw.dyn_short_lifetime: 5
 > net.inet.ip.fw.dyn_udp_lifetime: 10
 > net.inet.ip.fw.dyn_rst_lifetime: 1
 > net.inet.ip.fw.dyn_fin_lifetime: 1
 > net.inet.ip.fw.dyn_syn_lifetime: 20
 > net.inet.ip.fw.dyn_ack_lifetime: 300
 > net.inet.ip.fw.static_count: 26
 > net.inet.ip.fw.dyn_max: 4096
 > net.inet.ip.fw.dyn_count: 0
 > net.inet.ip.fw.curr_dyn_buckets: 256
 > net.inet.ip.fw.dyn_buckets: 256
 > net.inet.ip.fw.default_rule: 65535
 > net.inet.ip.fw.verbose_limit: 0
 > net.inet.ip.fw.verbose: 0
 > net.inet.ip.fw.debug: 1
 > net.inet.ip.fw.one_pass: 1
 > net.inet.ip.fw.autoinc_step: 100
 > net.inet.ip.fw.enable: 1
 >
 > rule set
 >
 > 00900    15945359     1264958184 allow ip from 10.18.0.6 to any out xmit bce0
 > 00901    44378882     8041056830 pipe 1 ip from 10.18.0.0/24 to any out xmit bce0
 > 00910      225165       18913860 pipe 1 ip from 10.18.2.1 to any out xmit bce0
 > 00915           0              0 pipe 2 ip from 10.18.2.2 to any out xmit bce0
 > 00920           0              0 pipe 3 ip from 10.18.2.3 to any out xmit bce0
 > 01000      265959       40261978 pipe 1 ip from 10.21.192.8/29 to any out xmit bce0
 > 01001    19627389     3444748305 pipe 1 ip from 10.20.0.0/20 to any out xmit bce0
 > 01002   180922838    31277916333 pipe 1 ip from 10.21.32.0/20 to any out xmit bce0
 > 01003   376697028    75171259985 pipe 1 ip from 10.20.64.0/20 to any out xmit bce0
 > 01004    68629348    12603497407 pipe 1 ip from 10.20.192.0/20 to any out xmit bce0
 > 01005   185005607    37413644435 pipe 1 ip from 10.20.176.0/20 to any out xmit bce0
 > 01006    59179395    10051795154 pipe 1 ip from 10.20.144.0/20 to any out xmit bce0
 > 01007   235863676    41845371053 pipe 1 ip from 10.20.128.0/20 to any out xmit bce0
 > 01008    51518999    10847386956 pipe 1 ip from 10.20.96.0/20 to any out xmit bce0
 > 01009    79734709    13723937026 pipe 1 ip from 10.20.112.0/20 to any out xmit bce0
 > 01010    30062283     4078276506 pipe 1 ip from 10.20.80.0/20 to any out xmit bce0
 > 01011    36318511     6974450810 pipe 1 ip from 10.20.208.0/20 to any out xmit bce0
 > 01012    19642496     3098922560 pipe 1 ip from 10.20.160.0/20 to any out xmit bce0
 > 01013    18209570     2964820411 pipe 1 ip from 10.20.224.0/20 to any out xmit bce0
 > 01014    13707931     3915527156 pipe 1 ip from 10.22.160.0/20 to any out xmit bce0
 > 01015     9893051     1175739042 pipe 1 ip from 10.21.112.0/20 to any out xmit bce0
 > 01016    39876049     7905051976 pipe 1 ip from 10.22.144.0/20 to any out xmit bce0
 > 01017           0              0 pipe 1 ip from 10.22.147.0/24 to any out xmit bce0
 > 01018   128251743    24972920481 pipe 1 ip from 10.21.0.0/20 to any out xmit bce0
 > 01019   102099411    27172050432 pipe 1 ip from 10.21.16.0/20 to any out xmit bce0
 > 65535 53581341116 24295389614128 allow ip from any to any
 >
 > and pipe spec
 >
 > # ipfw pipe list
 > 00001:  11.500 Mbit/s    0 ms  75 KB 1 queues (1 buckets) droptail
 >     mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
 > BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte
 > Drp
 >   0 tcp     10.21.44.163/47934   65.55.131.121/80    1735939204 358391598184
 > 1  191 10152696
 >
 >
 > You can see that the bw has been configured to 11.5Mbit/s which should be the
 > base 10 expression for bit/s meaning exactly 11,500,000 bit/s

Yes, but I don't see how you get 11.5Mbit/s from your specification
above of 1Mbit/s?  Or is this a different example?

 > And here lies the question again does this limit include layer 2 Ethernet
 > frame headers in the bit/s counting mechanism

I'd say not, but check with your provider that 1Mbit/s of IP is allowed;
that would be my expectation of such a spec, hereabouts anyway.

cheers, Ian
_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."