Extension of dummynet/ipfw to support userspace packet classification

[Joe R-4]

We at ironport have a requirement to do bandwidth management, but the
traffic classification (and selection of bandwidth pipes) is done in
userspace. The reason classification is done in userspace is because the
traffic classifications are something like streaming audio traffic, video
traffic, based on website categories etc.



Our appliance is based on FreeBSD, and so we decided to look at dummynet to
support our requirement. We could not use dummynet as such because it uses
ipfw for packet classification, where packet classification (and pipe
selection) is done in kernel based on tcp/ip parameters like IP and port.



So we decided to extended dummynet/ipfw to support packet classification in
userspace.

Our idea is to extended socket structure to have a pipe number and have a
setsockoption to associate the pipe number to a socket structure. Then have
a new ipfw target (mappedpipe), which will pass the packet to dummynet
(similar to pipe target) but with the pipe number in the socket structure if
it is non-zero.



I would like to know your comments on this proposal and if people are
interested, I will be happy to submit a patch on this.





Thanks,

Joe.
_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."

[Guy Helmer]

Joe R wrote:

> We at ironport have a requirement to do bandwidth management, but the
> traffic classification (and selection of bandwidth pipes) is done in
> userspace. The reason classification is done in userspace is because the
> traffic classifications are something like streaming audio traffic, video
> traffic, based on website categories etc.
>
>
>
> Our appliance is based on FreeBSD, and so we decided to look at dummynet to
> support our requirement. We could not use dummynet as such because it uses
> ipfw for packet classification, where packet classification (and pipe
> selection) is done in kernel based on tcp/ip parameters like IP and port.
>
>
>
> So we decided to extended dummynet/ipfw to support packet classification in
> userspace.
>
> Our idea is to extended socket structure to have a pipe number and have a
> setsockoption to associate the pipe number to a socket structure. Then have
> a new ipfw target (mappedpipe), which will pass the packet to dummynet
> (similar to pipe target) but with the pipe number in the socket structure if
> it is non-zero.
>
>
>
> I would like to know your comments on this proposal and if people are
> interested, I will be happy to submit a patch on this.
>
>  

I think it would be a very useful capability to apply a dummynet pipe to
a stream.

My thinking was that it would be nice to be able to build a dynamic
table of connections in ipfw and then ipfw could pass packets that
matched the dynamic connections list through a specified dummynet pipe.  
I think that is different than your design, though -- as I understand
it, your design would apply dummynet to packets written to a socket.

Guy
_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."

[Julian Elischer]

Guy Helmer wrote:

> Joe R wrote:
>> We at ironport have a requirement to do bandwidth management, but the
>> traffic classification (and selection of bandwidth pipes) is done in
>> userspace. The reason classification is done in userspace is because the
>> traffic classifications are something like streaming audio traffic, video
>> traffic, based on website categories etc.
>>
>>
>>
>> Our appliance is based on FreeBSD, and so we decided to look at
>> dummynet to
>> support our requirement. We could not use dummynet as such because it
>> uses
>> ipfw for packet classification, where packet classification (and pipe
>> selection) is done in kernel based on tcp/ip parameters like IP and port.
>>
>>
>>
>> So we decided to extended dummynet/ipfw to support packet
>> classification in
>> userspace.
>>
>> Our idea is to extended socket structure to have a pipe number and have a
>> setsockoption to associate the pipe number to a socket structure. Then
>> have
>> a new ipfw target (mappedpipe), which will pass the packet to dummynet
>> (similar to pipe target) but with the pipe number in the socket
>> structure if
>> it is non-zero.
>>
>>
>>
>> I would like to know your comments on this proposal and if people are
>> interested, I will be happy to submit a patch on this.
>>
>>  
> I think it would be a very useful capability to apply a dummynet pipe to
> a stream.
>
> My thinking was that it would be nice to be able to build a dynamic
> table of connections in ipfw and then ipfw could pass packets that
> matched the dynamic connections list through a specified dummynet pipe.  
> I think that is different than your design, though -- as I understand
> it, your design would apply dummynet to packets written to a socket.
>
> Guy

What they want to do is what I was going to do before I "left"
there .. which is to allow a userland process (e.g. proxy) classify
the session using some un-named method , assign some session key
to the socket that can be attached to the mbufs in some way
as they are generated.  an in-kernel flow control module (e.g.
dummynet) could then be left to enforce the bandwidth usage by that
session.

When I originally laid this out I thought we'd need
the following parts working to allow this to happen:

* ioctl to add value to a new field in the socket.
* a place to store a copy of the field in the mbuf, OR
   a way to reference the one in the socket.
* a way to get such packets to the right dummynet pipe.
   e.g. a new ipfw rule type.
* A frontend to set up the pipes (not our problem).





_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."

[Luigi Rizzo-5]

On Wed, Oct 07, 2009 at 12:46:24PM -0700, Joe R wrote:

> We at ironport have a requirement to do bandwidth management, but the
> traffic classification (and selection of bandwidth pipes) is done in
> userspace. The reason classification is done in userspace is because the
> traffic classifications are something like streaming audio traffic, video
> traffic, based on website categories etc.
>
>
>
> Our appliance is based on FreeBSD, and so we decided to look at dummynet to
> support our requirement. We could not use dummynet as such because it uses
> ipfw for packet classification, where packet classification (and pipe
> selection) is done in kernel based on tcp/ip parameters like IP and port.
>
>
>
> So we decided to extended dummynet/ipfw to support packet classification in
> userspace.
>
> Our idea is to extended socket structure to have a pipe number and have a
> setsockoption to associate the pipe number to a socket structure. Then have
> a new ipfw target (mappedpipe), which will pass the packet to dummynet
> (similar to pipe target) but with the pipe number in the socket structure if
> it is non-zero.
>
>
>
> I would like to know your comments on this proposal and if people are
> interested, I will be happy to submit a patch on this.

i think the feature is useful. However I would implement it as an
ipfw 'option' called "sockarg" (or similar) as follows:

        ipfw pipe tablearg sockarg

where 'sockarg' succeeds ONLY if the packet is associated to a socket
for which the special setsockoption has been issued, and in this
case sets the 'tablearg' to the value of the setsockopt. This is
somewhat similar to the 'uid' and 'gid' options (except for setting
tablearg).  This way the mechanism can be very general (not limited
to pipes) and the implementation is probably
simpler than the one you propose.

In terms of runtime costs, we can look at check_uidgid() function,
and there are two ways to implement this feature:
- as in check_uidgid() , actively lookup for a matching socket if one
  is not available. This is expensive but would allow the feature to
  match also incoming packets;
- only match if the args->inp parameter is non-null, otherwise do not
  call in_pcblookup_hash(). This is cheaper but clearly only works
  for locally generated packets.
Perhaps we could use an argument for 'sockarg' so we can decide
whether to call or not the in_pcblookup_hash() on a case-by-case
basis.

cheers
luigi
_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."

[Luigi Rizzo-5]

On Thu, Oct 08, 2009 at 12:54:52AM +0200, Luigi Rizzo wrote:

> On Wed, Oct 07, 2009 at 12:46:24PM -0700, Joe R wrote:
> > We at ironport have a requirement to do bandwidth management, but the
> > traffic classification (and selection of bandwidth pipes) is done in
> > userspace. The reason classification is done in userspace is because the
> > traffic classifications are something like streaming audio traffic, video
> > traffic, based on website categories etc.
> >
> >
> >
> > Our appliance is based on FreeBSD, and so we decided to look at dummynet to
> > support our requirement. We could not use dummynet as such because it uses
> > ipfw for packet classification, where packet classification (and pipe
> > selection) is done in kernel based on tcp/ip parameters like IP and port.
> >
> >
> >
> > So we decided to extended dummynet/ipfw to support packet classification in
> > userspace.
> >
> > Our idea is to extended socket structure to have a pipe number and have a
> > setsockoption to associate the pipe number to a socket structure. Then have
> > a new ipfw target (mappedpipe), which will pass the packet to dummynet
> > (similar to pipe target) but with the pipe number in the socket structure if
> > it is non-zero.
> >
> >
> >
> > I would like to know your comments on this proposal and if people are
> > interested, I will be happy to submit a patch on this.
>
> i think the feature is useful. However I would implement it as an
> ipfw 'option' called "sockarg" (or similar) as follows:
>
> ipfw pipe tablearg sockarg
>
> where 'sockarg' succeeds ONLY if the packet is associated to a socket
> for which the special setsockoption has been issued, and in this
> case sets the 'tablearg' to the value of the setsockopt. This is
> somewhat similar to the 'uid' and 'gid' options (except for setting
> tablearg).  This way the mechanism can be very general (not limited
> to pipes) and the implementation is probably
> simpler than the one you propose.
>
> In terms of runtime costs, we can look at check_uidgid() function,
> and there are two ways to implement this feature:
> - as in check_uidgid() , actively lookup for a matching socket if one
>   is not available. This is expensive but would allow the feature to
>   match also incoming packets;
> - only match if the args->inp parameter is non-null, otherwise do not
>   call in_pcblookup_hash(). This is cheaper but clearly only works
>   for locally generated packets.
> Perhaps we could use an argument for 'sockarg' so we can decide
> whether to call or not the in_pcblookup_hash() on a case-by-case
> basis.

To complete the analysis, I must say that I don't know how intrusive
is the setsockopt that can attach a classification tag to the socket.
This is my main concern for merging your proposal into the system
(and i am only concerned about the socket part, the ipfw change is
trivial).

Also for completeness, there is also another possible approach to
address your problem, which is more general and fully contained in
ipfw (so less intrusive for the OS):

  add a 'hashtable' structure to ipfw, which works in a way similar
  to the 'table' with the difference that entries would be the whole
  5-tuple of the packet.

There is already a hash table in ipfw (used for dynamic rules) so
it would be only a matter of adding the necessary glue to manipulate
the hash table from /sbin/ipfw. An additional bonus of this approach
is that one could use this new code to 'prime' the dynamic rule table
after a reboot, which is a feature that people ask from time to time.

        cheers
        luigi
_______________________________________________
freebsd-ipfw@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw
To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@..."