pf n00b

[ghe]

I'm fresh off the boat from Debian. I love OpenBSD's attitude, and the  
documentation is even pretty decipherable, but I'm still a little  
confused by pf. I managed to build a trivial filter, but there are a  
few things I don't understand.

I read somewhere (3 books, google, the website docs, and man) that a  
longer rule takes longer to do its work. Why? I don't understand how  
pf works -- I'd expect pfctl, while it's munging pf.conf, to make most  
of the conditions into a big mask that could just && with the IP  
header and make a decision on the result. So specifying the proto and  
both addresses and flags shouldn't make much difference in efficiency.  
No?

pf.conf consists largely of anchors (to fork on protocol) and sub-
anchors below them to fork on service -- I'm trying to reduce the  
count of rules seen by a packet to a minimum. But
pfctl -vs Anchors
gives this (in part):

> /root:# pfctl -vs Anchors
>   TCP_IN
>   TCP_IN/POP3
>   TCP_IN/SMTP
>   TCP_IN/SSH
>   TCP_IN/TCP_IN
>   TCP_IN/TCP_IN/POP3
>   TCP_IN/TCP_IN/SMTP
>   TCP_IN/TCP_IN/SSH

Here's the TCP_IN anchor:

> anchor "TCP_IN/*" in inet proto tcp from any port > 1023 to $OUTSIDE  
> flags S/SA {
>         anchor "SMTP/*" inet proto tcp to port smtp  {
>                 block drop in quick from <ASIA_BLK>
>                 block drop in quick from <SMTP_BLK>
>                 pass in quick
>         }
>         anchor "POP3/*" inet proto tcp to port pop3  {
>                 block drop in quick from <ASIA_BLK>
>                 block drop in quick from <POP_BLK>
>                 pass in quick
>         }
>         anchor "SSH/*" inet proto tcp to port ssh {
>                 block drop in quick from <ASIA_BLK>
>                 block drop in quick from <SSH_BLK>
>                 pass in quick
>         }
>         pass in inet proto tcp to port rsync
> }

Why does pfctl say there's a TCP_IN/TCP_IN?

Do there have to be "/*"s after all the anchor names?

Is it true that sub-anchors are 'evaluated' in alphabetical order as  
opposed to the order in the file? If so, is there a reason for this?

Is the pf list from Christmas Island still viable? I tried twice to  
subscribe and got bounced as spam both times.

And is there a way to get rid of an anchor without rebooting? When I  
change spellings,
pfctl -s Anchors
shows the old ones still in there.

TIA...

--
Glenn English
ghe@...

[Vadim Zhukov]

On 1 November 2009 c. 00:00:41 ghe wrote:

> I'm fresh off the boat from Debian. I love OpenBSD's attitude, and the
> documentation is even pretty decipherable, but I'm still a little
> confused by pf. I managed to build a trivial filter, but there are a
> few things I don't understand.
>
> I read somewhere (3 books, google, the website docs, and man) that a
> longer rule takes longer to do its work. Why? I don't understand how
> pf works -- I'd expect pfctl, while it's munging pf.conf, to make most
> of the conditions into a big mask that could just && with the IP
> header and make a decision on the result. So specifying the proto and
> both addresses and flags shouldn't make much difference in efficiency.
> No?

Not mask, it's a number of "if" checks to be done. But you should not
bother, it's fast enough, comparing to other things pf and network stack
do.

> pf.conf consists largely of anchors (to fork on protocol) and sub-
> anchors below them to fork on service -- I'm trying to reduce the
> count of rules seen by a packet to a minimum. But
> pfctl -vs Anchors

Bad idea. pf is not iptables. Read FAQ for examples, and start from
scratch using tricks from those examples, not from iptables. Sorry, I
wouldn't comment the next part of your message because you're moving in
the wrong direction anyway.

> Why does pfctl say there's a TCP_IN/TCP_IN?

Because you defined it, no? :)

> Do there have to be "/*"s after all the anchor names?

No, you need it just to evaluate subanchors of your anchor.

> Is it true that sub-anchors are 'evaluated' in alphabetical order as
> opposed to the order in the file? If so, is there a reason for this?

No.

> And is there a way to get rid of an anchor without rebooting? When I
> change spellings,
> pfctl -s Anchors
> shows the old ones still in there.

Yes, use "pfctl -f" or "pfctl -a anchorname -f" depending on what you
actually want to do.

--
  Best wishes,
    Vadim Zhukov

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

[Stuart Henderson]

On 2009-10-31, ghe <ghe@...> wrote:
> pf.conf consists largely of anchors (to fork on protocol) and sub-
> anchors below them to fork on service -- I'm trying to reduce the
> count of rules seen by a packet to a minimum. But

no need for that, we have automatic skip steps, and a ruleset
optimizer that re-orders where it makes sense.

see the 3 articles on undeadly about pf for some fundamentals,
starting here;

http://undeadly.org/cgi?action=article&sid=20060927091645

(this is for an old version; since then the optimizer is enabled
by default, pfctl -o isn't necessary).

[ghe]

On Oct 31, 2009, at 5:13 PM, Stuart Henderson wrote:

> no need for that, we have automatic skip steps, and a ruleset
> optimizer that re-orders where it makes sense.
>
> see the 3 articles on undeadly about pf for some fundamentals,
> starting here;
>
> http://undeadly.org/cgi?action=article&sid=20060927091645

Too bad Daniel's book deal fell through; I'd have understood things a  
lot better if it'd been at Amazon last week. This explains an awful  
lot -- nothing I saw before mentioned the skipping and reordering.  
Something somewhere gave me the impression that the pf optimization  
was just about adjusting timeouts. I don't have to think at all, do  
I? :-)

OTOH, I was pleased to see him write that what I was trying to do (use  
anchors as 'subroutines' to skip rules) would have worked, but that it  
just isn't necessary. Apparently I can just enter a bunch of rules in  
just about any old order and pf / pfctl will figure all that stuff out  
for me! (Within reason, I assume...)

And we n00bs are glad to know about undeadly, too...

Thanks, Stuart.

--
Glenn English
ghe@...

[Ryan McBride-3]

On Sat, Oct 31, 2009 at 03:00:41PM -0600, ghe wrote:
> I'm fresh off the boat from Debian. I love OpenBSD's attitude, and
> the documentation is even pretty decipherable, but I'm still a
> little confused by pf. I managed to build a trivial filter, but
> there are a few things I don't understand.
>
> I read somewhere (3 books, google, the website docs, and man) that a
> longer rule takes longer to do its work.

I can't speak for the books, and I KNOW google is full of lies, but can
you point out specifically what parts of the website docs and man page
talks about this? It should be removed.


> Why? I don't understand how pf works -- I'd expect pfctl, while it's
> munging pf.conf, to make most of the conditions into a big mask that
> could just && with the IP header and make a decision on the result.

PF is designed to have a considerably more flexible and fine-grained
filtering mechanism, so what goes on is considerably more complicated
than just a bitwise && against the header.


> So specifying the proto and both addresses and flags shouldn't make
> much difference in efficiency. No?

Actually, under many circumstances specifying the proto and addresses
will IMPROVE the performance of the ruleset evaluation even though it
makes the individual rule evaluation slower.

The number of rules evaluated makes a lot more difference than the
number of parameters evaluated per rule.


> pf.conf consists largely of anchors (to fork on protocol) and sub-
> anchors below them to fork on service -- I'm trying to reduce the
> count of rules seen by a packet to a minimum. But

This approach is almost guaranteed to have the opposite effect.



My number one advice for people who want to optimize their rulesets for
performance is: DON'T.

Seriously.

Writing firewall rules is hard, anything more than a trivial ruleset is
easy to screw up and challenging to test. So the #1 goal for your
ruleset should be readability and maintainability. While you're at it,
put your ruleset under revision control, and figure out a good way to
test any ruleset changes that get made.


That being said, here are some things you can do while you're doing the
above which will help performance.

        - stateful filtering (don't use 'no state')
        - pfctl optimizer (don't use 'set ruleset-optimization none')
        - use tables for lists of addresses
        - use as few rules as possible to get the filtering you want
          while keeping the ruleset readable.


Now, if you really, really need to optimize your ruleset for
performance, it's important to know that PF doesn't simply walk through
the rules as you've specified in your pf.conf when; pfctl optimizes the
rules as they are loaded into the kernel, and PF has a mechanism called
'skip steps' which will skip evaluation of rules if it's known in
advance that the rules cannot possibly match.

The skip steps attributes are the following:

        * interface
        * direction
        * address family
        * protocol
        * source address
        * source port
        * destination address
        * destination port

The best thing that you can actively do for ruleset performance is to
get out of the way of these mechanisms.

- Make use of the rule expansion in pf.conf (rules with items listed in
  braces { }) rather than manually expanding them. The expansion is done
  by pfctl in the skip steps order.
- Group your rules by the skip-steps parameters, in the order above.
  (ie, all rules for em0 together; within that group, all the 'in' rules
  together, within that, all ipv4 rules together...)
- For the above parameters, specify as much detail as possible without
  adding more rules; increased detail will give skip-steps more to work
  with.
- Make sparing use of "barrier" rule options, which prevent the ruleset
  optimizer from reordering the ruleset efficiently.
        * label
        * prob
        * state limits
                * max_states
                * max_src_nodes
                * max_src_stats
                * max_src_conn
                * max_src_conn_rate
        * anchor
- This means: Don't break the ruleset into anchors for performance
  reasons unless you _really_ know what you're doing. If you DO, it's
  probably best to break it up in skip-steps order (ie, by interface
  first).




If you're STILL having performance issues after this, there are a few
more things you can do. Remember though:

        "Premature optimization is the root of all evil"
                -- Donald Knuth

- Write your ruleset with 'first match' ordering (quick) on
  all/most rules, and use the 'profile' ruleset-optimization
- Filter only on one side of the firewall, using 'set skip' for the
  interface on the other side.
- Group rules using optimizer "breaks" together; the optimizer
  will only reorder or merge them if they are the same. (Roughly stated,
  these are the 'actions' that a rule can perform).
        * tag
        * keep state
        * queue
        * routing (rt, route-to, dup-to, reply-to)
        * nat/rdr
        * options
        * action (pass/block)
        * logging
        * quick
        * return_ttl
- Think hard about whether some categories of rules are necessary.
  Things like "stealth firewall" filtering against nmap probes, are
  more-or-less useless for security, do you really need these in your
  ruleset?

These shouldn't be done unless you _really_ understand the effects,
because these depend a great deal on your ruleset specifics, your
traffic, and your configuration. At this point you probably also want to
be looking at hardware and network architecture changes to improve
performance as well.



Oh, did I mention that I don't recommend optimizing your ruleset for
performance?

-Ryan

[ghe]

On Nov 1, 2009, at 1:12 AM, Toma Bodar wrote:

> I don't know if you find one document about PF, but here it is
> http://home.nuug.no/~peter/pf/en/  same author wrote book about PF.

Yup. That's one of the books I read -- but pf seems to have moved
since then. Thanks for the link to this major update.

--
Glenn English
ghe@...

[ghe]

On Oct 31, 2009, at 9:26 PM, Ryan McBride wrote:

> I can't speak for the books, and I KNOW google is full of lies, but  
> can
> you point out specifically what parts of the website docs and man page
> talks about this? It should be removed.

After going through the replies I've received, I'm thinking my trouble  
is probably a combination of last version (or older) info and my own  
misunderstandings.

You guys have pointed me to a number of places that google apparently  
didn't know about.

> The number of rules evaluated makes a lot more difference than the
> number of parameters evaluated per rule.

That's what I was trying to accomplish with the 'branching' anchors.

> My number one advice for people who want to optimize their rulesets  
> for
> performance is: DON'T.

So I hear :-)

> That being said, here are some things you can do while you're doing  
> the
> above which will help performance.
>
> - stateful filtering (don't use 'no state')
> - pfctl optimizer (don't use 'set ruleset-optimization none')
> - use tables for lists of addresses
> - use as few rules as possible to get the filtering you want
>  while keeping the ruleset readable.

The big (huge) thing I didn't know about is the optimizer. I was  
already aware of the others.

Thanks, Ryan. I'm used to hand optimizing for Cisco and iptables. I  
seem to be not in Kansas any more -- let's see how well his optimizer  
thing works...

--
Glenn English
ghe@...

[ghe]

On Oct 31, 2009, at 3:33 PM, Vadim Zhukov wrote:

> Bad idea. pf is not iptables. Read FAQ for examples, and start from
> scratch using tricks from those examples, not from iptables.

My biggest problem seems to have been total ignorance of the depth of  
the optimizer. I didn't see much in the way of tricks in the examples.  
I'll revisit them from a new direction.

>> Why does pfctl say there's a TCP_IN/TCP_IN?
>
> Because you defined it, no? :)

No. Not that I know of, anyway. I defined TCP_IN and TCP_IN/SMTP, but  
not TCP_IN/TCP_IN. So I guess my question should be, "Where/how did I  
define TCP_IN/TCP_IN?"

> No, you need it just to evaluate subanchors of your anchor.

So it'd need to be there for all but the lowest level?

--
Glenn English
ghe@...

[ghe]

On Oct 31, 2009, at 5:13 PM, Stuart Henderson wrote:

> no need for that, we have automatic skip steps, and a ruleset
> optimizer that re-orders where it makes sense.

Well, I'll be damned. The pf optimizer actually works! If I order the  
rules properly and put in enough info into them that pf can tell what  
I mean, the compiled ruleset skips over huge hunks of rules.

This does bring a question to my mind, though. Why is this ruleset  
optimization kept a secret? It's a *very* major piece of pf, IMHO. I  
did a significant amount of reading and looking around, and I never  
saw it discussed in any detail at all until I asked the list about my  
iptables wannabe pf ruleset...

As somebody said at undeadly, dhartmei's writings on pf at undeadly
( http://undeadly.org/cgi?action=article&sid=20060927091645 )
should be in the docs at OpenBSD.org.

Thanks to all who responded. Hopefully I can figure out the rest.

--
Glenn English
ghe@...

[fuzzyping]

On Sun, Nov 01, 2009 at 01:16:10PM -0700, ghe wrote:

> On Oct 31, 2009, at 5:13 PM, Stuart Henderson wrote:
>
>> no need for that, we have automatic skip steps, and a ruleset
>> optimizer that re-orders where it makes sense.
>
> Well, I'll be damned. The pf optimizer actually works! If I order the  
> rules properly and put in enough info into them that pf can tell what I
> mean, the compiled ruleset skips over huge hunks of rules.
>
> This does bring a question to my mind, though. Why is this ruleset  
> optimization kept a secret? It's a *very* major piece of pf, IMHO. I did
> a significant amount of reading and looking around, and I never saw it
> discussed in any detail at all until I asked the list about my iptables
> wannabe pf ruleset...

Because it just works the way a firewall *should*?  The OpenBSD
developers aren't distratcted by World Domination (TM) like some other
operating systems.

--
Jason Dixon
DixonGroup Consulting
http://www.dixongroup.net/

[Ted Unangst-2]

On Sun, Nov 1, 2009 at 4:16 PM, ghe <ghe@...> wrote:
> This does bring a question to my mind, though. Why is this ruleset
> optimization kept a secret? It's a *very* major piece of pf, IMHO. I did a
> significant amount of reading and looking around, and I never saw it
> discussed in any detail at all until I asked the list about my iptables
> wannabe pf ruleset...

The optimizer is documented in both the pfctl and pf.conf man pages,
and the one for pf.conf tells you exactly what it does.

[ghe]

On Nov 1, 2009, at 3:08 PM, Ted Unangst wrote:

> The optimizer is documented in both the pfctl and pf.conf man pages,
> and the one for pf.conf tells you exactly what it does.

In pfctl's man page (4.6), there is a statement that the kernel  
sometimes skips rules -- no mention of the optimizer playing any part  
in it.

The string "skip" appears three times in pf.conf's man page. Twice in  
relation to the "skip on <interface>" rule and once describing the  
effect of the "quick" keyword.

It does say that ruleset-optimization can "improve performance" and  
may reorder the rules, but it's pretty light on just what that might  
mean: nothing about examining rules to be able to skip evaluating  
some. Nor is there anything about the hierarchy of fields it uses to  
examine them. All this *is* explained at undead. A little cut&paste is  
in order here, IMHO.

I still claim this is a major 'undocumented feature' of pf. And that  
if this functionality had been implemented anywhere else, there'd be a  
lot of noise about it. I've spent years working on iptables and Cisco  
filters, hand optimizing them to get just part what pf gives me for  
next to free.

The earlier poster (Jason) is right: this *is* the way a firewall  
should work -- spend your time on implementing the security policy and  
let the 'compiler' worry about efficiency. But since the others don't,  
it might be a good idea to go into this at some length.

--
Glenn English
ghe@...

[Theo de Raadt]

> The earlier poster (Jason) is right: this *is* the way a firewall  
> should work -- spend your time on implementing the security policy and  
> let the 'compiler' worry about efficiency. But since the others don't,  
> it might be a good idea to go into this at some length.

Since it just does what a good system should do, what is there to go
into "at length" about?

Yes, other systems taught you to hand-optimise.  How do we go into
"don't do hand optimization" at length?

It is a manual page, not a howto.

[ghe]

On Nov 1, 2009, at 4:11 PM, Theo de Raadt wrote:

> Since it just does what a good system should do, what is there to go
> into "at length" about?

What it does. How it does it. If that were documented, it'd sure be  
easier to use the tools more effectively.

> Yes, other systems taught you to hand-optimise.  How do we go into
> "don't do hand optimization" at length?

http://undeadly.org/cgi?action=article&sid=20060927091645

> It is a manual page, not a howto.

I was responding here to the remark about the man pages, and making  
the point that, IMHO, the statement was not correct. The FAQ at the  
website and the books I'd been able to find don't explain this area  
either, although they do go into other areas in detail.

The earlier posts told me where to go to fill in a lot of holes; the  
info's out there. It's just hard to find for someone new to the  
'culture' (who didn't know about undead yet).

And it strikes me as odd that what looks like a significant advantage  
over other ways of doing things is so kept under wraps. I've been  
running OpenBSD for only a few days, and I'm just beginning to take a  
few baby steps -- I'm sorry if I stepped on some toes. I certainly  
didn't intend to.

FWIW, I'm willing to put my time where my mouth is and see if I could  
fix what I claim is bent. I'm way not qualified to do that on my own  
yet, but I might be able to help...

--
Glenn English
ghe@...

[Theo de Raadt]

> > Since it just does what a good system should do, what is there to go
> > into "at length" about?
>
> What it does. How it does it. If that were documented, it'd sure be  
> easier to use the tools more effectively.

It does what it does, how it does it, in the source code.  Manual pages
do not serve those purposes.

pf(4) describes what pf is capable of.
pf.conf(5) describes the grammer used to communicate with pf.
pfctl(8) describes the flags and features of the front-end parser
that converts text rules to requests against pf(4)'s capabilities.

None of the manual pages can exhaustively describe the workings of pf
without losing their purpose.  Each of those manual pages are already
far too long as it is.

> > Yes, other systems taught you to hand-optimise.  How do we go into
> > "don't do hand optimization" at length?
>
> http://undeadly.org/cgi?action=article&sid=20060927091645
>
> > It is a manual page, not a howto.
>
> I was responding here to the remark about the man pages, and making  
> the point that, IMHO, the statement was not correct.

That posting was written more than 3 years ago.  Then, as now, it was
written _in that forum_ because it does not belong in the manual pages.

> The FAQ at the  
> website and the books I'd been able to find don't explain this area  
> either, although they do go into other areas in detail.

Perhaps the way that pf works is not a FAQ.  And perhaps the book authors
did not research deeply enough into how pf works, to be able to write the
best book.  Perhaps their books simply regurgitate the "best most common"
ways to use pf.  I'd say that is fine for most.  You want to dig in deeper?
Well, the source code is available.

And yes, if lots of people are using pf wrong?  So what.  It meets their
purposes.

> The earlier posts told me where to go to fill in a lot of holes; the  
> info's out there. It's just hard to find for someone new to the  
> 'culture' (who didn't know about undead yet).
>
> And it strikes me as odd that what looks like a significant advantage  
> over other ways of doing things is so kept under wraps.

You've been told twice now that it is not under wraps.  Stop it:

     set ruleset-optimization
             basic     Enable basic ruleset optimization.  This is the default
                       behaviour.  Basic ruleset optimization does four things
                       to improve the performance of ruleset evaluations:

                       1.   remove duplicate rules
                       2.   remove rules that are a subset of another rule
                       3.   combine multiple rules into a table when advanta-
                            geous
                       4.   re-order the rules to improve evaluation perfor-
                            mance
             none      Disable the ruleset optimizer.
             profile   Uses the currently loaded ruleset as a feedback profile
                       to tailor the ordering of quick rules to actual network
                       traffic.

             It is important to note that the ruleset optimizer will modify
             the ruleset to improve performance.  A side effect of the ruleset
             modification is that per-rule accounting statistics will have
             different meanings than before.  If per-rule accounting is impor-
             tant for billing purposes or whatnot, either the ruleset optimiz-
             er should not be used or a label field should be added to all of
             the accounting rules to act as optimization barriers.

             Optimization can also be set as a command-line argument to
             pfctl(8), overriding the settings in pf.conf.

> I've been  
> running OpenBSD for only a few days, and I'm just beginning to take a  
> few baby steps -- I'm sorry if I stepped on some toes. I certainly  
> didn't intend to.
>
> FWIW, I'm willing to put my time where my mouth is and see if I could  
> fix what I claim is bent. I'm way not qualified to do that on my own  
> yet, but I might be able to help...

We always accept diffs.  If the manual pages are not super clear, try
to write small diffs to help improve them.  First of all that means
you must understand the "central purpose" to each manual page.
Whatever gets added to it must follow in the main direction of the
page.