pf changes the order of filtering rules

Hello!

I have strange behavior of pf on my 4.6 box.

Filtering rules are present in pf.conf in next order:
block in all
pass in quick on $ext_if proto tcp from any to ($ext_if) port ssh
pass out quick on $ext_if
pass in quick on $ext_if no state
pass in  quick on vlan609 from vlan609:network  to any                  no state
pass out quick on vlan609 from any              to vlan609:network      no state
pass in  quick on vlan621 from 10.51.109.16/29  to any                  no state
pass out quick on vlan621 from any              to 10.51.109.16/29      no state queue to_Akim
pass in  quick on vlan621 from 10.51.109.40/29  to any                  no state
pass out quick on vlan621 from any              to 10.51.109.40/29      no state queue to_Gonta
pass in  quick on vlan622 from vlan622:network  to any                  no state
pass out quick on vlan622 from any              to vlan622:network      no state
pass in  quick on vlan664 from vlan664:network  to any                  no state
pass out quick on vlan664 from any              to vlan664:network      no state
pass in  quick on vlan781 from vlan781:network  to any                  no state
pass out quick on vlan781 from any              to vlan781:network      no state
pass in  quick on vlan783 from vlan783:network  to any                  no state
pass out quick on vlan783 from any              to vlan783:network      no state



But after they loaded pfctl -sr shows another order:
block drop in all
pass in quick on vlan2 proto tcp from any to (vlan2) port = ssh flags S/SA keep state (if-bound)
pass out quick on vlan2 all flags S/SA keep state (if-bound)
pass in quick on vlan609 inet from 10.51.9.0/24 to any no state
pass in quick on vlan621 inet from 10.51.109.16/29 to any no state
pass in quick on vlan2 all no state
pass out quick on vlan609 inet from any to 10.51.9.0/24 no state
pass out quick on vlan621 inet from any to 10.51.109.16/29 no state queue to_Akim
pass in quick on vlan621 inet from 10.51.109.40/29 to any no state
pass out quick on vlan621 inet from any to 10.51.109.40/29 no state queue to_Gonta
pass in quick on vlan622 inet from 10.51.109.0/28 to any no state
pass in quick on vlan622 inet from 10.51.109.56/29 to any no state
pass in quick on vlan781 inet from 10.53.31.0/25 to any no state
pass in quick on vlan781 inet from 10.53.31.128/25 to any no state
pass in quick on vlan664 inet from 10.52.14.0/24 to any no state
pass in quick on vlan783 inet from 10.53.33.0/24 to any no state
pass out quick on vlan622 inet from any to 10.51.109.0/28 no state
pass out quick on vlan622 inet from any to 10.51.109.56/29 no state
pass out quick on vlan781 inet from any to 10.53.31.0/25 no state
pass out quick on vlan781 inet from any to 10.53.31.128/25 no state
pass out quick on vlan664 inet from any to 10.52.14.0/24 no state
pass out quick on vlan783 inet from any to 10.53.33.0/24 no state

Does anyone know how to disable this? Thanks in advance!

--
MINO-RIPE

[Paul de Weerd]

On Tue, Nov 03, 2009 at 03:32:29PM +0200, Alexander Shikoff wrote:
| Hello!
|
| I have strange behavior of pf on my 4.6 box.
|
| Filtering rules are present in pf.conf in next order:
| block in all
| pass in quick on $ext_if proto tcp from any to ($ext_if) port ssh
| pass out quick on $ext_if
| pass in quick on $ext_if no state
| pass in  quick on vlan609 from vlan609:network  to any                  no state
| pass out quick on vlan609 from any              to vlan609:network      no state
| pass in  quick on vlan621 from 10.51.109.16/29  to any                  no state
| pass out quick on vlan621 from any              to 10.51.109.16/29      no state queue to_Akim
| pass in  quick on vlan621 from 10.51.109.40/29  to any                  no state
| pass out quick on vlan621 from any              to 10.51.109.40/29      no state queue to_Gonta
| pass in  quick on vlan622 from vlan622:network  to any                  no state
| pass out quick on vlan622 from any              to vlan622:network      no state
| pass in  quick on vlan664 from vlan664:network  to any                  no state
| pass out quick on vlan664 from any              to vlan664:network      no state
| pass in  quick on vlan781 from vlan781:network  to any                  no state
| pass out quick on vlan781 from any              to vlan781:network      no state
| pass in  quick on vlan783 from vlan783:network  to any                  no state
| pass out quick on vlan783 from any              to vlan783:network      no state
|
|
|
| But after they loaded pfctl -sr shows another order:
| block drop in all
| pass in quick on vlan2 proto tcp from any to (vlan2) port = ssh flags S/SA keep state (if-bound)
| pass out quick on vlan2 all flags S/SA keep state (if-bound)
| pass in quick on vlan609 inet from 10.51.9.0/24 to any no state
| pass in quick on vlan621 inet from 10.51.109.16/29 to any no state
| pass in quick on vlan2 all no state
| pass out quick on vlan609 inet from any to 10.51.9.0/24 no state
| pass out quick on vlan621 inet from any to 10.51.109.16/29 no state queue to_Akim
| pass in quick on vlan621 inet from 10.51.109.40/29 to any no state
| pass out quick on vlan621 inet from any to 10.51.109.40/29 no state queue to_Gonta
| pass in quick on vlan622 inet from 10.51.109.0/28 to any no state
| pass in quick on vlan622 inet from 10.51.109.56/29 to any no state
| pass in quick on vlan781 inet from 10.53.31.0/25 to any no state
| pass in quick on vlan781 inet from 10.53.31.128/25 to any no state
| pass in quick on vlan664 inet from 10.52.14.0/24 to any no state
| pass in quick on vlan783 inet from 10.53.33.0/24 to any no state
| pass out quick on vlan622 inet from any to 10.51.109.0/28 no state
| pass out quick on vlan622 inet from any to 10.51.109.56/29 no state
| pass out quick on vlan781 inet from any to 10.53.31.0/25 no state
| pass out quick on vlan781 inet from any to 10.53.31.128/25 no state
| pass out quick on vlan664 inet from any to 10.52.14.0/24 no state
| pass out quick on vlan783 inet from any to 10.53.33.0/24 no state
|
| Does anyone know how to disable this? Thanks in advance!

Why do you want to disable this ? And why are you using no state ?
What you're seeing is the result of the ruleset optimizer. See
pf.conf(5) for more details and how to disable this. My suggestion is
to *not* disable it though. What problem does the reordering give
you ? Maybe you want to look into antispoof and urpf too, while you're
at it.

And, really, why are you using 'no state' ?

Paul 'WEiRD' de Weerd

--
>++++++++[<++++++++++>-]<+++++++.>+++[<------>-]<.>+++[<+
+++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
                 http://www.weirdnet.nl/                 

[Henning Brauer]

* Alexander Shikoff <minotaur@...> [2009-11-03 14:40]:
> I have strange behavior of pf on my 4.6 box.

no.

> Filtering rules are present in pf.conf in next order:
> But after they loaded pfctl -sr shows another order:

how about reading the manpages for a change. sometimes i wonder why we
write them.

oh sorry - forgot the HOWTO:
1) get shell access
2) type "man pf.conf"
3) look for "optimization"

--
Henning Brauer, hb@..., henning@...
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting

On Tue, Nov 03, 2009 at 02:49:36PM +0100, Paul de Weerd wrote:

> On Tue, Nov 03, 2009 at 03:32:29PM +0200, Alexander Shikoff wrote:
> | Hello!
> |
> | I have strange behavior of pf on my 4.6 box.
> |
> | Filtering rules are present in pf.conf in next order:
> | block in all
> | pass in quick on $ext_if proto tcp from any to ($ext_if) port ssh
> | pass out quick on $ext_if
> | pass in quick on $ext_if no state
> | pass in  quick on vlan609 from vlan609:network  to any                  no state
> | pass out quick on vlan609 from any              to vlan609:network      no state
> | pass in  quick on vlan621 from 10.51.109.16/29  to any                  no state
> | pass out quick on vlan621 from any              to 10.51.109.16/29      no state queue to_Akim
> | pass in  quick on vlan621 from 10.51.109.40/29  to any                  no state
> | pass out quick on vlan621 from any              to 10.51.109.40/29      no state queue to_Gonta
> | pass in  quick on vlan622 from vlan622:network  to any                  no state
> | pass out quick on vlan622 from any              to vlan622:network      no state
> | pass in  quick on vlan664 from vlan664:network  to any                  no state
> | pass out quick on vlan664 from any              to vlan664:network      no state
> | pass in  quick on vlan781 from vlan781:network  to any                  no state
> | pass out quick on vlan781 from any              to vlan781:network      no state
> | pass in  quick on vlan783 from vlan783:network  to any                  no state
> | pass out quick on vlan783 from any              to vlan783:network      no state
> |
> |
> |
> | But after they loaded pfctl -sr shows another order:
> | block drop in all
> | pass in quick on vlan2 proto tcp from any to (vlan2) port = ssh flags S/SA keep state (if-bound)
> | pass out quick on vlan2 all flags S/SA keep state (if-bound)
> | pass in quick on vlan609 inet from 10.51.9.0/24 to any no state
> | pass in quick on vlan621 inet from 10.51.109.16/29 to any no state
> | pass in quick on vlan2 all no state
> | pass out quick on vlan609 inet from any to 10.51.9.0/24 no state
> | pass out quick on vlan621 inet from any to 10.51.109.16/29 no state queue to_Akim
> | pass in quick on vlan621 inet from 10.51.109.40/29 to any no state
> | pass out quick on vlan621 inet from any to 10.51.109.40/29 no state queue to_Gonta
> | pass in quick on vlan622 inet from 10.51.109.0/28 to any no state
> | pass in quick on vlan622 inet from 10.51.109.56/29 to any no state
> | pass in quick on vlan781 inet from 10.53.31.0/25 to any no state
> | pass in quick on vlan781 inet from 10.53.31.128/25 to any no state
> | pass in quick on vlan664 inet from 10.52.14.0/24 to any no state
> | pass in quick on vlan783 inet from 10.53.33.0/24 to any no state
> | pass out quick on vlan622 inet from any to 10.51.109.0/28 no state
> | pass out quick on vlan622 inet from any to 10.51.109.56/29 no state
> | pass out quick on vlan781 inet from any to 10.53.31.0/25 no state
> | pass out quick on vlan781 inet from any to 10.53.31.128/25 no state
> | pass out quick on vlan664 inet from any to 10.52.14.0/24 no state
> | pass out quick on vlan783 inet from any to 10.53.33.0/24 no state
> |
> | Does anyone know how to disable this? Thanks in advance!
>
> Why do you want to disable this ? And why are you using no state ?
> What you're seeing is the result of the ruleset optimizer. See
> pf.conf(5) for more details and how to disable this. My suggestion is
> to *not* disable it though. What problem does the reordering give
> you ? Maybe you want to look into antispoof and urpf too, while you're
> at it.

Oh, my fault. Thanks for the tip.
 
> And, really, why are you using 'no state' ?
Because I need queuing for outgoing traffic on vlan* interfaces.
When keep state is used then queues for outgoing traffic do not work.
 
> Paul 'WEiRD' de Weerd
>
> --
> >++++++++[<++++++++++>-]<+++++++.>+++[<------>-]<.>+++[<+
> +++++++++++>-]<.>++[<------------>-]<+.--------------.[-]
>                  http://www.weirdnet.nl/                 

--
MINO-RIPE

[Vadim Zhukov]

On 3 November 2009 G. 16:32:29 Alexander Shikoff wrote:

> Hello!
>
> I have strange behavior of pf on my 4.6 box.
>
> Filtering rules are present in pf.conf in next order:
> block in all
> pass in quick on $ext_if proto tcp from any to ($ext_if) port ssh
> pass out quick on $ext_if
> pass in quick on $ext_if no state
> pass in  quick on vlan609 from vlan609:network  to any
>  no state pass out quick on vlan609 from any              to
> vlan609:network      no state pass in  quick on vlan621 from
> 10.51.109.16/29  to any                  no state pass out quick on
> vlan621 from any              to 10.51.109.16/29      no state queue
> to_Akim pass in  quick on vlan621 from 10.51.109.40/29  to any
>          no state pass out quick on vlan621 from any              to
> 10.51.109.40/29      no state queue to_Gonta pass in  quick on vlan622
> from vlan622:network  to any                  no state pass out quick
> on vlan622 from any              to vlan622:network      no state pass
> in  quick on vlan664 from vlan664:network  to any                  no
> state pass out quick on vlan664 from any              to
> vlan664:network      no state pass in  quick on vlan781 from
> vlan781:network  to any                  no state pass out quick on
> vlan781 from any              to vlan781:network      no state pass in
>  quick on vlan783 from vlan783:network  to any                  no
> state pass out quick on vlan783 from any              to
> vlan783:network      no state
>
>
>
> But after they loaded pfctl -sr shows another order:
> block drop in all
> pass in quick on vlan2 proto tcp from any to (vlan2) port = ssh flags
> S/SA keep state (if-bound) pass out quick on vlan2 all flags S/SA keep
> state (if-bound)
> pass in quick on vlan609 inet from 10.51.9.0/24 to any no state
> pass in quick on vlan621 inet from 10.51.109.16/29 to any no state
> pass in quick on vlan2 all no state
> pass out quick on vlan609 inet from any to 10.51.9.0/24 no state
> pass out quick on vlan621 inet from any to 10.51.109.16/29 no state
> queue to_Akim pass in quick on vlan621 inet from 10.51.109.40/29 to
> any no state pass out quick on vlan621 inet from any to
> 10.51.109.40/29 no state queue to_Gonta pass in quick on vlan622 inet
> from 10.51.109.0/28 to any no state pass in quick on vlan622 inet from
> 10.51.109.56/29 to any no state pass in quick on vlan781 inet from
> 10.53.31.0/25 to any no state pass in quick on vlan781 inet from
> 10.53.31.128/25 to any no state pass in quick on vlan664 inet from
> 10.52.14.0/24 to any no state pass in quick on vlan783 inet from
> 10.53.33.0/24 to any no state pass out quick on vlan622 inet from any
> to 10.51.109.0/28 no state pass out quick on vlan622 inet from any to
> 10.51.109.56/29 no state pass out quick on vlan781 inet from any to
> 10.53.31.0/25 no state pass out quick on vlan781 inet from any to
> 10.53.31.128/25 no state pass out quick on vlan664 inet from any to
> 10.52.14.0/24 no state pass out quick on vlan783 inet from any to
> 10.53.33.0/24 no state
>
> Does anyone know how to disable this? Thanks in advance!

1. Why do you want to disable this? Did you even noticed actual problems?

2. See pf.conf(5), particularly part about "ruleset-optimization" option.
But better you read this manual page thoroughly, then search mail
archives and then you'll find that likely you should not disable it.

--
  Best wishes,
    Vadim Zhukov

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

On Tue, Nov 03, 2009 at 05:00:39PM +0300, Vadim Zhukov wrote:
> > Does anyone know how to disable this? Thanks in advance!
>
> 1. Why do you want to disable this? Did you even noticed actual problems?
>
> 2. See pf.conf(5), particularly part about "ruleset-optimization" option.
> But better you read this manual page thoroughly, then search mail
> archives and then you'll find that likely you should not disable it.

Thank you too!
I've really missed this in man page.
I wish to disable this for the test period just for visual comfort.
I will enable this later, when I'm sure all works as needed.

--
MINO-RIPE

[Stuart Henderson]

On 2009-11-03, Alexander Shikoff <minotaur@...> wrote:
> Because I need queuing for outgoing traffic on vlan* interfaces.
> When keep state is used then queues for outgoing traffic do not work.

this (and variations) keep coming up (and has been answered 3 or 4
times in various mailing lists), but it's incorrect.

you can do this with stateful rules, you just need to do the queue
assignments in the right place/s to make sure that traffic gets
queued correctly.

- when you keep state, the state entry has a queue name associated
with it. any packets matching that state will be assigned to
the queue/s with that name on _any_ interface in the system.

- you can assign traffic on an interface other than the one holding
the queue. assuming you want to queue the downstream traffic you have
received on bge0 and want to forward to 1.2.3.4 on vlan1:

+--- --- -- -
| altq on vlan1 [...] queue { onetwothreefour [...] }
| queue onetwothreefour...
|
| ##  incoming connections to 1.2.3.4 get queued:
| pass in on bge0 to 1.2.3.4 queue { onetwothreefour }
|
| ##  outbound connections from 1.2.3.4 create state with a queue,
| ##  so _return traffic matching that state_ is queued:
| pass in on vlan1 from 1.2.3.4 queue { onetwothreefour }
+--- --- -- -

- if you're queuing both upstream and downstream, note that
queues on different interfaces can have the same name:

+--- --- -- -
| altq on vlan1 [...] queue { onetwothreefour [...] }
| altq on bge0 [...] queue { onetwothreefour [...] }
| queue onetwothreefour...
| pass in on vlan1 from 1.2.3.4 queue { onetwothreefour }
| pass in on bge0 to 1.2.3.4 queue { onetwothreefour }
+--- --- -- -

On Tue, Nov 03, 2009 at 10:44:08PM +0000, Stuart Henderson wrote:

> On 2009-11-03, Alexander Shikoff <minotaur@...> wrote:
> > Because I need queuing for outgoing traffic on vlan* interfaces.
> > When keep state is used then queues for outgoing traffic do not work.
>
> this (and variations) keep coming up (and has been answered 3 or 4
> times in various mailing lists), but it's incorrect.
>
> you can do this with stateful rules, you just need to do the queue
> assignments in the right place/s to make sure that traffic gets
> queued correctly.
>
> - when you keep state, the state entry has a queue name associated
> with it. any packets matching that state will be assigned to
> the queue/s with that name on _any_ interface in the system.
>
> - you can assign traffic on an interface other than the one holding
> the queue. assuming you want to queue the downstream traffic you have
> received on bge0 and want to forward to 1.2.3.4 on vlan1:
>
> +--- --- -- -
> | altq on vlan1 [...] queue { onetwothreefour [...] }
> | queue onetwothreefour...
> |
> | ##  incoming connections to 1.2.3.4 get queued:
> | pass in on bge0 to 1.2.3.4 queue { onetwothreefour }
> |
> | ##  outbound connections from 1.2.3.4 create state with a queue,
> | ##  so _return traffic matching that state_ is queued:
> | pass in on vlan1 from 1.2.3.4 queue { onetwothreefour }
> +--- --- -- -
>
> - if you're queuing both upstream and downstream, note that
> queues on different interfaces can have the same name:
>
> +--- --- -- -
> | altq on vlan1 [...] queue { onetwothreefour [...] }
> | altq on bge0 [...] queue { onetwothreefour [...] }
> | queue onetwothreefour...
> | pass in on vlan1 from 1.2.3.4 queue { onetwothreefour }
> | pass in on bge0 to 1.2.3.4 queue { onetwothreefour }
> +--- --- -- -

Thank you Stuart! I came to OpenBSD from FreeBSD which has not support
for altq on vlan-interfaces and it was great news for me to know that
OpenBSD does.

I'm wondering how to apply your solution to configuration
with enabled NAT? Let's assume that vlan1 is the LAN interface,
and on bge0 there is NAT enabled. User in LAN initiates connection.
Rule
- pass in on vlan1 from 10.0.0.10 queue { onetwothreefour }
creates a state and assigns outgoing packets to the queue 'onetwothreefour'.
Traffic flowing back to this user will be also assigned to the same queue.
It looks excellent while you do not need different queues for incoming
and outgoing traffic (remembering that we have NAT and all connections
are initiated from LAN).

- pass in  on vlan1 from 10.0.0.10 queue { from_onetwothreefour }
- pass out on vlan1 to   10.0.0.10 queue { to_onetwothreefour }
the second rule will not work, because first one created state.
Am I right?
Thanks in advance!

--
MINO-RIPE

[Cor-7]

Henning Brauer wrote:
>  
> how about reading the manpages for a change. sometimes i wonder why we
> write them.
>  

Please, PLEASE don't stop :)

Seriously, this is one area where OpenBSD (and probably other BSDs, too)
romp on Linux.  I was debugging my pf syntax last weekend and took the
time to learn how the "BNF" info at the end works (first time I've
really noticed that, but I'm not a CS major).  It was immensely helpful,
and way better than trial and error,  in getting my syntax correct.

C2

On Wed, Nov 04, 2009 at 01:43:08AM +0200, Alexander Shikoff wrote:

> On Tue, Nov 03, 2009 at 10:44:08PM +0000, Stuart Henderson wrote:
> > On 2009-11-03, Alexander Shikoff <minotaur@...> wrote:
> > > Because I need queuing for outgoing traffic on vlan* interfaces.
> > > When keep state is used then queues for outgoing traffic do not work.
> >
> > this (and variations) keep coming up (and has been answered 3 or 4
> > times in various mailing lists), but it's incorrect.
> >
> > you can do this with stateful rules, you just need to do the queue
> > assignments in the right place/s to make sure that traffic gets
> > queued correctly.
> >
> > - when you keep state, the state entry has a queue name associated
> > with it. any packets matching that state will be assigned to
> > the queue/s with that name on _any_ interface in the system.
> >
> > - you can assign traffic on an interface other than the one holding
> > the queue. assuming you want to queue the downstream traffic you have
> > received on bge0 and want to forward to 1.2.3.4 on vlan1:
> >
> > +--- --- -- -
> > | altq on vlan1 [...] queue { onetwothreefour [...] }
> > | queue onetwothreefour...
> > |
> > | ##  incoming connections to 1.2.3.4 get queued:
> > | pass in on bge0 to 1.2.3.4 queue { onetwothreefour }
> > |
> > | ##  outbound connections from 1.2.3.4 create state with a queue,
> > | ##  so _return traffic matching that state_ is queued:
> > | pass in on vlan1 from 1.2.3.4 queue { onetwothreefour }
> > +--- --- -- -
> >
> > - if you're queuing both upstream and downstream, note that
> > queues on different interfaces can have the same name:
> >
> > +--- --- -- -
> > | altq on vlan1 [...] queue { onetwothreefour [...] }
> > | altq on bge0 [...] queue { onetwothreefour [...] }
> > | queue onetwothreefour...
> > | pass in on vlan1 from 1.2.3.4 queue { onetwothreefour }
> > | pass in on bge0 to 1.2.3.4 queue { onetwothreefour }
> > +--- --- -- -
>
> Thank you Stuart! I came to OpenBSD from FreeBSD which has not support
> for altq on vlan-interfaces and it was great news for me to know that
> OpenBSD does.
>
> I'm wondering how to apply your solution to configuration
> with enabled NAT? Let's assume that vlan1 is the LAN interface,
> and on bge0 there is NAT enabled. User in LAN initiates connection.
> Rule
> - pass in on vlan1 from 10.0.0.10 queue { onetwothreefour }
> creates a state and assigns outgoing packets to the queue 'onetwothreefour'.
> Traffic flowing back to this user will be also assigned to the same queue.
> It looks excellent while you do not need different queues for incoming
> and outgoing traffic (remembering that we have NAT and all connections
> are initiated from LAN).
>
> - pass in  on vlan1 from 10.0.0.10 queue { from_onetwothreefour }
> - pass out on vlan1 to   10.0.0.10 queue { to_onetwothreefour }
> the second rule will not work, because first one created state.
> Am I right?
> Thanks in advance!

Stuart, at last I've understood pf's behaviour :)
I've solved my task.
Thank you and all community for the help very much!

Truly yours,
-- Alexander

--
MINO-RIPE

[mark hellewell]

2009/11/4 Henning Brauer <lists-openbsd@...>:
> how about reading the manpages for a change. sometimes i wonder why we
> write them.

don't worry;  some people really do read them and find them extremely
edifying,  myself included

mark

[Dave Anderson-4]

On Thu, 5 Nov 2009, mark hellewell wrote:

>2009/11/4 Henning Brauer <lists-openbsd@...>:
>> how about reading the manpages for a change. sometimes i wonder why we
>> write them.
>
>don't worry;  some people really do read them and find them extremely
>edifying,  myself included

I think one can safely s/some/many/.  We just don't post here very often
since, after carefully reading the manpages (and FAQ, etc.), we find
that our questions are usually all answered.  Many thanks to all
involved for the excellent documentation.

        Dave

--
Dave Anderson
<dave@...>