Nabble - freebsd-pf

Re: Lots of weird PF behavior on 7.2-STABLE

2009-12-15T12:33:24Z

2009/12/15 Linda Messerschmidt <linda.messerschmidt@...>:

> On Tue, Dec 15, 2009 at 11:08 AM, Peter Maxwell <peter@...> wrote:
>> I'm pretty sure you can run tcpdump against a packet capture from the
>> pflog interface on the pf box; that will include fields like
>> block/pass and rule number for each packet filtered.
>
> I have done that with "log" on all block rules. The packets shown as
> missing are *not* present in the dump when I do, so as far as I can
> tell they are not being dropped by a filter rule. Which makes sense,
> since none of the few block rules we have would touch packets in the
> middle of an established connection that was permitted.

Although it's not likely to be causing the problem (the default
"flags" on tcp rules are S/SA, which should exclude this possibility),
I'd check that the implicit pass rule isn't getting hit by the web
traffic. Add in an explicit "pass all" rule at the start and set the
log keyword on it. Make sure *none* of the web traffic is hitting
this rule.

>
> For comparison, endless streams of packets from those DNS guys we
> block *are* present in the tcpdump output, exactly as you describe, so
> I know pflog is working.
>
> So it really seems like something wrong in the internals of pf. I
> just don't know how to pursue it further.

If the box isn't too loaded, you may try using "log (all)" on the pass
rules (so that ALL packets are logged, not just the initial SYN) -
that way at least you'd find out which rules those data packets are
hitting, which would likely pin down the problem quite a bit.. those
missing packets must have went somewhere ;-) If it were me, that
would be my preferable option if it was available.

Barring that, I'd suggest simplifying your setup as much as possible -
is there too much traffic to remove the "route-to" and try it against
a single webserver? If it's not possible, you could try setting up a
simple TCP service on internal hosts and get something that works,
then build up (ECHO or TIME are not bad for this).

I'd also suggest removing the "scrub" directive until you have it
working properly. Is the "state-policy" floating or if-bound?

Does the problem affect other services in a similar manner, can you
replicate the exact same problem with the web servers with sshd, for
example?

What's annoying me is that I'm fairly sure I've seen this problem
before, but for the life of me I can't remember what caused it :-(

>
>> However one thing that I would strongly suggest is using proper packet
>> filter design: either decide upon a default deny then pass what you
>> want, or decide on a default pass and only deny what is bad.
>
> We are doing the latter; default pass and denying only what is bad.
> This isn't even really a firewall, it's for load balancing web
> connections. We just threw a couple of block rules in there because
> it was a good place to stop a particular attack. There are other
> "default deny" firewalls on other machines that handle all the traffic
> that isn't getting diverted by the load balance rule.
>
>> If you're having to use the "quick" keyword, you've probably*
>> done something wrong.
>
> Since we are using load balancing, the "pass" rule that wouldn't pass
> all the traffic we've just gone to the trouble to block would be
> outrageously complex. Hence, quick.
>

pf uses the last rule in the ruleset that matches for a given packet,
so for a first instance setup I'd suggest putting an explicit "pass
all" as the first rule, then any pass rules that do load-balancing and
the like, then a list of block rules. It makes it a lot easier to
read and debug. Then once it's working as you want, you can move the
block rules up to the top and add in the "quick" keyword for
performance purposes.

> If a case can be made that the use of "quick" is causing our packets
> to disappear, we'd probably be willing to tackle trying to restructure
> the rules.
>
> Thanks!
>
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Lots of weird PF behavior on 7.2-STABLE

2009-12-15T11:14:08Z

On Tue, Dec 15, 2009 at 11:08 AM, Peter Maxwell <peter@...> wrote:
> I'm pretty sure you can run tcpdump against a packet capture from the
> pflog interface on the pf box; that will include fields like
> block/pass and rule number for each packet filtered.

I have done that with "log" on all block rules. The packets shown as
missing are *not* present in the dump when I do, so as far as I can
tell they are not being dropped by a filter rule. Which makes sense,
since none of the few block rules we have would touch packets in the
middle of an established connection that was permitted.

For comparison, endless streams of packets from those DNS guys we
block *are* present in the tcpdump output, exactly as you describe, so
I know pflog is working.

So it really seems like something wrong in the internals of pf. I
just don't know how to pursue it further.

> However one thing that I would strongly suggest is using proper packet
> filter design: either decide upon a default deny then pass what you
> want, or decide on a default pass and only deny what is bad.

We are doing the latter; default pass and denying only what is bad.
This isn't even really a firewall, it's for load balancing web
connections. We just threw a couple of block rules in there because
it was a good place to stop a particular attack. There are other
"default deny" firewalls on other machines that handle all the traffic
that isn't getting diverted by the load balance rule.

> If you're having to use the "quick" keyword, you've probably*
> done something wrong.

Since we are using load balancing, the "pass" rule that wouldn't pass
all the traffic we've just gone to the trouble to block would be
outrageously complex. Hence, quick.

If a case can be made that the use of "quick" is causing our packets
to disappear, we'd probably be willing to tackle trying to restructure
the rules.

Thanks!
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Lots of weird PF behavior on 7.2-STABLE

2009-12-15T08:08:49Z

Hi Linda,

I'm pretty sure you can run tcpdump against a packet capture from the
pflog interface on the pf box; that will include fields like
block/pass and rule number for each packet filtered. That way you at
least know what rule is dropping/passing your packets. And if my
memory serves me right, pf uses a default pass rule - if it were me
I'd check that the SYN-ACK from the webserver isn't creating a second
state table entry using the default pass rule, or something equally as
annoying.

It is also possible I'm completely wrong, as it's been a while since
I've actually messed about with pf in any meaningful way.

However one thing that I would strongly suggest is using proper packet
filter design: either decide upon a default deny then pass what you
want, or decide on a default pass and only deny what is bad. This
methodology applies to any firewall whether pf, checkpoint, cisco,
etc. If you're having to use the "quick" keyword, you've probably*
done something wrong. The default deny approach is usually
preferable, so you should have one block all rule at the top of the
ruleset then all other rules should be pass rules.

Best wishes,

Peter

* there are some situations where you may have to use quick, but not
particularly often.

2009/12/15 Linda Messerschmidt <linda.messerschmidt@...>:

> Hi all,
>
> I have a PF machine that is giving fits. I see a lot of weird behavior.
>
> 1) TCP connections (mainly port 80) sometimes take 3 seconds to get
> started instead of being virtually instant.
> 2) Sometimes HTTP connections just stop responding. (Client program
> times out waiting for response.)
> 3) Sometimes connections get weirdly dropped ("Connection reset by peer.")
> 4) Sometimes if I am ssh'd through the firewall, something will happen
> and my inbound packets will start getting dropped, but outbound
> packets still pass. For example, if I'm at the shell prompt, it is
> non-responsive. But if I log alongside a stuck connection and "write"
> to that tty, I will see it no problem.
> 5) States that have no right to still be there continue to pile up
> into the hundreds of thousands.
>
> I kind of get the feeling that all of these are related. In
> particular, I think 2, 3, and 4.
>
> Of all of these, the only one I can document at the moment is #3.
>
> Here is a packet capture from the public (web client) interface:
>
> 20:00:02.038067 IP 1.2.3.4.61645 > 5.6.7.8.80: S
> 620577087:620577087(0) win 65535 <mss 1460,nop,wscale
> 9,sackOK,timestamp 953726452 0>
> 20:00:02.038328 IP 5.6.7.8.80 > 1.2.3.4.61645: S 40565958:40565958(0)
> ack 620577088 win 0 <mss 1460>
> 20:00:02.065678 IP 1.2.3.4.61645 > 5.6.7.8.80: . ack 1 win 65535
> 20:00:02.095158 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:02.378248 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:02.746163 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:03.282122 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:04.154112 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:05.698002 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:07.913721 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:12.145438 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:12.287038 IP 5.6.7.8.80 > 1.2.3.4.61645: F 1:1(0) ack 1 win 65535
> 20:00:20.408734 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:20.409874 IP 5.6.7.8.80 > 1.2.3.4.61645: R 40565959:40565959(0) win 0
>
> Here is a packet capture of the same session from the private (web
> server) interface:
>
> 20:00:02.038089 IP 1.2.3.4.61645 > 5.6.7.8.80: S
> 620577087:620577087(0) win 65535 <mss 1460,nop,wscale
> 9,sackOK,timestamp 953726452 0>
> 20:00:02.038311 IP 5.6.7.8.80 > 1.2.3.4.61645: S 40565958:40565958(0)
> ack 620577088 win 0 <mss 1460>
> 20:00:02.065694 IP 1.2.3.4.61645 > 5.6.7.8.80: . ack 1 win 65535
> 20:00:12.287026 IP 5.6.7.8.80 > 1.2.3.4.61645: F 1:1(0) ack 1 win 65535
> 20:00:20.408747 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:20.409859 IP 5.6.7.8.80 > 1.2.3.4.61645: R 40565959:40565959(0) win 0
>
> So that client -> server push packet is not making it through the
> firewall despite numerous retransmits, until 18 seconds later when the
> server has already given up on it.
>
> That connection hangs around in the state table for a long time as:
>
> all tcp 5.6.7.8:80 <- 1.2.3.4:61645 CLOSED:CLOSING
>
> This despite:
>
> set timeout tcp.closed 5
> set timeout tcp.closing 30
>
> To test, I stopped connections from 1.2.3.4 to 5.6.7.8. At present,
> there are *zero* established connections between 1.2.3.4 and 5.6.7.8.
> None. But:
>
> $ sudo pfctl -s state | fgrep 1.2.3.4: | fgrep :80 | wc
> 2243 13458 160932
>
> A few minutes later I broke this down by connection status:
> 1222 CLOSED:CLOSING
> 556 ESTABLISHED:ESTABLISHED
> 15 FIN_WAIT_2:CLOSING
> 27 SYN_SENT:FIN_WAIT_2
>
> That doesn't add up to 2243, so they *are* slowly dying off. I did
> some poking around, and the CLOSED:CLOSING ones expire after fifteen
> minutes, which is the timeout for tcp.opening. Um, OK.
>
> The 556 ESTABLISHED:ESTABLISHED states appear content to persist until
> they age off too, even though those connections are long gone.
>
> As far as the "3 second" thing, I noticed somebody here recently had a
> similar problem and made it go away by upping their states and
> dropping their timeouts. Well, he dropped his timeouts to where ours
> are, and we're at:
>
> set limit states 2000000
>
> We are definitely not out of states; we're seeing these problems right
> now and due to my playing around with the tcp.established timeout,
> we're at 66412 states right now. (Ordinarily it hovers around
> 350,000.) The machine is a dual-core Core 2 6320 with 2GB of RAM and
> nothing to but load balance this traffic. It shows as 95% idle all
> day.
>
> So sometimes pf loses packets related to connections that are still
> around, and sometimes it thinks connections are still around long
> after the packets are gone.
>
> I would be really, really grateful for any suggestions or help. I am
> completely lost here and at my wits' end!
>
> I've included my pf.conf below.
>
>
> --------------------------------------------------------------------------------------------
>
> set limit states 2000000
> set timeout tcp.established 86400
> set timeout tcp.closed 5
> set timeout tcp.closing 30
>
> ExtIf = "em0"
> IntIf = "em1"
>
> table <NoRouteIPs> { 127.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24,
> 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
> table <OurIPs> { ... }
> table <DNSServers> { ... }
> table <BalanceBlocks> { ... }
>
> scrub
>
> ## Block Reserved Addresses
> block log quick on $ExtIf from <NoRouteIPs> to any
> block log quick on $ExtIf from any to <NoRouteIPs>
>
> ## Block our own Addresses
> block in log quick on $ExtIf inet from <OurIPs> to any
>
> ## Anti-DDOS
> table <AntiDDOS> persist
> block quick from <AntiDDOS> to any
> block quick from any to <AntiDDOS>
>
> ## Block HTTP traffic to DNS servers
> block quick inet proto tcp from any to <DNSServers> port 80
>
> ## Weird DNS people added 2009-06-18
> block drop log quick proto 255
> table <GTExperimentDNS> { 61.220.4.0/24 }
> block drop in quick proto { udp, tcp } from <GTExperimentDNS> to any port 53
>
> ## Load Balancing
> pass in on $ExtIf route-to { ($IntIf 3.4.5.6), ($IntIf 3.4.5.7),
> ($IntIf 3.4.5.8), ($IntIf 3.4.5.9) } round-robin proto tcp from any to
> <BalanceBlocks> port 80
> _______________________________________________
> freebsd-pf@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."
>

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Lots of weird PF behavior on 7.2-STABLE

2009-12-15T07:01:47Z

On Tue, Dec 15, 2009 at 4:55 AM, Ermal Luçi <eri@...> wrote:
> Try enabling sticky connections here.

As a practical matter we don't care if two connections from the same
client go to the same server or not. Is there some reason to suspect
that this option would alter the behavior of single connections, like
the problem we're having?

Although even in that case, all the servers on the same interface so
if it were a problem with load balancing, I would expect to see the
stray packets addressed to the wrong MAC address, not to have them
disappear completely.

Thanks!
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Lots of weird PF behavior on 7.2-STABLE

2009-12-15T01:55:58Z

On Tue, Dec 15, 2009 at 7:21 AM, Linda Messerschmidt <
linda.messerschmidt@...> wrote:

> Hi all,
>
> I have a PF machine that is giving fits. I see a lot of weird behavior.
>
> 1) TCP connections (mainly port 80) sometimes take 3 seconds to get
> started instead of being virtually instant.
> 2) Sometimes HTTP connections just stop responding. (Client program
> times out waiting for response.)
> 3) Sometimes connections get weirdly dropped ("Connection reset by peer.")
> 4) Sometimes if I am ssh'd through the firewall, something will happen
> and my inbound packets will start getting dropped, but outbound
> packets still pass. For example, if I'm at the shell prompt, it is
> non-responsive. But if I log alongside a stuck connection and "write"
> to that tty, I will see it no problem.
> 5) States that have no right to still be there continue to pile up
> into the hundreds of thousands.
>
> I kind of get the feeling that all of these are related. In
> particular, I think 2, 3, and 4.
>
> Of all of these, the only one I can document at the moment is #3.
>
> Here is a packet capture from the public (web client) interface:
>
> 20:00:02.038067 IP 1.2.3.4.61645 > 5.6.7.8.80: S
> 620577087:620577087(0) win 65535 <mss 1460,nop,wscale
> 9,sackOK,timestamp 953726452 0>
> 20:00:02.038328 IP 5.6.7.8.80 > 1.2.3.4.61645: S 40565958:40565958(0)
> ack 620577088 win 0 <mss 1460>
> 20:00:02.065678 IP 1.2.3.4.61645 > 5.6.7.8.80: . ack 1 win 65535
> 20:00:02.095158 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:02.378248 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:02.746163 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:03.282122 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:04.154112 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:05.698002 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:07.913721 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:12.145438 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:12.287038 IP 5.6.7.8.80 > 1.2.3.4.61645: F 1:1(0) ack 1 win 65535
> 20:00:20.408734 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:20.409874 IP 5.6.7.8.80 > 1.2.3.4.61645: R 40565959:40565959(0) win 0
>
> Here is a packet capture of the same session from the private (web
> server) interface:
>
> 20:00:02.038089 IP 1.2.3.4.61645 > 5.6.7.8.80: S
> 620577087:620577087(0) win 65535 <mss 1460,nop,wscale
> 9,sackOK,timestamp 953726452 0>
> 20:00:02.038311 IP 5.6.7.8.80 > 1.2.3.4.61645: S 40565958:40565958(0)
> ack 620577088 win 0 <mss 1460>
> 20:00:02.065694 IP 1.2.3.4.61645 > 5.6.7.8.80: . ack 1 win 65535
> 20:00:12.287026 IP 5.6.7.8.80 > 1.2.3.4.61645: F 1:1(0) ack 1 win 65535
> 20:00:20.408747 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
> 20:00:20.409859 IP 5.6.7.8.80 > 1.2.3.4.61645: R 40565959:40565959(0) win 0
>
> So that client -> server push packet is not making it through the
> firewall despite numerous retransmits, until 18 seconds later when the
> server has already given up on it.
>
> That connection hangs around in the state table for a long time as:
>
> all tcp 5.6.7.8:80 <- 1.2.3.4:61645 CLOSED:CLOSING
>
> This despite:
>
> set timeout tcp.closed 5
> set timeout tcp.closing 30
>
> To test, I stopped connections from 1.2.3.4 to 5.6.7.8. At present,
> there are *zero* established connections between 1.2.3.4 and 5.6.7.8.
> None. But:
>
> $ sudo pfctl -s state | fgrep 1.2.3.4: | fgrep :80 | wc
> 2243 13458 160932
>
> A few minutes later I broke this down by connection status:
> 1222 CLOSED:CLOSING
> 556 ESTABLISHED:ESTABLISHED
> 15 FIN_WAIT_2:CLOSING
> 27 SYN_SENT:FIN_WAIT_2
>
> That doesn't add up to 2243, so they *are* slowly dying off. I did
> some poking around, and the CLOSED:CLOSING ones expire after fifteen
> minutes, which is the timeout for tcp.opening. Um, OK.
>
> The 556 ESTABLISHED:ESTABLISHED states appear content to persist until
> they age off too, even though those connections are long gone.
>
> As far as the "3 second" thing, I noticed somebody here recently had a
> similar problem and made it go away by upping their states and
> dropping their timeouts. Well, he dropped his timeouts to where ours
> are, and we're at:
>
> set limit states 2000000
>
> We are definitely not out of states; we're seeing these problems right
> now and due to my playing around with the tcp.established timeout,
> we're at 66412 states right now. (Ordinarily it hovers around
> 350,000.) The machine is a dual-core Core 2 6320 with 2GB of RAM and
> nothing to but load balance this traffic. It shows as 95% idle all
> day.
>
> So sometimes pf loses packets related to connections that are still
> around, and sometimes it thinks connections are still around long
> after the packets are gone.
>
> I would be really, really grateful for any suggestions or help. I am
> completely lost here and at my wits' end!
>
> I've included my pf.conf below.
>
>
>
> --------------------------------------------------------------------------------------------
>
> set limit states 2000000
> set timeout tcp.established 86400
> set timeout tcp.closed 5
> set timeout tcp.closing 30
>
> ExtIf = "em0"
> IntIf = "em1"
>
> table <NoRouteIPs> { 127.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24,
> 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
> table <OurIPs> { ... }
> table <DNSServers> { ... }
> table <BalanceBlocks> { ... }
>
> scrub
>
> ## Block Reserved Addresses
> block log quick on $ExtIf from <NoRouteIPs> to any
> block log quick on $ExtIf from any to <NoRouteIPs>
>
> ## Block our own Addresses
> block in log quick on $ExtIf inet from <OurIPs> to any
>
> ## Anti-DDOS
> table <AntiDDOS> persist
> block quick from <AntiDDOS> to any
> block quick from any to <AntiDDOS>
>
> ## Block HTTP traffic to DNS servers
> block quick inet proto tcp from any to <DNSServers> port 80
>
> ## Weird DNS people added 2009-06-18
> block drop log quick proto 255
> table <GTExperimentDNS> { 61.220.4.0/24 }
> block drop in quick proto { udp, tcp } from <GTExperimentDNS> to any port
> 53
>
> ## Load Balancing
> pass in on $ExtIf route-to { ($IntIf 3.4.5.6), ($IntIf 3.4.5.7),
> ($IntIf 3.4.5.8), ($IntIf 3.4.5.9) } round-robin proto tcp from any to
> <BalanceBlocks> port 80
>
> Try enabling sticky connections here.

--
Ermal
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Lots of weird PF behavior on 7.2-STABLE

2009-12-14T22:21:53Z

Hi all,

I have a PF machine that is giving fits. I see a lot of weird behavior.

1) TCP connections (mainly port 80) sometimes take 3 seconds to get
started instead of being virtually instant.
2) Sometimes HTTP connections just stop responding. (Client program
times out waiting for response.)
3) Sometimes connections get weirdly dropped ("Connection reset by peer.")
4) Sometimes if I am ssh'd through the firewall, something will happen
and my inbound packets will start getting dropped, but outbound
packets still pass. For example, if I'm at the shell prompt, it is
non-responsive. But if I log alongside a stuck connection and "write"
to that tty, I will see it no problem.
5) States that have no right to still be there continue to pile up
into the hundreds of thousands.

I kind of get the feeling that all of these are related. In
particular, I think 2, 3, and 4.

Of all of these, the only one I can document at the moment is #3.

Here is a packet capture from the public (web client) interface:

20:00:02.038067 IP 1.2.3.4.61645 > 5.6.7.8.80: S
620577087:620577087(0) win 65535 <mss 1460,nop,wscale
9,sackOK,timestamp 953726452 0>
20:00:02.038328 IP 5.6.7.8.80 > 1.2.3.4.61645: S 40565958:40565958(0)
ack 620577088 win 0 <mss 1460>
20:00:02.065678 IP 1.2.3.4.61645 > 5.6.7.8.80: . ack 1 win 65535
20:00:02.095158 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:02.378248 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:02.746163 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:03.282122 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:04.154112 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:05.698002 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:07.913721 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:12.145438 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:12.287038 IP 5.6.7.8.80 > 1.2.3.4.61645: F 1:1(0) ack 1 win 65535
20:00:20.408734 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:20.409874 IP 5.6.7.8.80 > 1.2.3.4.61645: R 40565959:40565959(0) win 0

Here is a packet capture of the same session from the private (web
server) interface:

20:00:02.038089 IP 1.2.3.4.61645 > 5.6.7.8.80: S
620577087:620577087(0) win 65535 <mss 1460,nop,wscale
9,sackOK,timestamp 953726452 0>
20:00:02.038311 IP 5.6.7.8.80 > 1.2.3.4.61645: S 40565958:40565958(0)
ack 620577088 win 0 <mss 1460>
20:00:02.065694 IP 1.2.3.4.61645 > 5.6.7.8.80: . ack 1 win 65535
20:00:12.287026 IP 5.6.7.8.80 > 1.2.3.4.61645: F 1:1(0) ack 1 win 65535
20:00:20.408747 IP 1.2.3.4.61645 > 5.6.7.8.80: P 1:80(79) ack 1 win 65535
20:00:20.409859 IP 5.6.7.8.80 > 1.2.3.4.61645: R 40565959:40565959(0) win 0

So that client -> server push packet is not making it through the
firewall despite numerous retransmits, until 18 seconds later when the
server has already given up on it.

That connection hangs around in the state table for a long time as:

all tcp 5.6.7.8:80 <- 1.2.3.4:61645 CLOSED:CLOSING

This despite:

set timeout tcp.closed 5
set timeout tcp.closing 30

To test, I stopped connections from 1.2.3.4 to 5.6.7.8. At present,
there are *zero* established connections between 1.2.3.4 and 5.6.7.8.
None. But:

$ sudo pfctl -s state | fgrep 1.2.3.4: | fgrep :80 | wc
2243 13458 160932

A few minutes later I broke this down by connection status:
1222 CLOSED:CLOSING
556 ESTABLISHED:ESTABLISHED
15 FIN_WAIT_2:CLOSING
27 SYN_SENT:FIN_WAIT_2

That doesn't add up to 2243, so they *are* slowly dying off. I did
some poking around, and the CLOSED:CLOSING ones expire after fifteen
minutes, which is the timeout for tcp.opening. Um, OK.

The 556 ESTABLISHED:ESTABLISHED states appear content to persist until
they age off too, even though those connections are long gone.

As far as the "3 second" thing, I noticed somebody here recently had a
similar problem and made it go away by upping their states and
dropping their timeouts. Well, he dropped his timeouts to where ours
are, and we're at:

set limit states 2000000

We are definitely not out of states; we're seeing these problems right
now and due to my playing around with the tcp.established timeout,
we're at 66412 states right now. (Ordinarily it hovers around
350,000.) The machine is a dual-core Core 2 6320 with 2GB of RAM and
nothing to but load balance this traffic. It shows as 95% idle all
day.

So sometimes pf loses packets related to connections that are still
around, and sometimes it thinks connections are still around long
after the packets are gone.

I would be really, really grateful for any suggestions or help. I am
completely lost here and at my wits' end!

I've included my pf.conf below.

--------------------------------------------------------------------------------------------

set limit states 2000000
set timeout tcp.established 86400
set timeout tcp.closed 5
set timeout tcp.closing 30

ExtIf = "em0"
IntIf = "em1"

table <NoRouteIPs> { 127.0.0.0/8, 169.254.0.0/16, 192.0.2.0/24,
192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
table <OurIPs> { ... }
table <DNSServers> { ... }
table <BalanceBlocks> { ... }

scrub

## Block Reserved Addresses
block log quick on $ExtIf from <NoRouteIPs> to any
block log quick on $ExtIf from any to <NoRouteIPs>

## Block our own Addresses
block in log quick on $ExtIf inet from <OurIPs> to any

## Anti-DDOS
table <AntiDDOS> persist
block quick from <AntiDDOS> to any
block quick from any to <AntiDDOS>

## Block HTTP traffic to DNS servers
block quick inet proto tcp from any to <DNSServers> port 80

## Weird DNS people added 2009-06-18
block drop log quick proto 255
table <GTExperimentDNS> { 61.220.4.0/24 }
block drop in quick proto { udp, tcp } from <GTExperimentDNS> to any port 53

## Load Balancing
pass in on $ExtIf route-to { ($IntIf 3.4.5.6), ($IntIf 3.4.5.7),
($IntIf 3.4.5.8), ($IntIf 3.4.5.9) } round-robin proto tcp from any to
<BalanceBlocks> port 80
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

RE: PF Transparent Bridge Firewall + CARP

2009-12-14T08:39:43Z

> -----Original Message-----
> From: Kevin [mailto:k@...]
> I have what I would consider not a standard firewall scenario that
> requires a second, redundant PF firewall. My first / main firewall is
> pf + transparent bridging with no internal network / ip addresses.

I realize that carp would require an ip address on both interfaces to work
properly... this is correct, right? Could I just assign the 1 ip address /
gateway on the bridge0 interface and add a carp interface to fail that over
to the 2nd firewall?

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

PF Transparent Bridge Firewall + CARP

2009-12-14T08:19:30Z

Hello,

I have what I would consider not a standard firewall scenario that requires
a second, redundant PF firewall. My first / main firewall is pf +
transparent bridging with no internal network / ip addresses.

I would like to implement a second failover firewall w/ CARP and have a
pretty good idea of how I can accomplish this -- however , I would like to
hear opinions / suggestions of implementing the most logical solution with
CARP.

I would like to implement CARP on the gateway IP address which will sit on
the bridge0 interface, which bridges br01 + br02.

Bridge0 will have no ip address assigned , and the gateway ip address will
be assigned to carp0. Will I have to NAT traffic from carp0 > bridge0 ? will
bridge0 be my ext_if in pf.conf , and int_if will be carp0? The main issue
is maintaining redundancy, for me.

It seems like an easy question, however Im just trying to wrap my brain
around the one that doesn't cost as much overhead and is the simplest / most
logical.

Pertinent info :

FreeBSD fw 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #4: Tue Dec 16 13:00:03 EST
2008 admin@fw:/usr/obj/usr/src/sys/FW i386

If you need additional information ,please let me know.

Suggestions are welcome.

Thanks,

Kevin

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: IPv6, PF problem

2009-12-14T06:54:46Z

On Saturday 12 December 2009 22:11:28 Aaron Stellman wrote:

> Hello there,
>
> > What does "pfctl -vvsr" give you for the rule? It should include the
> > number of addresses assigned to the interface in the braces - e.g. "...
> > (bge0:4) ..."
>
> @8 pass in on bge0 proto tcp from any to (bge0:4) port = ftp flags S/SA
> keep state [ Evaluations: 0 Packets: 0 Bytes: 0
> States: 0 ] [ Inserted: uid 0 pid 79900 ]
>
> > In addition, can you try to add separate rules for inet and inet6 - i.e.
> >
> > pass in on $ext_if inet proto tcp to ($ext_if) port 21
> > pass in on $ext_if inet6 proto tcp to ($ext_if) port 21
>
> @8 pass in on bge0 inet proto tcp from any to (bge0:2) port = ftp flags
> S/SA keep state [ Evaluations: 1 Packets: 17 Bytes: 916
> States: 1 ] [ Inserted: uid 0 pid 80198 ]
> @9 pass in on bge0 inet6 proto tcp from any to (bge0:2) port = ftp flags
> S/SA keep state [ Evaluations: 1 Packets: 0 Bytes: 0
> States: 0 ] [ Inserted: uid 0 pid 80198 ]
>
> and it passes inet6 connection with these two rules. Do you consider it
> a bug? This essentially forces me to have 2 separate rules for inet and
> inet6.

I do consider it a bug, but I can't reproduce it here. Can you think of
anything in your setup that might be special - e.g. the way you add the
addresses to your interface? Are you certain that you were testing with the
right rules in place (your output above shows zero rule evaluations) which is
a sign that something else went wrong.

Can anyone else reproduce this problem or did you see something similar?

Regards,

--
Max
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Current problem reports assigned to freebsd-pf@FreeBSD.org

2009-12-14T03:07:00Z

Note: to view an individual PR, use:
http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.

S Tracker Resp. Description
--------------------------------------------------------------------------------
o kern/140697 pf [pf] pf behaviour changes - must be documented
o kern/137982 pf [pf] when pf can hit state limits, random IP failures
o kern/136781 pf [pf] Packets appear to drop with pf scrub and if_bridg
o kern/135948 pf [pf] [gre] pf not natting gre protocol
o kern/135162 pf [pfsync] pfsync(4) not usable with GENERIC kernel
o kern/134996 pf [pf] Anchor tables not included when pfctl(8) is run w
o kern/133732 pf [pf] max-src-conn issue
o kern/132769 pf [pf] [lor] 2 LOR's with pf task mtx / ifnet and rtent
f kern/132176 pf [pf] pf stalls connection when using route-to [regress
o conf/130381 pf [rc.d] [pf] [ip6] ipv6 not fully configured when pf st
o kern/129861 pf [pf] [patch] Argument names reversed in pf_table.c:_co
o kern/127920 pf [pf] ipv6 and synproxy don't play well together
o conf/127814 pf [pf] The flush in pf_reload in /etc/rc.d/pf does not w
o kern/127439 pf [pf] deadlock in pf
f kern/127345 pf [pf] Problem with PF on FreeBSD7.0 [regression]
o kern/127121 pf [pf] [patch] pf incorrect log priority
o kern/127042 pf [pf] [patch] pf recursion panic if interface group is
o kern/125467 pf [pf] pf keep state bug while handling sessions between
s kern/124933 pf [pf] [ip6] pf does not support (drops) IPv6 fragmented
o kern/124364 pf [pf] [panic] Kernel panic with pf + bridge
o kern/122773 pf [pf] pf doesn't log uid or pid when configured to
o kern/122014 pf [pf] [panic] FreeBSD 6.2 panic in pf
o kern/121704 pf [pf] PF mangles loopback packets
o kern/120281 pf [pf] [request] lost returning packets to PF for a rdr
o kern/120057 pf [pf] [patch] Allow proper settings of ALTQ_HFSC. The c
o bin/118355 pf [pf] [patch] pfctl(8) help message options order false
o kern/114567 pf [pf] [lor] pf_ioctl.c + if.c
o kern/114095 pf [carp] carp+pf delay with high state limit
o kern/111220 pf [pf] repeatable hangs while manipulating pf tables
s conf/110838 pf [pf] tagged parameter on nat not working on FreeBSD 5.
o kern/103283 pf pfsync fails to sucessfully transfer some sessions
o kern/103281 pf pfsync reports bulk update failures
o kern/93825 pf [pf] pf reply-to doesn't work
o sparc/93530 pf [pf] Incorrect checksums when using pf's route-to on s
o kern/92949 pf [pf] PF + ALTQ problems with latency
o bin/86635 pf [patch] pfctl(8): allow new page character (^L) in pf.
o kern/82271 pf [pf] cbq scheduler cause bad latency

37 problems total.

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: IPv6, PF problem

2009-12-12T13:11:28Z

Hello there,

> What does "pfctl -vvsr" give you for the rule? It should include the number
> of addresses assigned to the interface in the braces - e.g. "... (bge0:4) ..."

@8 pass in on bge0 proto tcp from any to (bge0:4) port = ftp flags S/SA keep state
[ Evaluations: 0 Packets: 0 Bytes: 0 States: 0 ]
[ Inserted: uid 0 pid 79900 ]

> In addition, can you try to add separate rules for inet and inet6 - i.e.
>
> pass in on $ext_if inet proto tcp to ($ext_if) port 21
> pass in on $ext_if inet6 proto tcp to ($ext_if) port 21

@8 pass in on bge0 inet proto tcp from any to (bge0:2) port = ftp flags S/SA keep state
[ Evaluations: 1 Packets: 17 Bytes: 916 States: 1 ]
[ Inserted: uid 0 pid 80198 ]
@9 pass in on bge0 inet6 proto tcp from any to (bge0:2) port = ftp flags S/SA keep state
[ Evaluations: 1 Packets: 0 Bytes: 0 States: 0 ]
[ Inserted: uid 0 pid 80198 ]

and it passes inet6 connection with these two rules. Do you consider it
a bug? This essentially forces me to have 2 separate rules for inet and
inet6.
Thanks
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: IPv6, PF problem

2009-12-12T12:37:19Z

On Saturday 12 December 2009 02:25:08 Aaron Stellman wrote:

> Hello there,
> Here is the problem I've encountered on a dual stack amd64 FreeBSD 8.0p1
> machine.
>
> What works:
> pass in on $ext_if proto tcp to port 21
>
> What doesn't work:
> pass in on $ext_if proto tcp to ($ext_if) port 21
>
> here is what's logged when it doesn't work:
> listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size
> 1515 bytes
> 00:00:00.000000 rule 0/0(match): block in on bge0:
> 2001:1938:235:beef:21b:21ff:fe37:d799.11220 >
> 2001:1938:235:dead:226:b9ff:fe75:6e5e.21: Flags [S], seq 413041093, win
> 65535, options [mss 1440,nop,nop,sackOK,nop,wscale 1,nop,nop,TS val
> 3435338387 ecr 0], length 0

What does "pfctl -vvsr" give you for the rule? It should include the number
of addresses assigned to the interface in the braces - e.g. "... (bge0:4) ..."

In addition, can you try to add separate rules for inet and inet6 - i.e.

pass in on $ext_if inet proto tcp to ($ext_if) port 21
pass in on $ext_if inet6 proto tcp to ($ext_if) port 21

and check the number of addresses with pfctl -vvsr?

> ext_if="bge0"
>
> epsilon# ifconfig -a
> bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
> ether 00:26:b9:75:6e:5e
> inet 10.10.11.5 netmask 0xffffffe0 broadcast 10.10.11.31
> inet6 fe80::226:b9ff:fe75:6e5e%bge0 prefixlen 64 scopeid 0x1
> inet 10.10.11.8 netmask 0xffffffe0 broadcast 10.10.11.31
> inet6 2001:1938:235:dead:226:b9ff:fe75:6e5e prefixlen 64
> autoconf
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
> options=3<RXCSUM,TXCSUM>
> inet 127.0.0.1 netmask 0xff000000
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
> pflog0: flags=0<> metric 0 mtu 33152
>
>
> Notice, that it works as expected with IPv4; meaning that when I use "to
> ($ext_if)" and use ipv4 to connect, connection passes through, unlike
> IPv6.
> Also, OpenBSD pf works as expected with both IPv{4,6}
> _______________________________________________
> freebsd-pf@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."
>
>
> !DSPAM:4b22f113621191134040011!
>

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Transition from IPFW: PF flags for IPFW "setup" and "established" keywords

2009-12-12T05:58:17Z

Holger Rauch пишет:
> Hi to everybody,
>
> what are the correct combinations of flags for the IPFW "setup" and
> "established" keywords?

PF's equivalent of IPFW's "setup" is 'flags S/SA'.
Also, you have to include 'keep state' in the same rule
(for FreeBSD versions up to 6.4, in 7.x and 8.x - it's
a default behavior).

If connection is established, PF create state and match
thraffic "internally" whithout special dedicated rules.

E.g.,

pass in on fxp0 inet proto tcp from any to any port 80 flags S/SA keep state

will pass TCP traffic to port 80 if it starts as it should
beginning from the firts packet with only SYN-bit set
of two bits SYN and ACK. State will be created for this
flow if rest packets will follow usual three-way handshake.
After this all packets in this flow will pass automatically
untill connection will be closed (packets with FIN bits seen
by PF) or timed out.

Something like this. :)

--
Sincerely yours,
Artyom Viklenko.
-------------------------------------------------------
artem@... | http://www.aws-net.org.ua/~artem
FreeBSD: The Power to Serve - http://www.freebsd.org
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

IPv6, PF problem

2009-12-11T17:25:08Z

Hello there,
Here is the problem I've encountered on a dual stack amd64 FreeBSD 8.0p1
machine.

What works:
pass in on $ext_if proto tcp to port 21

What doesn't work:
pass in on $ext_if proto tcp to ($ext_if) port 21

here is what's logged when it doesn't work:
listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size
1515 bytes
00:00:00.000000 rule 0/0(match): block in on bge0:
2001:1938:235:beef:21b:21ff:fe37:d799.11220 >
2001:1938:235:dead:226:b9ff:fe75:6e5e.21: Flags [S], seq 413041093, win
65535, options [mss 1440,nop,nop,sackOK,nop,wscale 1,nop,nop,TS val
3435338387 ecr 0], length 0

ext_if="bge0"

epsilon# ifconfig -a
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500
options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
ether 00:26:b9:75:6e:5e
inet 10.10.11.5 netmask 0xffffffe0 broadcast 10.10.11.31
inet6 fe80::226:b9ff:fe75:6e5e%bge0 prefixlen 64 scopeid 0x1
inet 10.10.11.8 netmask 0xffffffe0 broadcast 10.10.11.31
inet6 2001:1938:235:dead:226:b9ff:fe75:6e5e prefixlen 64
autoconf
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
pflog0: flags=0<> metric 0 mtu 33152

Notice, that it works as expected with IPv4; meaning that when I use "to
($ext_if)" and use ipv4 to connect, connection passes through, unlike
IPv6.
Also, OpenBSD pf works as expected with both IPv{4,6}
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: FW: clientNatLookup: PF open failed: (13) Permission denied

2009-12-11T04:11:07Z

2009/12/11 John Dakos [ Enovation Technologies ] <gdakos@...>

>
> Hello all.
>
> I'm running Squid Version 3.0.STABLE20 on FreeBSD 8 Release with PF and
> ..
>
> --enable-pf-transparent'
>
> Squid is worked but in my cashe.log I have clientNatLookup: PF open
> failed: (13) Permission denied every time...
>
> I have in rc.conf squid_enable="YES"
>
> Any idea for that ?
>

Just allow the user with which you run squid permission of read(write?) to
/dev/pf.

--
Ermal
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

FW: clientNatLookup: PF open failed: (13) Permission denied

2009-12-11T04:04:07Z

Hello all.

I'm running Squid Version 3.0.STABLE20 on FreeBSD 8 Release with PF   and
..

--enable-pf-transparent'

Squid is worked but in my cashe.log   I have   clientNatLookup: PF open
failed: (13) Permission denied every time...

I have    in rc.conf     squid_enable="YES"

Any idea for that ?

Thanks

__________ Information from ESET NOD32 Antivirus, version of virus signature
database 4678 (20091211) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Transition from IPFW: PF flags for IPFW "setup" and "established" keywords

2009-12-11T03:59:01Z

Hi to everybody,

what are the correct combinations of flags for the IPFW "setup" and
"established" keywords?

I googled for this but found no real mapping to pf flags.

Any hints/links are welcome.

Thanks in advance & kind regards,

Holger

signature.asc (204 bytes) Download Attachment

Current problem reports assigned to freebsd-pf@FreeBSD.org

2009-12-07T03:07:00Z

RE: Limit connections doesn't work

2009-12-07T01:42:53Z

HI tom
I know, and this is what I said.
Yes you right and can replace the reload and put it into the a table with
the pf command.
But the server is on a 1mb/10mb ADSL line and trough put does not really
matter.
It was send as an example only not as a must do. This script have done 5
years ago as a quick hack
And because I'm not a expert with PF, I'm subscribed to this list so I can
learn without prejudice

Torsten

-----Original Message-----
From: Tom Uffner [mailto:tom@...]
Sent: 06 December 2009 23:01
To: Torsten Kersandt
Cc: freebsd-pf@...
Subject: Re: Limit connections doesn't work

Torsten Kersandt wrote:
> HI
> I personally have all ssh and alike ports closed on my servers.
> If I want to connect to the server per ssh or whatever function, I login
to a hidden php which adds my current IP to a sql table.
> I use sql because I'm not the only one using this and want to keep track
which admin is logging in.
> A cron job is running every minute looking in the table and adding the new
ip addresses to the pf include file and reloading PF
>
> Every night at 4am, I empty the text file and reload pf.
>
> I know that this could be done more elegant but KISS is what I like.

that script is horribly inefficient and disruptive to your firewall
throughput.

you could save a lot of unnecessary cpu cycles and speed up your
connections a bit by simply replacing the reloads with pfctl
commands that manipulate the table directly.

> #!/bin/sh
> ### MySQL Setup ###
> MUSER="username"
> MPASS="password"
> MHOST="localhost"
> MYSQL="/usr/local/bin/mysql"
> #
> ### Get all new IP addresses ###
> DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'select ipAddress from
intranet.ipCleared WHERE `timestamp` > (UNIX_TIMESTAMP()-60)')"
> for ip in $DBS
> do
> ## this bit is emailed to me over cron run-output if a new IP address was
found
> echo $ip >> /usr/local/etc/pf/pf.VNCallow
> echo "Added $ip to VNC Access from MYSQL Table"
> /etc/rc.d/pf reload
> done

that loop at the end is anything but KISS.

select the new addresses and add them to the table with something like

pfctl -t VNCallow -T add $DBS

instead of that do loop. for persistence across reboots, select all the
address in your SQL table & add them to the pf table when pf starts.
clear the table with

pfctl -t VNCallow -T flush

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Limit connections doesn't work

2009-12-06T15:01:16Z

Torsten Kersandt wrote:
> HI
> I personally have all ssh and alike ports closed on my servers.
> If I want to connect to the server per ssh or whatever function, I login to a hidden php which adds my current IP to a sql table.
> I use sql because I'm not the only one using this and want to keep track which admin is logging in.
> A cron job is running every minute looking in the table and adding the new ip addresses to the pf include file and reloading PF
>
> Every night at 4am, I empty the text file and reload pf.
>
> I know that this could be done more elegant but KISS is what I like.

that script is horribly inefficient and disruptive to your firewall
throughput.

you could save a lot of unnecessary cpu cycles and speed up your
connections a bit by simply replacing the reloads with pfctl
commands that manipulate the table directly.

> #!/bin/sh
> ### MySQL Setup ###
> MUSER="username"
> MPASS="password"
> MHOST="localhost"
> MYSQL="/usr/local/bin/mysql"
> #
> ### Get all new IP addresses ###
> DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'select ipAddress from intranet.ipCleared WHERE `timestamp` > (UNIX_TIMESTAMP()-60)')"
> for ip in $DBS
> do
> ## this bit is emailed to me over cron run-output if a new IP address was found
> echo $ip >> /usr/local/etc/pf/pf.VNCallow
> echo "Added $ip to VNC Access from MYSQL Table"
> /etc/rc.d/pf reload
> done

that loop at the end is anything but KISS.

select the new addresses and add them to the table with something like

pfctl -t VNCallow -T add $DBS

instead of that do loop. for persistence across reboots, select all the
address in your SQL table & add them to the pf table when pf starts.
clear the table with

pfctl -t VNCallow -T flush
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

RE: Limit connections doesn't work

2009-12-06T08:17:42Z

HI
I personally have all ssh and alike ports closed on my servers.
If I want to connect to the server per ssh or whatever function, I login to a hidden php which adds my current IP to a sql table.
I use sql because I'm not the only one using this and want to keep track which admin is logging in.
A cron job is running every minute looking in the table and adding the new ip addresses to the pf include file and reloading PF

Every night at 4am, I empty the text file and reload pf.

I know that this could be done more elegant but KISS is what I like.

In addition I have tcpserver running a perl script on a non privileged port to add a IP to the sql tables if apache fails.

#!/bin/sh
### MySQL Setup ###
MUSER="username"
MPASS="password"
MHOST="localhost"
MYSQL="/usr/local/bin/mysql"
#
### Get all new IP addresses ###
DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'select ipAddress from intranet.ipCleared WHERE `timestamp` > (UNIX_TIMESTAMP()-60)')"
for ip in $DBS
do
## this bit is emailed to me over cron run-output if a new IP address was found
echo $ip >> /usr/local/etc/pf/pf.VNCallow
echo "Added $ip to VNC Access from MYSQL Table"
/etc/rc.d/pf reload
done

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Limit connections doens't work

2009-12-06T05:18:21Z

Nico De Dobbeleer wrote:
> Hello,
>
> As most of the public ip's my servers are constantly under bruteforce attack see example:
>
> Dec 5 13:56:36 hosting sshd[18621]: Failed password for invalid user tim from 173.10.126.226 port 47871 ssh2
> Dec 5 13:56:37 hosting sshd[18623]: Invalid user support123 from 173.10.126.226
> Dec 5 13:56:39 hosting sshd[18623]: Failed password for invalid user support123 from 173.10.126.226 port 48289 ssh2

...

>
> Now I want to limit the connection over ssh to a specific ipaddress and I added the rules below for that.
> ------------------------------------------------------------------------------------------------------------------
> #Tables
> table <abusive_ips> persist file "/etc/pf.abusive_ips.block.list"
> table <brute> persist
>
> # Rules
>
> block quick from <abusive_ips>
> block quick from <brute>
>
>
> # Limit connections per IP
>
> pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
> (max-src-conn 10, max-src-conn-rate 3/15, overload <abusive_ips> flush)
> pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
> (max-src-conn 10, max-src-conn-rate 3/15, overload <brute> flush)
> pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
> (max-src-conn 10, max-src-conn-rate 3/15, overload <abusive_ips> flush)
> --------------------------------------------------------------------------------------------------------------------
>
> The only problem is that it doesn't work. These rules don't write the abusive ip in the abusif list file or in the <brute> table.
>
> Anyone an idea why it doesn't overload the ip's when the connections per ip are more then 10 of more then 3/15?

- which FreeBSD version is this 6.x 7.x/8.x?

- avoid the quick keyword in the rules with overload

- pf can preload IP's from file specified in
"table <tablename> ... file "/filename" but does not write IP's into
the file. I use the script below to do this on a OpenBSD machine.

- rewrite your rule and avoid the any keyword
pass in on { $ext_if, $int_if, $mng_if } inet proto tcp \
from !<brute> to xx.xx.xx.xx port ssh flags S/SA keep state \
(max-src-conn 10, max-src-conn-rate 3/15, overload <brute> flush)

hint:
- look for the additional keyword global (flush global)
- If the IP in your rule is your base IP on $ext_if write it with as
$ext_if:0

this script writes IP's from the bf_* tables into a file so you can
preload them next time pf rules are installed or the machine reboots.
additional it can send you a mail with IP's added to the table and if
GeoIP is installed you get the GeoIP info.

With a little modification of the script/rules It will work for you

#!/bin/sh
##################################################################
# $Source: RCS/pftable_to_file.sh,v $
# OS: OpenBSD
#
# olli hauer
#
##################################################################

# sample rule for pf
# ---------------------------
# block in log quick proto { tcp, udp } from <bf_ssh> \
# to any port ssh label BRUTFORCE-SSH # table for overload connections
#
# pass in log on $if_ext inet proto tcp from ! <bf_ssh> to $if_ext \
# port = ssh flags S/SA keep state \
# (source-track rule, max-src-conn 10, \
# max-src-conn-rate 3/90, overload <bf_ssh> \
# flush global, if-bound, src.track 90) \
# label "SSH"

umask 077

PF_TABLES="bf_mail bf_ssh bf_web"
OUTDIR="/etc/pf"
GEOIP=/usr/local/bin/geoiplookup

# hold the output from pfctl -tx -Ts
TMP_PFCTL=`mktemp /tmp/.tmp_pf_table.XXXXXXXXXX` || exit 1

# hold the diff between old and new
TMP_DIFF=`mktemp /tmp/.tmp_diff.XXXXXXXXXX` || exit 1

trap 'rm -f ${TMP_PFCTL} ${TMP_DIFF}' 0 1 2 3 13 15

[ -d ${OUTDIR} ] || mkdir -p ${OUTDIR}

for TABLE in ${PF_TABLES}; do
# make sure the output file exists
[ -f ${OUTDIR}/${TABLE} ] || /usr/bin/touch ${OUTDIR}/${TABLE}

# extraxt IP's from table
/sbin/pfctl -t${TABLE} -Ts | awk '{print $1}' > ${TMP_PFCTL}

# we need only the '+diff' to grep for this later
/usr/bin/diff -bu ${OUTDIR}/${TABLE} ${TMP_PFCTL} > ${TMP_DIFF}
RETVAL=$?

case ${RETVAL} in
0) continue ;;
1)
# save the old file
if [ -f ${OUTDIR}/${TABLE} ]; then
cp ${OUTDIR}/${TABLE} ${OUTDIR}/${TABLE}.old
fi

# mail message header
date
echo "change in table: ${TABLE}"
echo "------------------------------------"

# lookup the IP in the GeoIP database
if [ -x ${GEOIP} ]; then
for IP in `egrep "^\+[0-9]" ${TMP_DIFF} | tr -d \+`; do

# print the IP wo. linefeed
printf "%-20s # " ${IP}

# strip netmask if we add NET by hand
IPT=`echo ${IP} | sed 's/\/[[:digit:]]*//g'`

# make a short GeoIP output
${GEOIP} ${IPT} | sed 's/ Country Edition//g'
done
else
egrep "^\+[0-9]" ${TMP_DIFF} | tr -d \+
fi

mv ${TMP_PFCTL} ${OUTDIR}/${TABLE}
;;

*) echo "error in diff" ;;
esac
done

small snippet from my bf_ssh file (places with IP rangees I don't visit
in near time)

snippet from file:/etc/pf/bf_ssh
12.0.0.0/8
21.0.0.0/8
24.0.0.0/8
25.0.0.0/8
26.0.0.0/8
28.0.0.0/8
29.0.0.0/8
30.0.0.0/8
32.0.0.0/8
33.0.0.0/8
38.0.0.0/8
58.0.0.0/8
59.0.0.0/8
60.0.0.0/8
61.0.0.0/8
62.0.0.0/8
63.0.0.0/8
64.0.0.0/8
...
216.0.0.0/8
217.0.0.0/8
218.0.0.0/8
219.0.0.0/8
220.0.0.0/8
221.0.0.0/8
222.0.0.0/8

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: Limit connections doens't work

2009-12-06T01:19:11Z

--- Original Message ---
From: Nico De Dobbeleer <nico@...>
To: freebsd-pf@...
Date: 5 december, 16:09:52
Subject: Limit connections doens't work

Hello,

As most of the public ip's my servers are constantly under bruteforce attack see example:

Dec 5 13:56:36 hosting sshd[18621]: Failed password for invalid user tim from 173.10.126.226 port 47871 ssh2
Dec 5 13:56:37 hosting sshd[18623]: Invalid user support123 from 173.10.126.226
Dec 5 13:56:39 hosting sshd[18623]: Failed password for invalid user support123 from 173.10.126.226 port 48289 ssh2
Dec 5 13:56:41 hosting sshd[18625]: Invalid user support from 173.10.126.226
Dec 5 13:56:43 hosting sshd[18625]: Failed password for invalid user support from 173.10.126.226 port 48676 ssh2
Dec 5 13:56:47 hosting sshd[18627]: Invalid user jnanchito from 173.10.126.226
Dec 5 13:56:50 hosting sshd[18627]: Failed password for invalid user jnanchito from 173.10.126.226 port 49122 ssh2
Dec 5 13:56:51 hosting sshd[18629]: Invalid user rtorres from 173.10.126.226
Dec 5 13:56:53 hosting sshd[18629]: Failed password for invalid user rtorres from 173.10.126.226 port 49872 ssh2
Dec 5 13:56:55 hosting sshd[18631]: Invalid user jatema from 173.10.126.226
Dec 5 13:56:57 hosting sshd[18631]: Failed password for invalid user jatema from 173.10.126.226 port 50293 ssh2
Dec 5 13:57:01 hosting sshd[18633]: Failed password for invalid user root from 173.10.126.226 port 50702 ssh2
Dec 5 13:57:04 hosting sshd[18635]: Failed password for invalid user root from 173.10.126.226 port 51154 ssh2
Dec 5 13:57:06 hosting sshd[18637]: Invalid user boss from 173.10.126.226
Dec 5 13:57:08 hosting sshd[18637]: Failed password for invalid user boss from 173.10.126.226 port 51507 ssh2
Dec 5 13:57:09 hosting sshd[18639]: Invalid user sasha from 173.10.126.226
Dec 5 13:57:11 hosting sshd[18639]: Failed password for invalid user sasha from 173.10.126.226 port 51929 ssh2
Dec 5 13:57:13 hosting sshd[18641]: Invalid user vic from 173.10.126.226
Dec 5 13:57:14 hosting sshd[18641]: Failed password for invalid user vic from 173.10.126.226 port 52321 ssh2
Dec 5 13:57:16 hosting sshd[18643]: Invalid user ranjith from 173.10.126.226
Dec 5 13:57:18 hosting sshd[18643]: Failed password for invalid user ranjith from 173.10.126.226 port 52650 ssh2
Dec 5 13:57:21 hosting sshd[18645]: Failed password for invalid user root from 173.10.126.226 port 53087 ssh2
Dec 5 13:57:25 hosting sshd[18647]: Failed password for invalid user root from 173.10.126.226 port 53447 ssh2
Dec 5 13:57:29 hosting sshd[18649]: Failed password for invalid user root from 173.10.126.226 port 53852 ssh2

Now I want to limit the connection over ssh to a specific ipaddress and I added the rules below for that.
------------------------------------------------------------------------------------------------------------------
#Tables
table <abusive_ips> persist file "/etc/pf.abusive_ips.block.list"
table <brute> persist

# Rules

block quick from <abusive_ips>
block quick from <brute>

# Limit connections per IP

pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
(max-src-conn 10, max-src-conn-rate 3/15, overload <abusive_ips> flush)
pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
(max-src-conn 10, max-src-conn-rate 3/15, overload <brute> flush)
pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
(max-src-conn 10, max-src-conn-rate 3/15, overload <abusive_ips> flush)
--------------------------------------------------------------------------------------------------------------------

The only problem is that it doesn't work. These rules don't write the abusive ip in the abusif list file or in the <brute> table.

Anyone an idea why it doesn't overload the ip's when the connections per ip are more then 10 of more then 3/15?

With kind regards,
Nico De Dobbeleer

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

I think you should specify

source-track rule (rule or lobal) in your rulesLike this:

pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
(max 10, source-track rule, max-src-conn 10, max-src-conn-rate 3/15, overload <abusive_ips> flush)

See in PF FAQ

Stateful Tracking Options.

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Limit connections doens't work

2009-12-05T06:09:52Z

Hello,

As most of the public ip's my servers are constantly under bruteforce attack see example:

Dec 5 13:56:36 hosting sshd[18621]: Failed password for invalid user tim from 173.10.126.226 port 47871 ssh2
Dec 5 13:56:37 hosting sshd[18623]: Invalid user support123 from 173.10.126.226
Dec 5 13:56:39 hosting sshd[18623]: Failed password for invalid user support123 from 173.10.126.226 port 48289 ssh2
Dec 5 13:56:41 hosting sshd[18625]: Invalid user support from 173.10.126.226
Dec 5 13:56:43 hosting sshd[18625]: Failed password for invalid user support from 173.10.126.226 port 48676 ssh2
Dec 5 13:56:47 hosting sshd[18627]: Invalid user jnanchito from 173.10.126.226
Dec 5 13:56:50 hosting sshd[18627]: Failed password for invalid user jnanchito from 173.10.126.226 port 49122 ssh2
Dec 5 13:56:51 hosting sshd[18629]: Invalid user rtorres from 173.10.126.226
Dec 5 13:56:53 hosting sshd[18629]: Failed password for invalid user rtorres from 173.10.126.226 port 49872 ssh2
Dec 5 13:56:55 hosting sshd[18631]: Invalid user jatema from 173.10.126.226
Dec 5 13:56:57 hosting sshd[18631]: Failed password for invalid user jatema from 173.10.126.226 port 50293 ssh2
Dec 5 13:57:01 hosting sshd[18633]: Failed password for invalid user root from 173.10.126.226 port 50702 ssh2
Dec 5 13:57:04 hosting sshd[18635]: Failed password for invalid user root from 173.10.126.226 port 51154 ssh2
Dec 5 13:57:06 hosting sshd[18637]: Invalid user boss from 173.10.126.226
Dec 5 13:57:08 hosting sshd[18637]: Failed password for invalid user boss from 173.10.126.226 port 51507 ssh2
Dec 5 13:57:09 hosting sshd[18639]: Invalid user sasha from 173.10.126.226
Dec 5 13:57:11 hosting sshd[18639]: Failed password for invalid user sasha from 173.10.126.226 port 51929 ssh2
Dec 5 13:57:13 hosting sshd[18641]: Invalid user vic from 173.10.126.226
Dec 5 13:57:14 hosting sshd[18641]: Failed password for invalid user vic from 173.10.126.226 port 52321 ssh2
Dec 5 13:57:16 hosting sshd[18643]: Invalid user ranjith from 173.10.126.226
Dec 5 13:57:18 hosting sshd[18643]: Failed password for invalid user ranjith from 173.10.126.226 port 52650 ssh2
Dec 5 13:57:21 hosting sshd[18645]: Failed password for invalid user root from 173.10.126.226 port 53087 ssh2
Dec 5 13:57:25 hosting sshd[18647]: Failed password for invalid user root from 173.10.126.226 port 53447 ssh2
Dec 5 13:57:29 hosting sshd[18649]: Failed password for invalid user root from 173.10.126.226 port 53852 ssh2

Now I want to limit the connection over ssh to a specific ipaddress and I added the rules below for that.
------------------------------------------------------------------------------------------------------------------
#Tables
table <abusive_ips> persist file "/etc/pf.abusive_ips.block.list"
table <brute> persist

# Rules

block quick from <abusive_ips>
block quick from <brute>

# Limit connections per IP

pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
(max-src-conn 10, max-src-conn-rate 3/15, overload <abusive_ips> flush)
pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
(max-src-conn 10, max-src-conn-rate 3/15, overload <brute> flush)
pass in quick on { $ext_if, $int_if, $mng_if } inet proto tcp from any to xx.xx.xx.xx port ssh flags S/SA keep state
(max-src-conn 10, max-src-conn-rate 3/15, overload <abusive_ips> flush)
--------------------------------------------------------------------------------------------------------------------

The only problem is that it doesn't work. These rules don't write the abusive ip in the abusif list file or in the <brute> table.

Anyone an idea why it doesn't overload the ip's when the connections per ip are more then 10 of more then 3/15?

With kind regards,
Nico De Dobbeleer

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: PF + load balancing over 100Mbit traffic

2009-12-04T01:42:14Z

Adam PAPAI wrote on 2009-12-03 09:19:

> Do you have any advice? Is it time to get a Layer 7 switch and do load
> balancing with it? Or is it possible to do it in a PF way without a
> Content Switch?

My advice is to use content switching via some 3rd-party software, for
example HAProxy - perfect tool for rr/lb. I have webclusters set up this
way, some of them handle even 200-250Mbps in peak. No problems. If you
need any additional info, let me know.

--
* Maciej Wierzbicki * At paranoia's poison door *
* VOO1-RIPE *
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: PF + load balancing over 100Mbit traffic [SOLVED]

2009-12-03T03:19:42Z

Gergely CZUCZY wrote:

> On Thu, 03 Dec 2009 09:19:29 +0100
> Adam PAPAI <wooh@...> wrote:
>
>> Dear List,
>>
>> I have a feeling that PF can't do perfect round-robin load balancing
>> over 100Mbit.
>>
>> When our PF server's (Dual Quad Core 3Ghz with 8GB ram) network
>> traffic goes over 100Mbit, the 80 port's connect time increases to
>> 3-5-10 sec instead of the stable 0.001-0.002 sec. The web servers
>> feel good, they don't have load, the redundant master-slave database
>> servers feel good, they dont have high load. So everything seems
>> fine, except the connect time. (Our checker script asks only a HEAD
>> request from the web servers)
> Have you adjusted the TCP timeout parameters? this can be caused by the
> standard 30sec timeouts and your state table is getting filled up. I'd
> check the following parameters:
> - timeout tcp.{closing,finwait,closed}
> - interval
> - limit states
>

Thanks,

I really had to change the pf.conf and set the values below.

set limit states 40000
set timeout interval 5

In the future i should increase the limit states and reduce the timeout
interval :)

Since i've changed the values, the connection time is between 0.001 and
0.004, the traffic is over than 110Mbit.

\o/

Have a nice day :)

--
Adam PAPAI
NETIDEA Informatikai Szolgaltato Kft.
http://www.netidea.hu
E-mail: wooh@...

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: PF + load balancing over 100Mbit traffic

2009-12-03T01:13:27Z

On Thu, 03 Dec 2009 09:19:29 +0100
Adam PAPAI <wooh@...> wrote:

> Dear List,
>
> I have a feeling that PF can't do perfect round-robin load balancing
> over 100Mbit.
>
> When our PF server's (Dual Quad Core 3Ghz with 8GB ram) network
> traffic goes over 100Mbit, the 80 port's connect time increases to
> 3-5-10 sec instead of the stable 0.001-0.002 sec. The web servers
> feel good, they don't have load, the redundant master-slave database
> servers feel good, they dont have high load. So everything seems
> fine, except the connect time. (Our checker script asks only a HEAD
> request from the web servers)

Have you adjusted the TCP timeout parameters? this can be caused by the
standard 30sec timeouts and your state table is getting filled up. I'd
check the following parameters:
- timeout tcp.{closing,finwait,closed}
- interval
- limit states

pftop can be a great help for checking pf's behaviour, it's available
in ports.

>
> The internal network has Gbit connection so as the internet side.
>
> Do you have any advice? Is it time to get a Layer 7 switch and do
> load balancing with it? Or is it possible to do it in a PF way
> without a Content Switch?
>
> 2 web servers and 2 database servers are involved.
>
> [web 1] ---|
> [web 2] ---|
> [db 1] ---|---[pf/web 3/default gw]---internet
> [db 2] ---|
>
>
> For a while the web server on the PF server is down to test, but it
> does the same connection time with a running apache and without a
> running apache.
>
> Any idea? Our internet traffic average is 100Mbit-130Mbit and the
> connect time makes me so sad.
>
> Thanks in advance,
>

--
Sincerely,
Gergely CZUCZY
Harmless Digital Bt

+36-30-9702963
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

PF + load balancing over 100Mbit traffic

2009-12-03T00:19:29Z

Dear List,

I have a feeling that PF can't do perfect round-robin load balancing
over 100Mbit.

When our PF server's (Dual Quad Core 3Ghz with 8GB ram) network traffic
goes over 100Mbit, the 80 port's connect time increases to 3-5-10 sec
instead of the stable 0.001-0.002 sec. The web servers feel good, they
don't have load, the redundant master-slave database servers feel good,
they dont have high load. So everything seems fine, except the connect
time. (Our checker script asks only a HEAD request from the web servers)

The internal network has Gbit connection so as the internet side.

Do you have any advice? Is it time to get a Layer 7 switch and do load
balancing with it? Or is it possible to do it in a PF way without a
Content Switch?

2 web servers and 2 database servers are involved.

[web 1] ---|
[web 2] ---|
[db 1] ---|---[pf/web 3/default gw]---internet
[db 2] ---|

For a while the web server on the PF server is down to test, but it does
the same connection time with a running apache and without a running apache.

Any idea? Our internet traffic average is 100Mbit-130Mbit and the
connect time makes me so sad.

Thanks in advance,

--
Adam PAPAI
NETIDEA Informatikai Szolgaltato Kft.
http://www.netidea.hu
E-mail: wooh@...

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

PF + load balancing over 100Mbit

2009-12-03T00:16:54Z

Current problem reports assigned to freebsd-pf@FreeBSD.org

2009-11-30T03:06:58Z

trying to figure out how to altq single interface..

2009-11-29T07:20:11Z

I'm trying to setup a valid test case on a single machine running
FreeBSD 8.0R. (amd64 and i386)

Seems all of the pf/altq examples assume that FreeBSD is the router, and
you are working with more than on interface.

I am trying to shape my traffic on a per physical server basis with on a
*single* interface where the NAT is performed elsewhere.

I understand the logic that you can only queue outgoing.

My goal is to have full bandwidth for the local network (interface
speed) with ack priq if possible - and - have queues for services when
the "not local network" users access them.

There are ASA devices further up the line from me which I have no
control over; they are providing NAT..

we have a large private network (10.20.0.0/18) but we have some machines
that serve the public as well as serves the lan.

(feel free to interject reality into my logic where need be.. )

here is what I have come up with as far as altq/queues is concerned:

pfctl -sq
queue root_bge0 on bge0 bandwidth 1Mb priority 0 cbq( wrr root ) {idef,
iack, http, dns, mua, icmp, smtp, ssh}
queue idef on bge0 bandwidth 100Kb cbq( borrow default )
queue iack on bge0 bandwidth 100Kb priority 7 cbq( borrow )
queue http on bge0 bandwidth 800Kb priority 5 cbq( borrow )
queue dns on bge0 bandwidth 200Kb priority 6 cbq( borrow )
queue mua on bge0 bandwidth 800Kb priority 2 cbq( borrow )
queue icmp on bge0 bandwidth 50Kb priority 6
queue smtp on bge0 bandwidth 500Kb cbq( borrow )
queue ssh on bge0 bandwidth 100Kb priority 6 cbq( borrow ) {scp, term}
queue scp on bge0 bandwidth 80Kb priority 0
queue term on bge0 bandwidth 20Kb priority 7

(or if it matters - directly from pf.conf.local)

48 altq on $ext_if cbq bandwidth 1Mb queue { idef, iack, http, dns,
mua, icmp, smtp, ssh }
49 queue idef bandwidth 10% cbq(default borrow)
50 queue iack bandwidth 10% cbq(borrow) priority 7
51 queue http bandwidth 80% cbq(borrow) priority 5
52 queue dns bandwidth 20% cbq(borrow) priority 6
53 queue mua bandwidth 80% cbq(borrow) priority 2
54 queue icmp bandwidth 5% priority 6
55 queue smtp bandwidth 50% cbq(borrow) priority 1
56 queue ssh bandwidth 10% cbq(borrow) priority 6 {
scp, term }
57 queue scp bandwidth 80% priority 0
58 queue term bandwidth 20% priority 7

My question(s) are:

When do I apply things to pass in and when do I apply to pass out?

It seems when I don't apply a queue rule to a pass in/out rule the
default kicks in, which is fine; but why does it 'queue on inbound' when
it can only 'queue on outbound'? - is keeping state what is altering that?

pfctl -sr | cat -n
1 scrub in all no-df random-id fragment reassemble
2 block return in log all
3 block return in log quick from <blocksshd> to any
4 pass out on bge0 inet proto icmp from (bge0) to any keep state
queue icmp
5 pass out on bge0 inet proto udp from (bge0) to any port = domain
keep state queue dns
6 pass out on bge0 inet proto udp from (bge0) to any port = ntp
keep state queue dns
7 pass out on bge0 inet proto udp from (bge0) to any port = snmp
keep state queue dns
8 pass out on bge0 inet proto tcp from (bge0) to any port = ssh
flags S/SA keep state queue(scp, term)
9 block drop in log quick on ! bge0 inet from 10.20.0.0/25 to any
10 block drop in log quick inet from 10.20.0.5 to any
11 block drop in log quick inet from 10.20.0.4 to any
12 block drop in log quick inet from 10.20.0.19 to any
13 pass in quick on bge0 inet proto udp from 10.20.0.0/23 to
10.20.0.4 port = domain keep state queue dns
14 pass in quick on bge0 inet proto udp from 10.20.0.0/23 to
10.20.0.4 port = ntp keep state queue dns
15 pass in quick on bge0 inet proto udp from 10.20.0.0/23 to
10.20.0.4 port = snmp keep state queue dns
16 pass in quick on bge0 inet proto udp from 10.20.0.0/23 to
10.20.0.4 port = syslog keep state queue dns
17 pass in quick on bge0 inet proto udp from any to 10.20.0.19 port
= domain keep state queue dns
18 pass in quick on bge0 inet proto tcp from 10.20.0.0/25 to (bge0)
port = smtp flags S/SA keep state
19 pass in quick on bge0 inet proto tcp from 10.20.0.0/25 to (bge0)
port = rsync flags S/SA keep state
20 pass in quick on bge0 inet proto tcp from any to 10.20.0.5 port
= ssh flags S/SA keep state queue(scp, term)
21 pass in quick on bge0 inet proto tcp from any to 10.20.0.5 port
= http flags S/SA keep state queue(http, iack)
22 pass in quick on bge0 inet proto tcp from any to 10.20.0.5 port
= https flags S/SA keep state queue(http, iack)
23 pass in quick on bge0 inet proto tcp from any to 10.20.0.5 port
= 2359 flags S/SA keep state queue(http, iack)
24 pass in quick on bge0 inet proto tcp from any to 10.20.0.5 port
= 2812 flags S/SA keep state queue(http, iack)
25 pass in quick on bge0 inet proto udp from 10.20.0.0/25 to (bge0)
port = domain keep state
26 pass in quick on bge0 inet proto udp from 10.20.0.0/25 to (bge0)
port = ntp keep state
27 pass in quick on bge0 inet proto udp from 10.20.0.0/25 to (bge0)
port = snmp keep state
28 pass in quick on bge0 inet proto udp from 10.20.0.0/25 to (bge0)
port = syslog keep state
29 pass in quick on bge0 inet proto icmp from any to (bge0)
icmp-type echoreq code 0 keep state

All of these rules might not quite be valid public services, but I was
looking for real services that I could test with.

Also afaict FreeBSD 8 is running with (approximately) version 4.1 of
OpenBSDs PF; is that correct? Assumed from pftop compile output of:

cc -O2 -pipe -DHAVE_ALTQ=1 -fno-strict-aliasing -Wall -DOS_LEVEL=41
-std=gnu99 -fstack-protector -c pftop.c
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: sending mail with attachments always fails (FreeBSD/pf)

2009-11-23T14:17:18Z

Victor Lyapunov <fullblaststorm@...> wrote:

>
> After that i tried to send mail to a server that does not require ssl
> and i got this:
>
> rule 1/0(match): pass in on em0: 192.168.0.5.2035 > 94.100.177.1.25: S
> 237079791:237079791(0) win 65535 <mss 1460,nop,nop,sackOK>
> rule 1/0(match): pass out on em0: 192.168.0.5.2035 > 94.100.177.1.25:
> S 237079791:237079791(0) win 65535 <mss 1460,nop,nop,sackOK>
> 2 packets captured
> 2 packets received by filter
> 0 packets dropped by kernel

This doesn't appear to be the same problem you originally submitted,
about SMTP connections with no attachments working fine, but with
attachments they fail. Seems like you are now describing that SMTP
doesn't work at all.

> 192.168.0.1 -- Router
> 192.168.0.3 -- The FreeBSD box
> 192.168.0.5 -- Windows machine with default gateway set to 192.168.0.3

This is probably the source of your problems. Your router and your
firewall and your firewalled client are all on the same subnet together.
There is nothing preventing the router from sending packets directly
back to the Windows box, bypassing your firewall.

As such, the firewall cannot see any of the reply traffic, and so it
cannot follow the TCP state correctly, so eventually it begins to block
the traffic. If you turn on logging with "pfctl -x loud" you will
probably see a lot of messages about TCP state mismatches.

The proper way to fix this is to rearchitect your network so that your
firewall has two interfaces, one public, one private. The public
interface connects only to your router, while the private interface
connects to all your firewall clients. This forces the firewall to be
the only path to and from the network, giving enhanced security.

--
David DeSimone == Network Admin == fox@...
"I don't like spinach, and I'm glad I don't, because if I
liked it I'd eat it, and I just hate it." -- Clarence Darrow

This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you.
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Re: block ip's and ports

2009-11-23T08:22:41Z

Sife Mailling wrote:

> Salamo Alikom
> i setup a firewall for personnel home computer ,now i want every packets block if it is not pass to specified ports .
> this my pf.conf :
> net_card="sis0"
> tcp_ports="{80 ,https ,domain ,auth ,21}"
> udp_ports="{domain}"
> table <banned> file "/etc/pf/banned"
> table <banned2> {www.google.com}
> block in log (all) on $net_card proto {tcp ,udp} all
> pass in on $net_card proto tcp from any to any port $tcp_ports
> pass in on $net_card proto udp from any to any port $udp_ports
> pass in on $net_card proto tcp from 192.168.0.0/16 to 192.168.0.0/16
> block in on $net_card proto tcp from { <banned>, <banned2> } to any port $tcp_ports
> pass out on $net_card proto tcp from any to any port $tcp_ports
> pass out on $net_card proto udp from any to any port $udp_ports
> pass out on $net_card inet proto tcp from any to any port ftp
> pass out on $net_card inet proto tcp from any to any port > 1023
>
> now skype is work and the both tables banned and banned2 i can browse sites including theme .
>

Try the quick keyword, so traffic is not allowed in later rules.

Additional disable outgoing traffic since if you create a connect from
inside to <banned> a state which permits incoming traffic is created.

example ordering:

table <banned> file "/etc/pf/banned"
table <banned2> {www.google.com}
block in log (all) on $net_card proto {tcp ,udp} all
block in quick on $net_card proto tcp from { <banned>, <banned2> } \
to any port $tcp_ports label blockin
block out quick on $net_card proto tcp from { <banned>, <banned2> } \
to any port $tcp_ports label blockout
pass in on $net_card proto tcp from any to any port $tcp_ports
_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

block ip's and ports

2009-11-23T07:35:08Z

Salamo Alikom
i setup a firewall for personnel home computer ,now i want every packets block if it is not pass to specified ports .
this my pf.conf :
net_card="sis0"
tcp_ports="{80 ,https ,domain ,auth ,21}"
udp_ports="{domain}"
table <banned> file "/etc/pf/banned"
table <banned2> {www.google.com}
block in log (all) on $net_card proto {tcp ,udp} all
pass in on $net_card proto tcp from any to any port $tcp_ports
pass in on $net_card proto udp from any to any port $udp_ports
pass in on $net_card proto tcp from 192.168.0.0/16 to 192.168.0.0/16
block in on $net_card proto tcp from { <banned>, <banned2> } to any port $tcp_ports
pass out on $net_card proto tcp from any to any port $tcp_ports
pass out on $net_card proto udp from any to any port $udp_ports
pass out on $net_card inet proto tcp from any to any port ftp
pass out on $net_card inet proto tcp from any to any port > 1023

now skype is work and the both tables banned and banned2 i can browse sites including theme .

_______________________________________________
freebsd-pf@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscribe@..."

Current problem reports assigned to freebsd-pf@FreeBSD.org

2009-11-23T03:07:00Z