ipv6/pf/relayd/totd

[Stephan A. Rickauer]

I started playing with ipv6. It feels like back in the early 90's, when
I had to learn how 'the Internet' works ;)

Here's the setup:

An ipv6 only host with a non-link-local address should be able to use
the ipv4 world. I don't want to deal with a tunnel broker, nor do I have
native ipv6 access to the internet.

The ipv6 only client gets its ipv6 address via the rtadvd running on the
gatway's internal interface. The gateway's external interface is ipv4
only.

The ipv6 host can already ping6 the gatway. DNS I have 'fixed' with
totd, so ipv4 addressed are mapped into the ipv6 space:

ipv6-client:~$ host www.google.ch
www.l.google.com has address 74.125.39.147
www.l.google.com has IPv6 address 2001:620:10:1401::4a7d:2767


The default ipv6-gateway of my ipv6 client is properly set
in /etc/mygate.

I try to use pf on the gateway to intercept tcp/ip6 traffic and to feed
it into relayd. The relevant parts are as follows:


---pf.conf--
rdr pass inet6 proto tcp from lan:network -> :: port 8081
---pf.conf--


---relayd.conf---
tcp protocol tcpgeneric {
        tcp { backlog 128, nodelay, sack, socket buffer 131072 }
}

relay tcp6to4 {
        listen on :: port 8081
        forward to nat lookup inet
        protocol tcpgeneric
}
---relayd.conf---


After that kinda long intro, here's the problem:

Though name resolution works, an actual connection to an ipv6 address on
port 80 wouldn't work and isn't 'seen' by relayd either. If I tcpdump on
the gateway I see that the client, after it got the faked ipv6 address,
sends an "icmp6: neighbor sol: who has 2001:620:10:1401::4a7d:2767".

So, it believes google is part of 'our' name space, which is probably
wrong. I then tried to change the prefix of totd to a non-local prefix,
like 2001:620:10:1400:: (instead of :1401::) so that a 'host
www.google.ch' results in 2001:620:10:1400::4a7d:2767 and thus can't be
treated as 'local'.

When I do this I can see the traffic on the gatway:
2001:620:10:1401:20d:60ff:fe2e:251b.13239 >
2001:620:10:1400::4a7d:2768.80

but it's still not seen by relayd.

Can someone with some degree of patience shed some light on my dark
spots?

--

 Stephan A. Rickauer

 -----------------------------------------------------------
 Institute of Neuroinformatics         Tel  [phone]
 University / ETH Zurich               Sec  [phone]
 Winterthurerstrasse 190               Fax  [phone]
 CH-8057 Zurich                        Web    www.ini.uzh.ch

[Dirk Mast]

Stephan A. Rickauer wrote:

> I started playing with ipv6. It feels like back in the early 90's, when
> I had to learn how 'the Internet' works ;)
>
> Here's the setup:
>
> An ipv6 only host with a non-link-local address should be able to use
> the ipv4 world. I don't want to deal with a tunnel broker, nor do I have
> native ipv6 access to the internet.
>
> The ipv6 only client gets its ipv6 address via the rtadvd running on the
> gatway's internal interface. The gateway's external interface is ipv4
> only.
>
> The ipv6 host can already ping6 the gatway. DNS I have 'fixed' with
> totd, so ipv4 addressed are mapped into the ipv6 space:
>
> ipv6-client:~$ host www.google.ch
> www.l.google.com has address 74.125.39.147
> www.l.google.com has IPv6 address 2001:620:10:1401::4a7d:2767
>
>
> The default ipv6-gateway of my ipv6 client is properly set
> in /etc/mygate.
>
> I try to use pf on the gateway to intercept tcp/ip6 traffic and to feed
> it into relayd. The relevant parts are as follows:
>
>
> ---pf.conf--
> rdr pass inet6 proto tcp from lan:network -> :: port 8081
> ---pf.conf--
>
>
> ---relayd.conf---
> tcp protocol tcpgeneric {
>         tcp { backlog 128, nodelay, sack, socket buffer 131072 }
> }
>
> relay tcp6to4 {
>         listen on :: port 8081
>         forward to nat lookup inet
>         protocol tcpgeneric
> }
> ---relayd.conf---
>
>
> After that kinda long intro, here's the problem:
>
> Though name resolution works, an actual connection to an ipv6 address on
> port 80 wouldn't work and isn't 'seen' by relayd either. If I tcpdump on
> the gateway I see that the client, after it got the faked ipv6 address,
> sends an "icmp6: neighbor sol: who has 2001:620:10:1401::4a7d:2767".
>
> So, it believes google is part of 'our' name space, which is probably
> wrong. I then tried to change the prefix of totd to a non-local prefix,
> like 2001:620:10:1400:: (instead of :1401::) so that a 'host
> www.google.ch' results in 2001:620:10:1400::4a7d:2767 and thus can't be
> treated as 'local'.
>
> When I do this I can see the traffic on the gatway:
> 2001:620:10:1401:20d:60ff:fe2e:251b.13239 >
> 2001:620:10:1400::4a7d:2768.80
>
> but it's still not seen by relayd.
>
> Can someone with some degree of patience shed some light on my dark
> spots?
>
> --
>
>  Stephan A. Rickauer
>
>  -----------------------------------------------------------
>  Institute of Neuroinformatics         Tel  [phone]
>  University / ETH Zurich               Sec  [phone]
>  Winterthurerstrasse 190               Fax  [phone]
>  CH-8057 Zurich                        Web    www.ini.uzh.ch

Hi, have you already seen this great post on undeadly?

http://undeadly.org/cgi?action=article&sid=20080724184757

Perhaps it might help you with your setup.

[Stephan A. Rickauer]

On Tue, 2008-12-16 at 16:32 +0100, Dirk Mast wrote:
> Hi, have you already seen this great post on undeadly?

Yes, I have. Without it, I wouldn't have come so far ;)

[Matthew Dempsky-3]

On Tue, Dec 16, 2008 at 7:14 AM, Stephan A. Rickauer
<stephan.rickauer@...> wrote:
> An ipv6 only host with a non-link-local address should be able to use
> the ipv4 world.

Is this just for fun/practice, or is there a reason you can't just
configure the host with both an IPv4 and an IPv6 address?

> I don't want to deal with a tunnel broker, nor do I have
> native ipv6 access to the internet.

I was hesitant to setup a tunnel broker for a while too, but it turned
out to be much less painful than I expected.

[Jeroen Massar]

Matthew Dempsky wrote:
> On Tue, Dec 16, 2008 at 7:14 AM, Stephan A. Rickauer
> <stephan.rickauer@...> wrote:
>> An ipv6 only host with a non-link-local address should be able to use
>> the ipv4 world.
>
> Is this just for fun/practice, or is there a reason you can't just
> configure the host with both an IPv4 and an IPv6 address?

I guess there is a practical use here, that is, if your tools all
understand IPv6, because then you only have an IPv6 "NAT" to IPv4 and
you skip the IPv4 NAT to IPv4 in case you don't have any IPv4 addresses
for your local network.

I have to note that although these mechanisms exist (and they can work
when properly configured, I did it once one long time ago, but forgot
how it exactly was broken/setup) the "IETF" doesn't recommend using
these kind of mechanisms and recommend that one does dual-stack, thus
IPv4 native or NATted and IPv6 native.

>> I don't want to deal with a tunnel broker, nor do I have
>> native ipv6 access to the internet.
>
> I was hesitant to setup a tunnel broker for a while too, but it turned
> out to be much less painful than I expected.

As he has native IPv6 connectivity he doesn't need a tunnel broker.

I guess he confused a 'tunnel broker' with an entity that can convert
IPv6->IPv4.

Greets,
 Jeroen

[demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]

[Matthew Dempsky-3]

On Wed, Dec 17, 2008 at 12:43 AM, Jeroen Massar <jeroen@...> wrote:
> I guess there is a practical use here, that is, if your tools all
> understand IPv6, because then you only have an IPv6 "NAT" to IPv4 and
> you skip the IPv4 NAT to IPv4 in case you don't have any IPv4 addresses
> for your local network.

It's possible he has a network so large that he has run out of IPv4
addresses, but I suspect if that was the case, he wouldn't be asking
for help here. :-)

In practical terms, running a dual-homed client is much simpler than
running a pure IPv6 client and configuring a proxying router.  That's
all I was trying to point out.

> As he has native IPv6 connectivity he doesn't need a tunnel broker.

No, he explicitly stated that he does not have native IPv6
connectivity in the sentence I quoted.  He also reaffirmed this later
by saying "The gateway's external interface is ipv4 only."

[Stephan A. Rickauer]

On Wed, 2008-12-17 at 00:59 -0800, Matthew Dempsky wrote:

> On Wed, Dec 17, 2008 at 12:43 AM, Jeroen Massar <jeroen@...>
> wrote:
> > I guess there is a practical use here, that is, if your tools all
> > understand IPv6, because then you only have an IPv6 "NAT" to IPv4
> and
> > you skip the IPv4 NAT to IPv4 in case you don't have any IPv4
> addresses
> > for your local network.
>
> It's possible he has a network so large that he has run out of IPv4
> addresses, but I suspect if that was the case, he wouldn't be asking
> for help here. :-)

Bingo! Our institute ran out of ipv4 addresses and we were assigned a
couple of ipv6 address ranges by University. However, University who
operates the network (gateways, routers etc.) don't route/care
about/support ipv6 yet officially. Well, it's a University, why would
they be interested in being ahead ...

Anyhow, this is why I wanted to use ipv6-only clients behind an ipv4-to
ipv6 relay (I know that we could just simply nat more private ipv4
addresses, but that is not an option right now).

For sure OpenBSD brings all you need to do it - it's just that I am
stuck at the moment and can't see why... but maybe I'll figure it out
today!

--

 Stephan A. Rickauer

 -----------------------------------------------------------
 Institute of Neuroinformatics         Tel  [phone]
 University / ETH Zurich               Sec  [phone]
 Winterthurerstrasse 190               Fax  [phone]
 CH-8057 Zurich                        Web    www.ini.uzh.ch

[Jeroen Massar]

Matthew Dempsky wrote:
> On Wed, Dec 17, 2008 at 12:43 AM, Jeroen Massar <jeroen@...> wrote:
>> I guess there is a practical use here, that is, if your tools all
>> understand IPv6, because then you only have an IPv6 "NAT" to IPv4 and
>> you skip the IPv4 NAT to IPv4 in case you don't have any IPv4 addresses
>> for your local network.
>
> It's possible he has a network so large that he has run out of IPv4
> addresses, but I suspect if that was the case, he wouldn't be asking
> for help here. :-)

SWITCH, which is the NREN for the universities in Switzerland, has
enough address space to go by as far as I know, and most universities
have been assigned enough address space to have fun with it too, thus I
don't think that is a problem.

> In practical terms, running a dual-homed client is much simpler than
> running a pure IPv6 client and configuring a proxying router.  That's
> all I was trying to point out.

I fully agree.

>> As he has native IPv6 connectivity he doesn't need a tunnel broker.
>
> No, he explicitly stated that he does not have native IPv6
> connectivity in the sentence I quoted.  He also reaffirmed this later
> by saying "The gateway's external interface is ipv4 only."

Hmm, indeed, thus he is using a chunk of the real allocated IPv6 address
space from uzh.ch (2001:620:10::/48), but doesn't actually route it,
that is the part that confused me.

I am wondering how totd handles hosts that have a real IPv6 address,
especially as he seems to have a disconnected network, thus in the case
that totd keeps the real IPv6 address, he would have blackholed himself
that way.

I am now also wondering what the point of this is, as it is much easier
to then just NAT an IPv4 block, because then stuff just works without
any trickery. Especially as a lot of clients simply don't support IPv6
properly yet. HTTP is not the only protocol and such. And if you are
going to configure all the clients anyway and expect only to use HTTP,
then it is much easier to just setup an HTTP proxy on the gateway than
do all this trickery.

Greets,
 Jeroen

[demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]

[Jeroen Massar]

Stephan A. Rickauer wrote:
[..]
> Bingo! Our institute ran out of ipv4 addresses and we were assigned a
> couple of ipv6 address ranges by University. However, University who
> operates the network (gateways, routers etc.) don't route/care
> about/support ipv6 yet officially. Well, it's a University, why would
> they be interested in being ahead ...

I call BS there, I would suggest you kick your "network administrators"
hard, as all the universities have more than plenty of IPv4 address
space. I would be delighted to see any real concrete evidence to the
contrary. If it is anything it mostly just either means lazy people or
something to do with billing, those are L7 issues though.

I am also very sure that SWITCH will love to hear about this.

As for your network guys not routing the block they gave you, well if
they give it to you they better also route it, thus kick them.
Note that it reaches the uni at least:

 9  swiZH2-10GE-1-1.switch.ch (2001:620:0:c027::2)  5.02 ms  5.041 ms
4.981 ms
10  uzhix1-te1-1.uzh.ch (2001:620:0:fff5::2)  4.981 ms  5.142 ms  4.951 ms
11  internet-router-ifi-csg.uzh.ch (2001:620:10:ffff::2)  5.489 ms !S
5.27 ms !S  5.226 ms !S

There is a firewall there, thus can't tell what is behind it.

> Anyhow, this is why I wanted to use ipv6-only clients behind an ipv4-to
> ipv6 relay (I know that we could just simply nat more private ipv4
> addresses, but that is not an option right now).

You are trying to solve the wrong problem with the wrong hammer.

Just NAT IPv4, what you are trying to do now will only cause a lot of
other issues you don't want and will only cause you headaches.

To solve your problem VERY quickly: install apache2 or polipo or some
other HTTP proxy and you are set; unless you want to use other
protocols, but those are hard to "NAT" anyway. You could of course
always set up a SOCKS proxy as those things also properly do IPv4->IPv6
IPv6->IPv4 IPv4->IPv4 IPv6->IPv6 etc etc etc.

Greets,
 Jeroen

[demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]

[Stephan A. Rickauer]

On Wed, 2008-12-17 at 11:11 +0100, Jeroen Massar wrote:
> You are trying to solve the wrong problem with the wrong hammer.

"I need to go the bus station. Do you know the way?"
"If I were you, I wouldn't start from here."

Sorry, but relayd is _exactly_ what can fix this issue, without being a
hammer at all. If you knew something about the political structures of
SWITCH and of UZH you wouldn't recommend "kicking the network
administrator".

CTRL^D

[Jeroen Massar]

Stephan A. Rickauer wrote:
> On Wed, 2008-12-17 at 11:11 +0100, Jeroen Massar wrote:
>> You are trying to solve the wrong problem with the wrong hammer.
>
> "I need to go the bus station. Do you know the way?"
> "If I were you, I wouldn't start from here."
>
> Sorry, but relayd is _exactly_ what can fix this issue, without being a
> hammer at all.

It is a hammer, as you can just NAT IPv4 or use a HTTP proxy. Solves
your problem without doing difficult. IPv4 NAT is a very well understood
 concept that can be configured easily and works for most situations.

What you are trying to do now is not a well understood concept and has
not been used in a lot of situations except for some experimental
setups, mostly by the people who used the tools, who generally then
dropped it because of all the hassle it causes.

> If you knew something about the political structures of
> SWITCH and of UZH you wouldn't recommend "kicking the network
> administrator".

If you would have read the rest of the message you might have noticed
that I said you have an L7 issue, and therefor I still recommend kicking
them, very hard, because they are not doing their jobs properly by
assigning you address space that you can't use, unless that is their job
of course.

But enjoy the next few weeks trying to get something to work that will
not do what you actually want it to do, while the rest of the kids are
nicely going to go outside and play in the snow ;)

Greets,
 Jeroen
  (who just loves snow and is enjoying every second of it ;)

[demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]

[Diana Eichert]

On Wed, 17 Dec 2008, Jeroen Massar wrote:

> Stephan A. Rickauer wrote:
>>
>> If you knew something about the political structures of
>> SWITCH and of UZH you wouldn't recommend "kicking the network
>> administrator".
>
> If you would have read the rest of the message you might have noticed
> that I said you have an L7 issue, and therefor I still recommend kicking
> them, very hard, because they are not doing their jobs properly by
> assigning you address space that you can't use, unless that is their job
> of course.

First to the OP issue.  Why can't you internally NAT IPv4 hosts?

Now we're venturing OT.  Don't blame the Network Admins, there are
historic reasons why lots of IP space isn't used.  We have several legacy
/16 assigned at my day job work place.  Back in the day we distributed
/24 to internal orgs as they requested them.  Some of these /24 are
barely utilized, but the in service IPs are not sequentially located on
the /24.  We can't just tell these people to compress their IP space down
to a /25, /26, /27 or /28 and return the rest, because they talk to
external hosts running apps that are not easily modified.  Besides of
course the internal politics of telling people in other orgs how to run
their business.

diana

[Stephan A. Rickauer]

On Wed, 2008-12-17 at 06:54 -0700, Diana Eichert wrote:

> On Wed, 17 Dec 2008, Jeroen Massar wrote:
>
> > Stephan A. Rickauer wrote:
> >>
> >> If you knew something about the political structures of
> >> SWITCH and of UZH you wouldn't recommend "kicking the network
> >> administrator".
> >
> > If you would have read the rest of the message you might have
> noticed
> > that I said you have an L7 issue, and therefor I still recommend
> kicking
> > them, very hard, because they are not doing their jobs properly by
> > assigning you address space that you can't use, unless that is their
> job
> > of course.
>
> First to the OP issue.  Why can't you internally NAT IPv4 hosts?

We already do so with two ranges. We could just go for another one
(which we would have to apply for, wait weeks, argue to people, fill out
forms, bow for the masters and whatnot). True. But on the one hand I
wanted to have a real-world example on how to make an ipv6-only client
work and on the other hand we already got the ipv6 ranges assigned.

If it worked we could make dedicated machines ipv6-only, save some ipv4
IPs and could go like that for another year or two - until University
finally decides to route ipv6 traffic...

Reyks very nice article on undeadly motivated me to play with it and see
how it goes. It looked just obvious to me, and actually I learned a lot
already.

1. internal ipv6 network works (client -> gateway)
2. DNS for client works, including ipv4 mapping
3. pf does redirect inet6 traffic to relayd
4. relayd doesn't see the redirected traffic.

So from my understanding, there must be some glitch why properly rdr'd
inet6 traffic to relayd doesn't arrive there. That's the last issue - I
can imagine that this is the last step to make my client work with tcp6.

--

 Stephan A. Rickauer

 -----------------------------------------------------------
 Institute of Neuroinformatics         Tel  [phone]
 University / ETH Zurich               Sec  [phone]
 Winterthurerstrasse 190               Fax  [phone]
 CH-8057 Zurich                        Web    www.ini.uzh.ch

[Todd T. Fries-2]

Penned by Stephan A. Rickauer on 20081216 16:14.32, we have:
| I started playing with ipv6. It feels like back in the early 90's, when
| I had to learn how 'the Internet' works ;)

Yes, I recall sitting in a basement with friends around that time, deciding
with enough parts and computers we would learn how to setup an IPv4 network..

Lots of trials and errors later...
 
| Here's the setup:
|
| An ipv6 only host with a non-link-local address should be able to use
| the ipv4 world. I don't want to deal with a tunnel broker, nor do I have
| native ipv6 access to the internet.

Your assumption is invalid.  If you have a non link local address you have,
by definition, a global IPv6 address, in which case someone (a tunnel broker
or a native provider) has allocated you an IPv6 address.  If you can get to
anywhere in the IPv6 world, you can use places like:

        http://<domain>.sixxs.org/

to visit IPv4 hosts via a free IPv6 `proxy' setup by sixxs.net.  Note it
doesn't just spit out pages verbatim, but tries to tweak them to refer back
to the proxy; most of the time it succeeds.

| The ipv6 only client gets its ipv6 address via the rtadvd running on the
| gatway's internal interface. The gateway's external interface is ipv4
| only.

So however you've managed it you have an IPv6 subnet internally. But it is
not routed to the world?  Shame.  Go get a tunnel broker and fix this!  You
really are missing out..
 
| The ipv6 host can already ping6 the gatway. DNS I have 'fixed' with
| totd, so ipv4 addressed are mapped into the ipv6 space:
|
| ipv6-client:~$ host www.google.ch
| www.l.google.com has address 74.125.39.147
| www.l.google.com has IPv6 address 2001:620:10:1401::4a7d:2767
|
|
| The default ipv6-gateway of my ipv6 client is properly set
| in /etc/mygate.
|
| I try to use pf on the gateway to intercept tcp/ip6 traffic and to feed
| it into relayd. The relevant parts are as follows:
|
| ---pf.conf--
| rdr pass inet6 proto tcp from lan:network -> :: port 8081
| ---pf.conf--

Wrong.  Try this instead:

 rdr pass inet6 proto tcp from lan:network -> lan port 8081

You cannot redirect to `::', a wildcard address.  You must redirect to
a specific address.
 
| ---relayd.conf---
| tcp protocol tcpgeneric {
|         tcp { backlog 128, nodelay, sack, socket buffer 131072 }
| }
|
| relay tcp6to4 {
|         listen on :: port 8081
|         forward to nat lookup inet
|         protocol tcpgeneric
| }
| ---relayd.conf---

This relayd.conf looks like what I've done.  here is my setup on the gateway
with a couple little twists (I'm using abcd::/48 as an example allocation):

 -- pf.conf --
 table <6to4ok> { abcd::/48 } # who is permitted to use this relay?
 table <6to4net> { abcd:0:0:ffff::/96 } # the 6to4 prefix

 rdr pass inet6 proto tcp from <6to4ok> to <6to4net> port { 80 8080 } -> abcd::1 port 8080
 rdr pass inet6 proto tcp from <6to4ok> to <6to4net> -> abcd::1 port 8081
 -- pf.conf --

 -- relayd.conf --
 tcp protocol tcpgeneric {
        tcp { backlog 128, nodelay, sack, socket buffer 131072 }
 }
 http protocol httpgeneric {
        header append "$REMOTE_ADDR" to "X-Forwarded-For"
        header append "$SERVER_ADDR:$SERVER_PORT" to \
                "X-Forwarded-By"
        header change "Connection" to "close"

        tcp { backlog 128, nodelay, sack, socket buffer 131072 }

 }
 relay tcp6to4 {
        listen on :: port 8081
        forward to nat lookup inet
        protocol tcpgeneric
 }
 relay http6to4 {
        listen on :: port 8080
        forward to nat lookup inet
        protocol httpgeneric
 }
 -- relayd.conf --

.. this way http traffic gets some info injected about being forwarded.

| After that kinda long intro, here's the problem:
|
| Though name resolution works, an actual connection to an ipv6 address on
| port 80 wouldn't work and isn't 'seen' by relayd either. If I tcpdump on
| the gateway I see that the client, after it got the faked ipv6 address,
| sends an "icmp6: neighbor sol: who has 2001:620:10:1401::4a7d:2767".
|
| So, it believes google is part of 'our' name space, which is probably
| wrong. I then tried to change the prefix of totd to a non-local prefix,
| like 2001:620:10:1400:: (instead of :1401::) so that a 'host
| www.google.ch' results in 2001:620:10:1400::4a7d:2767 and thus can't be
| treated as 'local'.
|
| When I do this I can see the traffic on the gatway:
| 2001:620:10:1401:20d:60ff:fe2e:251b.13239 >
| 2001:620:10:1400::4a7d:2768.80
|
| but it's still not seen by relayd.
|
| Can someone with some degree of patience shed some light on my dark
| spots?

I think the pf.conf tweak may be all thats necessary for you to see traffic
via relayd.

However, I'll go a step further and document what I was able to accomplish
without totd.

Without totd, just using pf.conf on the client and relayd on the client in
addition to the above that I've already documented on the gateway, I was
able to use IPv4 by sending only native IPv6 packets from the client to
the gateway.

On the client, I did the following in pf.conf:

 #set skip on lo # Important that this is commented!
 scrub in

 no rdr inet proto tcp from 127.0.0.1
 no rdr inet proto tcp to 127.0.0.1

 rdr inet proto tcp tag tcp4to6 -> 127.0.0.1 port 8081

 pass out route-to (lo0 127.0.0.1) inet proto tcp tagged tcp4to6

On the client, I did the following in relayd.conf:

 relay tcp4to6 {
        listen on 127.0.0.1 port 8081
        forward to nat lookup inet6 abcd:0:0:ffff::
 }

If you have no global or rfc internal IPv4 address on the system, you still
need a `default' IPv4 route.  You should be able to do this:

  route add default 127.0.0.1

though I did not need to since I had another interface with an IPv4 address I
already had a default route through.

I generally do IPv4 over IPv6 inside of IPSec, for access to home afs and
other reasons. I've added specific bypass rules to bypass that tunnel to
test the relay stuff. For example:

 me4="10.0.0.206"
 rmt6="abcd::1"
 ike dynamic esp from $me4 to any peer $rmt6 \
        srcid me.example.com dstid rmt.example.com

 flow protocol tcp from 0.0.0.0/0 to 0.0.0.0/0 port 80 type bypass
 flow protocol tcp from 0.0.0.0/0 to 0.0.0.0/0 port 22 type bypass

 flow from $me4 to $me4 type bypass

As a true tangent but relevent to those roaming around on the ancient IPv4
network wanting to have a fixed IPv6 address .. I submit the following (because
the bypass rules are required and took a bit to realize why.. ndp traffic over
IPSec is not a useful waste of bandwith..):

 me6="abcd:42::feed:8ee"
 rmt4="1.2.3.4"
 ike dynamic esp from $me6 to any peer $rmt4 \
        srcid me.example.com dstid rmt.example.com

 flow from $me6 to $me6 type bypass

 flow from ::/0 to ff02::/16 type bypass
 flow out from ff02::/16 to ::/0 type bypass
 flow from ::/0 to fe80::/16 type bypass
 flow out from fe80::/16 to ::/0 type bypass

For both of the above IPSec examples, $me6 and $me4 are on trunk1 with no
trunkports but the interface is up.  For IPv4, 'route add default $me4' works.
For IPv6, 'route add -inet6 default $me6' works.  Why?  Because traffic without
a destination in the routing table does not make it to the IPSec stack.. so
some route (any route) must be present for traffic to flow.. at which point
IPSec can inspect and do its thing if the situation merits.

Hope this provides some useful pointers!
--
Todd Fries .. todd@...

 _____________________________________________
|                                             \  1.636.410.0632 (voice)
| Free Daemon Consulting, LLC                 \  1.405.227.9094 (voice)
| http://FreeDaemonConsulting.com             \  1.866.792.3418 (FAX)
| "..in support of free software solutions."  \          250797 (FWD)
|                                             \
 \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
                                                 
              37E7 D3EB 74D0 8D66 A68D  B866 0326 204E 3F42 004A
                        http://todd.fries.net/pgp.txt

[Stephan A. Rickauer]

Thanks a lot for your help, Todd.

On Wed, 2008-12-17 at 13:01 -0600, Todd T. Fries wrote:
> | The ipv6 only client gets its ipv6 address via the rtadvd running on the
> | gatway's internal interface. The gateway's external interface is ipv4
> | only.
>
> So however you've managed it you have an IPv6 subnet internally. But it is
> not routed to the world?  Shame.  Go get a tunnel broker and fix this!  You
> really are missing out..

Yep, University gave us five ipv6 ranges without being able to route
them (yet).
 

> | The ipv6 host can already ping6 the gatway. DNS I have 'fixed' with
> | totd, so ipv4 addressed are mapped into the ipv6 space:
> |
> | ipv6-client:~$ host www.google.ch
> | www.l.google.com has address 74.125.39.147
> | www.l.google.com has IPv6 address 2001:620:10:1401::4a7d:2767
> |
> |
> | The default ipv6-gateway of my ipv6 client is properly set
> | in /etc/mygate.
> |
> | I try to use pf on the gateway to intercept tcp/ip6 traffic and to feed
> | it into relayd. The relevant parts are as follows:
> |
> | ---pf.conf--
> | rdr pass inet6 proto tcp from lan:network -> :: port 8081
> | ---pf.conf--
>
> Wrong.  Try this instead:
>
>  rdr pass inet6 proto tcp from lan:network -> lan port 8081

> You cannot redirect to `::', a wildcard address.  You must redirect to
> a specific address.

Oh, yes. This is wrong indeed. I wonder why pfctl hasn't bailed out.
However, using "-> ::1" should then do the trick as well, right?

Unfortunately, I still see the same effect, here are the packets on
doing an 'ssh my.external.ipv4.host' from my ipv6-client:

gw# tcpdump -evni pflog0 -s 512 ip6

10:44:55.701935 rule 32/(match) [uid 0, pid 28859] pass in on em0:
2001:620:10:1401:20d:60ff:fe2e:251b.27021 > 2001:620:10:1401::eeee.53:
42719+ AAAA? merry.ini.uzh.ch. (34) [flowlabel 0xbc4e3] (len 42, hlim
64)

10:44:55.710561 rule 32/(match) [uid 0, pid 28859] pass in on em0:
2001:620:10:1401:20d:60ff:fe2e:251b.21304 > 2001:620:10:1401::eeee.53:
61177+ A? merry.ini.uzh.ch. (34) [flowlabel 0xdcf20] (len 42, hlim 64)

10:44:55.717571 rule 11/(match) [uid 0, pid 28859] rdr in on em0:
2001:620:10:1401:20d:60ff:fe2e:251b.37356 > ::1.8081: S
3170155212:3170155212(0) win 16384 <mss 1440,nop,nop,sackOK,nop,wscale
0,nop,nop,timestamp 1991902115 0> [flowlabel 0xc4399] (len 44, hlim 64)
 
So, the traffic *is* redirected to ::1.8081 but the client connection
times out after a while and the relayd log doesn't show anything.

However, if I test with 'telnet ::1 8081', I do see a connection attempt
in relayd's log file.

> | ---relayd.conf---
> | tcp protocol tcpgeneric {
> |         tcp { backlog 128, nodelay, sack, socket buffer 131072 }
> | }
> |
> | relay tcp6to4 {
> |         listen on :: port 8081
> |         forward to nat lookup inet
> |         protocol tcpgeneric
> | }
> | ---relayd.conf---
>
> This relayd.conf looks like what I've done.  here is my setup on the gateway
> with a couple little twists (I'm using abcd::/48 as an example allocation):
>
>  -- pf.conf --
>  table <6to4ok> { abcd::/48 } # who is permitted to use this relay?
>  table <6to4net> { abcd:0:0:ffff::/96 } # the 6to4 prefix
>
>  rdr pass inet6 proto tcp from <6to4ok> to <6to4net> port { 80 8080 } -> abcd::1 port 8080
>  rdr pass inet6 proto tcp from <6to4ok> to <6to4net> -> abcd::1 port 8081
>  -- pf.conf --
>
>  -- relayd.conf --
>  tcp protocol tcpgeneric {
>         tcp { backlog 128, nodelay, sack, socket buffer 131072 }
>  }
>  http protocol httpgeneric {
>         header append "$REMOTE_ADDR" to "X-Forwarded-For"
>         header append "$SERVER_ADDR:$SERVER_PORT" to \
>                 "X-Forwarded-By"
>         header change "Connection" to "close"
>
>         tcp { backlog 128, nodelay, sack, socket buffer 131072 }
>
>  }
>  relay tcp6to4 {
>         listen on :: port 8081
>         forward to nat lookup inet
>         protocol tcpgeneric
>  }
>  relay http6to4 {
>         listen on :: port 8080
>         forward to nat lookup inet
>         protocol httpgeneric
>  }
>  -- relayd.conf --
>
> .. this way http traffic gets some info injected about being forwarded.

I will take care of http as soon as the basic setup works.

> | After that kinda long intro, here's the problem:
> |
> | Though name resolution works, an actual connection to an ipv6 address on
> | port 80 wouldn't work and isn't 'seen' by relayd either. If I tcpdump on
> | the gateway I see that the client, after it got the faked ipv6 address,
> | sends an "icmp6: neighbor sol: who has 2001:620:10:1401::4a7d:2767".
> |
> | So, it believes google is part of 'our' name space, which is probably
> | wrong. I then tried to change the prefix of totd to a non-local prefix,
> | like 2001:620:10:1400:: (instead of :1401::) so that a 'host
> | www.google.ch' results in 2001:620:10:1400::4a7d:2767 and thus can't be
> | treated as 'local'.
> |
> | When I do this I can see the traffic on the gatway:
> | 2001:620:10:1401:20d:60ff:fe2e:251b.13239 >
> | 2001:620:10:1400::4a7d:2768.80
> |
> | but it's still not seen by relayd.
> |
> | Can someone with some degree of patience shed some light on my dark
> | spots?
>
> I think the pf.conf tweak may be all thats necessary for you to see traffic
> via relayd.

Unfortunately, it doesn't. The packets aren't blocked by pf but are
properly redirected to relayd. Relayd stays quiet.

On a side note: I also don't understand why the wrong default gateway is
advertised to my client. Instead of my global IPv6 address, the
local-link address is propagated. I was under the impression rtadvd will
take care of it:

gw$ cat /etc/rtadvd.conf
em0:\
        :addr="2001:620:10:1401::":prefixlen#64:raflags#0:


client$ sudo route -n show -inet6 | grep default
default fe80::20c:f1ff:fe8f:a9c4%em0   UG   0       43      -   em0

client$ cat /etc/mygate
2001:620:10:1401::eeee

> However, I'll go a step further and document what I was able to accomplish
> without totd.
>
> Without totd, just using pf.conf on the client and relayd on the client in
> addition to the above that I've already documented on the gateway, I was
> able to use IPv4 by sending only native IPv6 packets from the client to
> the gateway.
>
> On the client, I did the following in pf.conf:
>
>  #set skip on lo # Important that this is commented!
>  scrub in
>
>  no rdr inet proto tcp from 127.0.0.1
>  no rdr inet proto tcp to 127.0.0.1
>
>  rdr inet proto tcp tag tcp4to6 -> 127.0.0.1 port 8081
>
>  pass out route-to (lo0 127.0.0.1) inet proto tcp tagged tcp4to6
>
> On the client, I did the following in relayd.conf:
>
>  relay tcp4to6 {
>         listen on 127.0.0.1 port 8081
>         forward to nat lookup inet6 abcd:0:0:ffff::
>  }
>
> If you have no global or rfc internal IPv4 address on the system, you still
> need a `default' IPv4 route.  You should be able to do this:
>
>   route add default 127.0.0.1
>
> though I did not need to since I had another interface with an IPv4 address I
> already had a default route through.
>
> I generally do IPv4 over IPv6 inside of IPSec, for access to home afs and
> other reasons. I've added specific bypass rules to bypass that tunnel to
> test the relay stuff. For example:
>
>  me4="10.0.0.206"
>  rmt6="abcd::1"
>  ike dynamic esp from $me4 to any peer $rmt6 \
> srcid me.example.com dstid rmt.example.com
>
>  flow protocol tcp from 0.0.0.0/0 to 0.0.0.0/0 port 80 type bypass
>  flow protocol tcp from 0.0.0.0/0 to 0.0.0.0/0 port 22 type bypass
>
>  flow from $me4 to $me4 type bypass
>
> As a true tangent but relevent to those roaming around on the ancient IPv4
> network wanting to have a fixed IPv6 address .. I submit the following (because
> the bypass rules are required and took a bit to realize why.. ndp traffic over
> IPSec is not a useful waste of bandwith..):
>
>  me6="abcd:42::feed:8ee"
>  rmt4="1.2.3.4"
>  ike dynamic esp from $me6 to any peer $rmt4 \
> srcid me.example.com dstid rmt.example.com
>
>  flow from $me6 to $me6 type bypass
>
>  flow from ::/0 to ff02::/16 type bypass
>  flow out from ff02::/16 to ::/0 type bypass
>  flow from ::/0 to fe80::/16 type bypass
>  flow out from fe80::/16 to ::/0 type bypass
>
> For both of the above IPSec examples, $me6 and $me4 are on trunk1 with no
> trunkports but the interface is up.  For IPv4, 'route add default $me4' works.
> For IPv6, 'route add -inet6 default $me6' works.  Why?  Because traffic without
> a destination in the routing table does not make it to the IPSec stack.. so
> some route (any route) must be present for traffic to flow.. at which point
> IPSec can inspect and do its thing if the situation merits.
>
> Hope this provides some useful pointers!

Well, at least my pf.conf is fixed now! Thanks again. But I still
struggle with relayd. I'll try to setup this case at home on my much
simpler environment over christmess. Maybe that'll work.

Cheers,

--

 Stephan A. Rickauer

 -----------------------------------------------------------
 Institute of Neuroinformatics         Tel  [phone]
 University / ETH Zurich               Sec  [phone]
 Winterthurerstrasse 190               Fax  [phone]
 CH-8057 Zurich                        Web    www.ini.uzh.ch

[Todd T. Fries-2]

Penned by Stephan A. Rickauer on 20081219 11:01.16, we have:
| Thanks a lot for your help, Todd.
|
| On Wed, 2008-12-17 at 13:01 -0600, Todd T. Fries wrote:
| > | The ipv6 only client gets its ipv6 address via the rtadvd running on the
| > | gatway's internal interface. The gateway's external interface is ipv4
| > | only.
| >
| > So however you've managed it you have an IPv6 subnet internally. But it is
| > not routed to the world?  Shame.  Go get a tunnel broker and fix this!  You
| > really are missing out..
|
| Yep, University gave us five ipv6 ranges without being able to route
| them (yet).

Yecht.  *sigh*.  Hopefully this changes ;-)
 
| > | The ipv6 host can already ping6 the gatway. DNS I have 'fixed' with
| > | totd, so ipv4 addressed are mapped into the ipv6 space:
| > |
| > | ipv6-client:~$ host www.google.ch
| > | www.l.google.com has address 74.125.39.147
| > | www.l.google.com has IPv6 address 2001:620:10:1401::4a7d:2767
| > |
| > |
| > | The default ipv6-gateway of my ipv6 client is properly set
| > | in /etc/mygate.
| > |
| > | I try to use pf on the gateway to intercept tcp/ip6 traffic and to feed
| > | it into relayd. The relevant parts are as follows:
| > |
| > | ---pf.conf--
| > | rdr pass inet6 proto tcp from lan:network -> :: port 8081
| > | ---pf.conf--
| >
| > Wrong.  Try this instead:
| >
| >  rdr pass inet6 proto tcp from lan:network -> lan port 8081
|
| > You cannot redirect to `::', a wildcard address.  You must redirect to
| > a specific address.
|
| Oh, yes. This is wrong indeed. I wonder why pfctl hasn't bailed out.
| However, using "-> ::1" should then do the trick as well, right?

Sorry I was not clear.  With IPv6, unlike IPv4, it is not possible to
redirect to `localhost'.  You must redirect to a global scope address.

[..]
| > .. this way http traffic gets some info injected about being forwarded.
|
| I will take care of http as soon as the basic setup works.

Sure.

[..]
| > I think the pf.conf tweak may be all thats necessary for you to see traffic
| > via relayd.
|
| Unfortunately, it doesn't. The packets aren't blocked by pf but are
| properly redirected to relayd. Relayd stays quiet.
|
| On a side note: I also don't understand why the wrong default gateway is
| advertised to my client. Instead of my global IPv6 address, the
| local-link address is propagated. I was under the impression rtadvd will
| take care of it:
|
| gw$ cat /etc/rtadvd.conf
| em0:\
|         :addr="2001:620:10:1401::":prefixlen#64:raflags#0:

You have a wrong understanding of IPv6.  It is recommended to use the link
local address for the router(s) since they will always be link local.  Routing
to a global scope address is a last choice.  Don't over-ride the defaults
here, you have no good reason to.
 
| client$ sudo route -n show -inet6 | grep default
| default fe80::20c:f1ff:fe8f:a9c4%em0   UG   0       43      -   em0
|
| client$ cat /etc/mygate
| 2001:620:10:1401::eeee

Choose one or the other.  You either need a default route in /etc/mygate
and a static IP for the client or you need rtsol(d).

One trick I picked up from ISC is if you want your client to be '::eeee' then
set this in the hostname.if file:

 inet6 fe80::eeee
 rtsol
 
.. and you'll get global scope addresses on that host that end in ::eeee.

[..]
| > Hope this provides some useful pointers!
|
| Well, at least my pf.conf is fixed now! Thanks again. But I still
| struggle with relayd. I'll try to setup this case at home on my much
| simpler environment over christmess. Maybe that'll work.

I'm still convinced the pf.conf is the problem, redirect to a global scope
IPv6 address and I suspect you'll be much better off.

Thanks,
--
Todd Fries .. todd@...

 _____________________________________________
|                                             \  1.636.410.0632 (voice)
| Free Daemon Consulting, LLC                 \  1.405.227.9094 (voice)
| http://FreeDaemonConsulting.com             \  1.866.792.3418 (FAX)
| "..in support of free software solutions."  \          250797 (FWD)
|                                             \
 \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
                                                 
              37E7 D3EB 74D0 8D66 A68D  B866 0326 204E 3F42 004A
                        http://todd.fries.net/pgp.txt

[Stephan A. Rickauer]

On Fri, 2008-12-19 at 08:13 -0600, todd@... wrote:

> | > Wrong.  Try this instead:
> | >
> | >  rdr pass inet6 proto tcp from lan:network -> lan port 8081
> |
> | > You cannot redirect to `::', a wildcard address.  You must redirect to
> | > a specific address.
> |
> | Oh, yes. This is wrong indeed. I wonder why pfctl hasn't bailed out.
> | However, using "-> ::1" should then do the trick as well, right?
>
> Sorry I was not clear.  With IPv6, unlike IPv4, it is not possible to
> redirect to `localhost'.  You must redirect to a global scope address.

You Made My Week.

It's working. Awesome. Thanks a lot!
This is a good example where ipv4 concepts can't be applied to the ipv6
world in a straight forward way. I would not have thought that it's no
longer possible to redirect to localhost, ever. Well, time to put
localhost off the network then ;)

> | On a side note: I also don't understand why the wrong default gateway is
> | advertised to my client. Instead of my global IPv6 address, the
> | local-link address is propagated. I was under the impression rtadvd will
> | take care of it:
> |
> | gw$ cat /etc/rtadvd.conf
> | em0:\
> |         :addr="2001:620:10:1401::":prefixlen#64:raflags#0:
>
> You have a wrong understanding of IPv6.  

Will be fixed soon. O'Reillys "IPv6 Network Administration" has been
downloaded today...

> >It is recommended to use the link
> local address for the router(s) since they will always be link local.  Routing
> to a global scope address is a last choice.  Don't over-ride the defaults
> here, you have no good reason to.
>  
> | client$ sudo route -n show -inet6 | grep default
> | default fe80::20c:f1ff:fe8f:a9c4%em0   UG   0       43      -   em0
> |
> | client$ cat /etc/mygate
> | 2001:620:10:1401::eeee
>
> Choose one or the other.  You either need a default route in /etc/mygate
> and a static IP for the client or you need rtsol(d).
>
> One trick I picked up from ISC is if you want your client to be '::eeee' then
> set this in the hostname.if file:
>
>  inet6 fe80::eeee
>  rtsol

> .. and you'll get global scope addresses on that host that end in ::eeee.

Cool. Will try that.


> I'm still convinced the pf.conf is the problem, redirect to a global scope
> IPv6 address and I suspect you'll be much better off.

Yes, that fixed it. Thanks again.
I owe you a pint, well ... two pints ;)

Cheers,

--

 Stephan A. Rickauer

 -----------------------------------------------------------
 Institute of Neuroinformatics         Tel  [phone]
 University / ETH Zurich               Sec  [phone]
 Winterthurerstrasse 190               Fax  [phone]
 CH-8057 Zurich                        Web    www.ini.uzh.ch

[Todd T. Fries-2]

Penned by Stephan A. Rickauer on 20081219 16:44.47, we have:
| On Fri, 2008-12-19 at 08:13 -0600, todd@... wrote:
| > | > Wrong.  Try this instead:
| > | >
| > | >  rdr pass inet6 proto tcp from lan:network -> lan port 8081
| > |
| > | > You cannot redirect to `::', a wildcard address.  You must redirect to
| > | > a specific address.
| > |
| > | Oh, yes. This is wrong indeed. I wonder why pfctl hasn't bailed out.
| > | However, using "-> ::1" should then do the trick as well, right?
| >
| > Sorry I was not clear.  With IPv6, unlike IPv4, it is not possible to
| > redirect to `localhost'.  You must redirect to a global scope address.
|
| You Made My Week.
|
| It's working. Awesome. Thanks a lot!
| This is a good example where ipv4 concepts can't be applied to the ipv6
| world in a straight forward way. I would not have thought that it's no
| longer possible to redirect to localhost, ever. Well, time to put
| localhost off the network then ;)

To be clear, this is due to the way pf rdr and IPv6 interaction works.  Not
sure if 'ever' is the right outlook here.

[..]
| > I'm still convinced the pf.conf is the problem, redirect to a global scope
| > IPv6 address and I suspect you'll be much better off.
|
| Yes, that fixed it. Thanks again.

Welcome.
--
Todd Fries .. todd@...

 _____________________________________________
|                                             \  1.636.410.0632 (voice)
| Free Daemon Consulting, LLC                 \  1.405.227.9094 (voice)
| http://FreeDaemonConsulting.com             \  1.866.792.3418 (FAX)
| "..in support of free software solutions."  \          250797 (FWD)
|                                             \
 \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
                                                 
              37E7 D3EB 74D0 8D66 A68D  B866 0326 204E 3F42 004A
                        http://todd.fries.net/pgp.txt