/dev/tap and tcpdump don't go together very well?

[Rhialto]

For the ethernet access of a PDP-10 emulator I am experimenting with
/dev/tap in addition to the traditional use of /dev/bpf, so that the
local host can see ethernet packets that are destined for it. Packets
sent through /dev/bpf only go to the wire, not into the network input
stream. On the other hand, packets written to /dev/tap do go in (if
there is no matching IP address they will be ignored but that is another
matter).

However, it seems that as soon as I use ``tcpdump -i tap0'' (and hence
bpf) to see the packets flowing through the tap, the packets are only
seen by tcpdump and not anymore by the host anymore. So I get the
situation where tcpdump shows me that "it works" but it doesn't really.

I conclude this from the fact that I can reliably freeze a telnet
connection from the host into the emulator by doing a tcpdump on tap0.
It shows me the returning packets to the host, but the connection
remains frozen. When I stop tcpdump, my typed-in characters appear after
all (no doubt due to TCP retransmission, or conceivably because the
packets have been buffered somewhere but that seems quite unlikely).

I'm using a -current of some time ago (4.99.5).

Does this ring a bell?

-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Greg Troxel]

Please be far more clear about how this is all supposed to work.
Running tcpdump on the host as '-i tap0' will put the tap device in
promiscuous mode.  But packets should still function in both directions.
Try adding -p to not force promiscuous mode.

--
    Greg Troxel <gdt@...>

[Rhialto]

On Sun 08 Apr 2007 at 06:47:14 -0400, Greg Troxel wrote:
> Please be far more clear about how this is all supposed to work.

I wish I could point to some easy example program, but alas. What I'm
using involves a pdp-10 emulator with TOPS-20 (http://panda.trailing-edge.com)
plus some private patches to add /dev/tap to it.
(http://www.falu.nl/~rhialto/klh-diffs should be all, if I did it right)

Is there some system program or possibly something in pkgsrc that I can
try, that uses /dev/tap, so that we can verify independently? Writing a
separate program that injects some packets into /dev/tap isn't so
difficult but I am currently wondering how one would verify that they
arrive at the other end.

> Running tcpdump on the host as '-i tap0' will put the tap device in
> promiscuous mode.  But packets should still function in both directions.

I looked at the source (of course) and I could find no obvious
explanation. That's why I was wondering if perhaps it was some
non-obvious but known problem.

> Try adding -p to not force promiscuous mode.

Interesting. I still see (the? all?) packets with tcpdump, but the
telnet connection still functions. This may be a clue towards finding
the cause.

>     Greg Troxel <gdt@...>
-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Greg Troxel]

So you have an emulator program, and it opens a /dev/tap, and
reads/writes packets and makes those appear to show up/come from the
emulated network interface on the PDP-10?

Then on the host are you bridging /dev/tap0 to a regular ethernet?

I think you are saying that if you then run 'tcpdump -i tap0' on the
host, then packets from host to emulator and emulator to host are
printed by tcpdump.  But, without -p, then the packets from the host
towards the emulator are apparently not received.

I suggest adding a printf to the emulator when reading packets from the
tap device.

In src/sys/dev/if_tap.c, see tap_dev_read.  The call to bpf_tap looks
normal relative to other drivers.

Setting promiscuous mode will call net/if.c:ifpromisc which will call
net/if_ethersubr.c:ether_ioctl which will call tap_init.

tap_init calls tap_start, and that seems to introduce an extra wakeup
and OACTIVE flag, and that seems wrong.  Try removing the tap_start call
from tap_init and rebuilding the host kernel.

[Johnny Billquist-4]

This all reminds me of another thing.
Are there any way, or plans to be able to, change the mac address of an
ethernet interface in NetBSD? I don't think there is, but I might have
missed something.

When doing stuff like Olaf is doing here, changing the MAC address might
be desireable (in his case if he wants his PDP-10 to run TOPS-20 with
DECnet enabled, since DECnet wants specific MAC addresses on interfaces...)

        Johnny

Rhialto wrote:

> On Sun 08 Apr 2007 at 06:47:14 -0400, Greg Troxel wrote:
>
>>Please be far more clear about how this is all supposed to work.
>
>
> I wish I could point to some easy example program, but alas. What I'm
> using involves a pdp-10 emulator with TOPS-20 (http://panda.trailing-edge.com)
> plus some private patches to add /dev/tap to it.
> (http://www.falu.nl/~rhialto/klh-diffs should be all, if I did it right)
>
> Is there some system program or possibly something in pkgsrc that I can
> try, that uses /dev/tap, so that we can verify independently? Writing a
> separate program that injects some packets into /dev/tap isn't so
> difficult but I am currently wondering how one would verify that they
> arrive at the other end.
>
>
>>Running tcpdump on the host as '-i tap0' will put the tap device in
>>promiscuous mode.  But packets should still function in both directions.
>
>
> I looked at the source (of course) and I could find no obvious
> explanation. That's why I was wondering if perhaps it was some
> non-obvious but known problem.
>
>
>>Try adding -p to not force promiscuous mode.
>
>
> Interesting. I still see (the? all?) packets with tcpdump, but the
> telnet connection still functions. This may be a clue towards finding
> the cause.
>
>
>>    Greg Troxel <gdt@...>
>
> -Olaf.

--
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt@...             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[bqt.vcf]

---

begin:vcard
fn:Johnny Billquist
n:Billquist;Johnny
email;internet:bqt@...
tel;cell:[phone]
x-mozilla-html:FALSE
version:2.1
end:vcard

---

[Rhialto]

On Sun 08 Apr 2007 at 09:39:01 -0400, Greg Troxel wrote:
> So you have an emulator program, and it opens a /dev/tap, and
> reads/writes packets and makes those appear to show up/come from the
> emulated network interface on the PDP-10?

Partly. What it was originally doing was to use /dev/bpf to do that.
That works fine if the emulator wants to communicate with other hosts
that are actually on the wire. BPF sees packets that come in from the
wire and optionally (default on; BIOCSSEESENT; apparently undocumented)
the packets that the host sends to the wire. Packets injected into BPF
are sent to the wire only, not to the host.

My intention was to add /dev/tap to this scenario in order to add the
missing part: packets from the emulator to the host.

> Then on the host are you bridging /dev/tap0 to a regular ethernet?

That is another scenario I have been considering, but I rejected it so
far because it is more than a "small change". It would be the better
thing to do in the long run though (assuming that all systems that have
tap also have bridge, otherwise it is not much help)

> I think you are saying that if you then run 'tcpdump -i tap0' on the
> host, then packets from host to emulator and emulator to host are
> printed by tcpdump.  But, without -p, then the packets from the host
> towards the emulator are apparently not received.

Yes. In the emulator -> host direction, that is. There are currently
none in the other.

> I suggest adding a printf to the emulator when reading packets from the
> tap device.

It has full debugging options for both directions, indeed. I could see
packets going out of the emulator, I could see then in /dev/tap
(tcpdump), and yet the telnet connection was "frozen".

> In src/sys/dev/if_tap.c, see tap_dev_read.  The call to bpf_tap looks
> normal relative to other drivers.

Doesn't this placement of bpf_tap() in tap_dev_read() mean that only
packets that are actually read by the program are sent to bpf, rather
than the ones that are sent by the kernel? (I think this is a
side-issue, if one at all).

> Setting promiscuous mode will call net/if.c:ifpromisc which will call
> net/if_ethersubr.c:ether_ioctl which will call tap_init.
>
> tap_init calls tap_start, and that seems to introduce an extra wakeup
> and OACTIVE flag, and that seems wrong.  Try removing the tap_start call
> from tap_init and rebuilding the host kernel.

It seems to make no difference. I see however that OACTIVE is set all
the time (at least when I look), also when it is working.

tap0: flags=8c43<UP,BROADCAST,RUNNING,OACTIVE,SIMPLEX,MULTICAST> mtu 1500
        address: f2:0b:a4:7d:17:04
        media: Ethernet autoselect
        inet 10.0.0.12 netmask 0xffffff00 broadcast 10.0.0.255
        inet6 fe80::f00b:a4ff:fe7d:1704%tap0 prefixlen 64 scopeid 0x4

-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Rhialto]

On Sun 08 Apr 2007 at 16:21:48 +0200, Rhialto wrote:
> On Sun 08 Apr 2007 at 09:39:01 -0400, Greg Troxel wrote:
> > In src/sys/dev/if_tap.c, see tap_dev_read.  The call to bpf_tap looks
> > normal relative to other drivers.
>
> Doesn't this placement of bpf_tap() in tap_dev_read() mean that only
> packets that are actually read by the program are sent to bpf, rather
> than the ones that are sent by the kernel? (I think this is a
> side-issue, if one at all).
...
> It seems to make no difference. I see however that OACTIVE is set all
> the time (at least when I look), also when it is working.
>
> tap0: flags=8c43<UP,BROADCAST,RUNNING,OACTIVE,SIMPLEX,MULTICAST> mtu 1500
> address: f2:0b:a4:7d:17:04
> media: Ethernet autoselect
> inet 10.0.0.12 netmask 0xffffff00 broadcast 10.0.0.255
> inet6 fe80::f00b:a4ff:fe7d:1704%tap0 prefixlen 64 scopeid 0x4

Ah, of course! OACTIVE is on precisely because the emulator never reads
from /dev/tap! This could be a very nasty memory leak, if these packets
get to hang on the queue indefinitely.

I was thinking that (as I expected) there were no packets from the
kernel via tap to the emulator, because none showed up in tcpdump. But
that is only because the emulator doesn't try to read them... This
should be improved somehow, I think.

I can't see in if_tap.c where packets get put on the queue. If the
network code does that itself, should that not also be where bpf_tap()
should be called? (It looks like that is in ifq_enqueue() in
/sys/net/if.c)

-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Quentin Garnier]

On Sun, Apr 08, 2007 at 06:47:14AM -0400, Greg Troxel wrote:
> Please be far more clear about how this is all supposed to work.
> Running tcpdump on the host as '-i tap0' will put the tap device in
> promiscuous mode.  But packets should still function in both directions.
> Try adding -p to not force promiscuous mode.

That's strange.  A tap(4) interface is, by essence, always in
promiscuous mode (it doesn't filter what gets written to the device).

--
Quentin Garnier - cube@... - cube@...
"You could have made it, spitting out benchmarks
Owe it to yourself not to fail"
Amplifico, Spitting Out Benchmarks, Hometakes Vol. 2, 2005.

[Rhialto]

I think I found it. When packets come in from a promiscuous interface,
they are given more scrutiny, and my packets fail. See in
/sys/net/if_ethersubr.c, in ether_input():

724 if ((m->m_flags & (M_BCAST|M_MCAST)) == 0 &&
            (ifp->if_flags & IFF_PROMISC) != 0 &&
            memcmp(LLADDR(ifp->if_sadl), eh->ether_dhost,
               ETHER_ADDR_LEN) != 0) {
            m->m_flags |= M_PROMISC;
   
During the problematic tcpdump, IFF_PROMISC is indeed set, and the
destination ethernet address is that of the physical interface, not the
tap one (I send the same packets via tun as I would via bpf, after all).
So M_PROMISC gets set.

Further down,

775 switch (etype) {
...
855 default:
        if (m->m_flags & M_PROMISC) {
            m_freem(m);
            return;
        }

it is dropped.

So not only do I need to clone the IP address and netmask from the
physical interface, I need to clone the ethernet address as well
(SIOCSIFPHYADDR). And, that works. So thanks Greg for suggesting the
promiscuous angle!

-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Rhialto]

Possible conclusions from this thead:

- It may make sense to allow opening /dev/tap write-only. All packets
  written from the kernel side should be dropped (and sent to bpf).
  See attached diff. That plus opening /dev/tap O_WRONLY keeps the
  OACTIVE off and tcpdump sees some icmp6 packets that apparently get
  sent.

- Maybe tap should silently never set the IFF_PROMISC flag since it
  already is, and setting it has unexpected side effects.
  See diff, but untested.

- Other (hardware) interfaces should be able to change their MAC address
  too.

Index: if_tap.c
===================================================================
RCS file: /cvsroot/src/sys/net/if_tap.c,v
retrieving revision 1.24
diff -u -u -r1.24 if_tap.c
--- if_tap.c 24 Nov 2006 01:04:30 -0000 1.24
+++ if_tap.c 8 Apr 2007 19:42:42 -0000
@@ -108,6 +108,7 @@
 #define TAP_ASYNCIO 0x00000002 /* user is using async I/O (SIGIO) on the device */
 #define TAP_NBIO 0x00000004 /* user wants calls to avoid blocking */
 #define TAP_GOING 0x00000008 /* interface is being destroyed */
+#define TAP_WRITEONLY 0x00000010 /* no packets from kernel side */
  struct selinfo sc_rsel;
  pid_t sc_pgid; /* For async. IO */
  struct lock sc_rdlock;
@@ -157,7 +158,7 @@
 };
 
 /* Helper for cloning open() */
-static int tap_dev_cloner(struct lwp *);
+static int tap_dev_cloner(struct lwp *, int);
 
 /* Character device routines */
 static int tap_cdev_open(dev_t, int, int, struct lwp *);
@@ -451,7 +452,8 @@
  struct tap_softc *sc = (struct tap_softc *)ifp->if_softc;
  struct mbuf *m0;
 
- if ((sc->sc_flags & TAP_INUSE) == 0) {
+ if ((sc->sc_flags & TAP_INUSE) == 0 ||
+    (sc->sc_flags & TAP_WRITEONLY) != 0) {
  /* Simply drop packets */
  for(;;) {
  IFQ_DEQUEUE(&ifp->if_snd, m0);
@@ -502,6 +504,10 @@
  case SIOCSIFPHYADDR:
  error = tap_lifaddr(ifp, cmd, (struct ifaliasreq *)data);
  break;
+ case SIOCSIFFLAGS:
+ ifp->if_flags &= ~IFF_PROMISC;
+ error = 0;
+ break;
  default:
  error = ether_ioctl(ifp, cmd, data);
  if (error == ENETRESET)
@@ -541,7 +547,7 @@
 {
  ifp->if_flags |= IFF_RUNNING;
 
- tap_start(ifp);
+ /*tap_start(ifp);*/
 
  return (0);
 }
@@ -664,7 +670,7 @@
  struct tap_softc *sc;
 
  if (minor(dev) == TAP_CLONER)
- return tap_dev_cloner(l);
+ return tap_dev_cloner(l, flags);
 
  sc = (struct tap_softc *)device_lookup(&tap_cd, minor(dev));
  if (sc == NULL)
@@ -674,6 +680,8 @@
  if (sc->sc_flags & TAP_INUSE)
  return (EBUSY);
  sc->sc_flags |= TAP_INUSE;
+ if ((flags & FREAD) == 0)
+    sc->sc_flags |= TAP_WRITEONLY;
  return (0);
 }
 
@@ -700,7 +708,7 @@
  */
 
 static int
-tap_dev_cloner(struct lwp *l)
+tap_dev_cloner(struct lwp *l, int flags)
 {
  struct tap_softc *sc;
  struct file *fp;
@@ -716,8 +724,10 @@
  }
 
  sc->sc_flags |= TAP_INUSE;
+ if ((flags & FREAD) == 0)
+    sc->sc_flags |= TAP_WRITEONLY;
 
- return fdclone(l, fp, fd, FREAD|FWRITE, &tap_fileops,
+ return fdclone(l, fp, fd, flags & (FREAD|FWRITE), &tap_fileops,
     (void *)(intptr_t)device_unit(&sc->sc_dev));
 }


Index: tap.4
===================================================================
RCS file: /cvsroot/src/share/man/man4/tap.4,v
retrieving revision 1.6
diff -u -u -r1.6 tap.4
--- tap.4 18 Dec 2006 00:16:10 -0000 1.6
+++ tap.4 8 Apr 2007 19:59:16 -0000
@@ -125,6 +125,21 @@
 buffer.
 If the buffer is not large enough, the frame will be truncated.
 .Pp
+If you don't intend to read frames from the
+.Nm
+interface, open the special file with access mode O_WRONLY.
+In that case, frames sent by the kernel are immediately discarded
+rather than queued indefinitely.
+.Pp
+If you use
+.Xr bpf 4 ,
+for instance through
+.Xr tcpdump 8 ,
+frames from the kernel side to the userland side are produced when
+they are received, not when they are sent.
+They are also produced when discarded from the interface queue,
+either because the device is destroyed or because it was opened O_WRONLY.
+.Pp
 .Nm
 character devices support the
 .Dv FIONREAD
 
-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Quentin Garnier]

On Sun, Apr 08, 2007 at 10:08:47PM +0200, Rhialto wrote:
>
> Possible conclusions from this thead:
>
> - It may make sense to allow opening /dev/tap write-only. All packets
>   written from the kernel side should be dropped (and sent to bpf).
>   See attached diff. That plus opening /dev/tap O_WRONLY keeps the
>   OACTIVE off and tcpdump sees some icmp6 packets that apparently get
>   sent.

I'm not sure it actually makes sense to do that.  The philosophy of
tap(4) is to have a virtual Ethernet device.  If it is opened by a
process, then said process is the backend of the interface;  it's up to
it to tell whether it wants frames or not, I don't see why the kernel
would have to outsmart it.  Granted, explicitely opening the device only
for writing can be considered enough of a hint, but it still feels a bit
hackish.

If you're going to use bpf(4) to read packets coming "out" the
interface, then why not using bpf(4) to write "incoming" frames?

In that case, the API that you're missing is something that does "create
a tap interface and tell me its unit, but don't open it".  Or, simpler,
a way of closing /dev/tap (lack of unit number is important) without
destroying the device.

> - Maybe tap should silently never set the IFF_PROMISC flag since it
>   already is, and setting it has unexpected side effects.
>   See diff, but untested.

The fact that it doesn't affect its operation doesn't mean it will
always be the case;  you could imagine having a filter in the write
handler...  It might actully provide a way to test multicast stuff or
whatnot.  The initial idea is that for all intents and purposes, it is
an Ethernet device.

> - Other (hardware) interfaces should be able to change their MAC address
>   too.

That's a different issue.  I agree that the way it is done in tap(4) is
far from ideal.

--
Quentin Garnier - cube@... - cube@...
"You could have made it, spitting out benchmarks
Owe it to yourself not to fail"
Amplifico, Spitting Out Benchmarks, Hometakes Vol. 2, 2005.

[Rhialto]

On Sun 08 Apr 2007 at 22:39:27 +0200, Quentin Garnier wrote:
> I'm not sure it actually makes sense to do that.  The philosophy of
> tap(4) is to have a virtual Ethernet device.  If it is opened by a
> process, then said process is the backend of the interface;  it's up to
> it to tell whether it wants frames or not, I don't see why the kernel
> would have to outsmart it.  Granted, explicitely opening the device only
> for writing can be considered enough of a hint, but it still feels a bit
> hackish.

I agree that there should be a better way for unclaimed packets to be
dropped eventually, but I'm not sure what it would be. This was my first
idea. At least it avoids complexity like using timers etc.

> If you're going to use bpf(4) to read packets coming "out" the
> interface, then why not using bpf(4) to write "incoming" frames?

The problem with sending frames through bpf(4)[1] is that although they
get sent out on the wire, the local host does not see them.
Asymmetrically, packets that are sent by the local host *are* copied
into bpf(4) (although this is undocumentedly optional, but default on).
In a way this makes sense, since, being a packet filter it is more meant
to see packets that are on the wire and sending packets out does not
really fit in its philosophy.

But this gives the weird effect that the emulator can successfully
communicate with the whole world, *execpt* the host it is running on,
and that in one direction only.

So to fill this 1/4 gap, I added some optional code to KLH to use tap(4)
when available. I might bite the bullet and add proper tap(4) +
bridge(4) code, but it would be nice ("orthogonal") if corner cases like
this would work too.

On the other hand, bpf(4) could get fixed too. I looked at it briefly,
but ether_input() consumes the packet so it was not just a matter of
adding one call. Probably needs a 2 calls: a copy too.

Whatever the fix will turn out to be, SIMH (used for VAX emulation) will
also benefit from it, and hence the VAX port.

[1] and in this emulator, like others I have seen, it attaches bpf(4) to
the real interface of course, not to the tap(4).

> In that case, the API that you're missing is something that does "create
> a tap interface and tell me its unit, but don't open it".  Or, simpler,
> a way of closing /dev/tap (lack of unit number is important) without
> destroying the device.

For sockets there is shutdown(2), maybe the idea could be extended to
other types of file descriptors.

> > - Maybe tap should silently never set the IFF_PROMISC flag since it
> >   already is, and setting it has unexpected side effects.
> >   See diff, but untested.
>
> The fact that it doesn't affect its operation doesn't mean it will
> always be the case;  you could imagine having a filter in the write
> handler...  It might actully provide a way to test multicast stuff or
> whatnot.  The initial idea is that for all intents and purposes, it is
> an Ethernet device.

In such a case, the manual should also mention things like size checks,
which are currently explicitly not done.

> Quentin Garnier - cube@... - cube@...
-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Greg Troxel]

Johnny Billquist <bqt@...> writes:

> This all reminds me of another thing.
> Are there any way, or plans to be able to, change the mac address of
> an ethernet interface in NetBSD? I don't think there is, but I might
> have missed something.

I think someone just needs to write device-indenpendent support and then
modify various drivers.

> When doing stuff like Olaf is doing here, changing the MAC address
> might be desireable (in his case if he wants his PDP-10 to run TOPS-20
> with DECnet enabled, since DECnet wants specific MAC addresses on
> interfaces...)

I don't think the tap's Ethernet address is relevant.  The PDP-10's
emulated Ethernet interface (how modern to plug one of those in, instead
of CHAOSnet or an IMP....) will have an address, but that's controlled
entirely within the emulator.  Packets with that source address are
generated, and then written to /dev/tapN to make them show up on the
virtual interface on the host.

[Rhialto]

On Mon 09 Apr 2007 at 10:42:42 -0400, Greg Troxel wrote:
> I don't think the tap's Ethernet address is relevant.  The PDP-10's
> emulated Ethernet interface (how modern to plug one of those in, instead
> of CHAOSnet or an IMP....) will have an address, but that's controlled

I think ITS can use an IMP (it's also emulated for such a
configuration). TOPS-10 and -20 get configurations with a NI20. If they
could even use an IMP, I don't know.

-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Perry E. Metzger]

Johnny Billquist <bqt@...> writes:
> Are there any way, or plans to be able to, change the mac address of
> an ethernet interface in NetBSD? I don't think there is, but I might
> have missed something.

Someone posted patches a long time ago to do that (adding an ioctl and
a hook in ifconfig(8)), but they got yelled at because there was a
proposal to "do it better" which was never implemented. Much later, I
needed to do this (because I had two machines that arrived with the
same MAC address) and I asked if I could commit the code, but after a
long time no one made any decision on the subject and I gave up.

The patches to do this the straightforward way are quite simple. You
can find the code in the archives (and possibly in the PR database, I
can't remember if it is there as well or not.) It would be a good
thing if they finally got committed, if only until such time in the
distant future when the "perfect" way gets coded.

Perry

[Johnny Billquist-4]

Greg Troxel wrote:

> Johnny Billquist <bqt@...> writes:
>
>
>>This all reminds me of another thing.
>>Are there any way, or plans to be able to, change the mac address of
>>an ethernet interface in NetBSD? I don't think there is, but I might
>>have missed something.
>
> I think someone just needs to write device-indenpendent support and then
> modify various drivers.
>
>>When doing stuff like Olaf is doing here, changing the MAC address
>>might be desireable (in his case if he wants his PDP-10 to run TOPS-20
>>with DECnet enabled, since DECnet wants specific MAC addresses on
>>interfaces...)
>
> I don't think the tap's Ethernet address is relevant.  The PDP-10's
> emulated Ethernet interface (how modern to plug one of those in, instead
> of CHAOSnet or an IMP....) will have an address, but that's controlled
> entirely within the emulator.  Packets with that source address are
> generated, and then written to /dev/tapN to make them show up on the
> virtual interface on the host.

Are you suggesting that we'll always be in promiscous mode then?
I'm not just thinking of source address information, but also reception
of packets from the outside destined to the virtual machine.
A PDP-10 can be running both DECnet and TCP/IP, in which case you'll
want it's ethernet address to show up in other machines arp tables as
well, and so on. And maybe you'd like to dedicate a specific ethernet
interface to the virtual machine, so that it's actually directly hooked
into the physical network. So you'd really need the ability to change
the mac address of specific interfaces.

        Johnny

--
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt@...             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol

[bqt.vcf]

---

begin:vcard
fn:Johnny Billquist
n:Billquist;Johnny
email;internet:bqt@...
tel;cell:[phone]
x-mozilla-html:FALSE
version:2.1
end:vcard

---

[Greg Troxel]

Johnny Billquist <bqt@...> writes:

> Greg Troxel wrote:
>> Johnny Billquist <bqt@...> writes:
>>
>>>When doing stuff like Olaf is doing here, changing the MAC address
>>>might be desireable (in his case if he wants his PDP-10 to run TOPS-20
>>>with DECnet enabled, since DECnet wants specific MAC addresses on
>>>interfaces...)
>>
>> I don't think the tap's Ethernet address is relevant.  The PDP-10's
>> emulated Ethernet interface (how modern to plug one of those in, instead
>> of CHAOSnet or an IMP....) will have an address, but that's controlled
>> entirely within the emulator.  Packets with that source address are
>> generated, and then written to /dev/tapN to make them show up on the
>> virtual interface on the host.
>
> Are you suggesting that we'll always be in promiscous mode then?

See at end.

> I'm not just thinking of source address information, but also
> reception of packets from the outside destined to the virtual machine.

This is why you want to use a bridge to connect the physical Ethernet to
the virtual Ethernet that has the tap0 (on the host) and the emulated
machine's interface.

> A PDP-10 can be running both DECnet and TCP/IP, in which case you'll
> want it's ethernet address to show up in other machines arp tables as
> well, and so on.

Sure, and bridging will do this.  Ethernet packets are created within
the emulator, and written into /dev/tap.  They then appear on tapN (the
struct ifnet side), as if they had been received from a hardware
interface.  So the source and destination addresses have already been
set by the emulator.  Those packets then get bridged.

> And maybe you'd like to dedicate a specific ethernet
> interface to the virtual machine, so that it's actually directly
> hooked into the physical network. So you'd really need the ability to
> change the mac address of specific interfaces.

Yes, but not of any interface on the *host*.

This is exactly what I (and many others) are doing on Xen, except that
instead of tap there are a pair of interfaces xvifN.M on the host (dom0)
and xennetM on the emulated machine (domU).  Essentially the equivalent of
tap is inside this, so that when the domU sends a packet it appears to
be received on the xvif interface on the dom0.

On my dom0:

xvif6.0: flags=8963<UP,BROADCAST,NOTRAILERS,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        address: aa:00:00:fa:00:04
        inet6 fe80::a800:ff:fefa:4%xvif6.0 prefixlen 64 scopeid 0x12

On my domU:

xennet0: flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        address: aa:00:00:f9:00:04
        inet [redacted] netmask 0xffffff00 broadcast [redacted]
        inet6 fe80::a800:ff:fef9:4%xennet0 prefixlen 64 scopeid 0x2
        inet6 [redacted] prefixlen 64


The MAC address for xennet0 is set in a xen config file, and the address
on xvif6.0 is apparently made up from that.

I have a bridge configured on the dom0, with a real interface and all
the xvifN.0 for the domUs.

On a real machine on the same ethernet, the xennet0 MAC address shows up
in ARP tables.

I think this is exactly how a PDP-10 emulator should work, except that
xennet0 is whatetever Ethernet interface that plugs in to the PDP-10 and
as seen by TOPS-20, xvif is tap, and instead of a xen hypervisor the
emulator passes packets.

So, with a bridge, yes all interfaces will be in promiscuous mode.  But
this shouldn't really matter.

[Rhialto]

On Tue 10 Apr 2007 at 10:33:55 -0400, Greg Troxel wrote:
> This is why you want to use a bridge to connect the physical Ethernet to
> the virtual Ethernet that has the tap0 (on the host) and the emulated
> machine's interface.

I am now trying to code something like that, and I am struck with the
strangely different way that one has to create a bridge(4) compared to
creating a tap(4).

For a tap(4), one open(2)s /dev/tap, and asks with
ioctl(,TAPGIFNAME,...) which number one got.

Same, more or less, with bpf(4) except that the number isn't
interesting. The old way was to try /dev/bpf0, /dev/bpf1, etc, until
successful.

bridge(4) is a lot more inconvenient. There is only
ioctl(,SIOCIFCREATE,) and you must try it with bridge0, bridge1, etc,
until successful.

And yet, ifconfig -C lists bridge as a cloning device.

-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.

[Anders Hjalmarsson]

On Sun, 08 Apr 2007 23:28:03 +0200, Rhialto wrote:

>
> The problem with sending frames through bpf(4)[1] is that although they
> get sent out on the wire, the local host does not see them.
> Asymmetrically, packets that are sent by the local host *are* copied
> into bpf(4) (although this is undocumentedly optional, but default on).
> In a way this makes sense, since, being a packet filter it is more meant
> to see packets that are on the wire and sending packets out does not
> really fit in its philosophy.
>
> But this gives the weird effect that the emulator can successfully
> communicate with the whole world, *execpt* the host it is running on,
> and that in one direction only.
>

 ...
>
> On the other hand, bpf(4) could get fixed too. I looked at it briefly,
> but ether_input() consumes the packet so it was not just a matter of
> adding one call. Probably needs a 2 calls: a copy too.
>


This patch is based on something Manuel Bouyer posted some years ago, but
fixed so that two bpf-using programs on the same host can communicate,
e.g. simh-vax and mopd.



Index: if_ethersubr.c
===================================================================
RCS file: /cvsroot/src/sys/net/if_ethersubr.c,v
retrieving revision 1.148
diff -u -r1.148 if_ethersubr.c
--- if_ethersubr.c 7 Mar 2007 22:20:05 -0000 1.148
+++ if_ethersubr.c 11 Apr 2007 21:43:43 -0000
@@ -205,6 +205,7 @@
  struct mbuf *m = m0;
  struct rtentry *rt;
  struct mbuf *mcopy = (struct mbuf *)0;
+ struct mbuf *minput;
  struct ether_header *eh;
  struct ifnet *ifp = ifp0;
  ALTQ_DECL(struct altq_pktattr pktattr;)
@@ -433,6 +434,14 @@
     sizeof(edst));
  /* AF_UNSPEC doesn't swap the byte order of the ether_type. */
  etype = ((const struct ether_header *)dst->sa_data)->ether_type;
+ /* set M_BCAST or M_MCAST accrodingly, upper layer (e.g. bpf)
+ * may not have set it.
+ */
+
+ if (memcmp(etherbroadcastaddr, edst, ETHER_ADDR_LEN) == 0)
+ m->m_flags |= M_BCAST;
+ else if (ETHER_IS_MULTICAST(edst))
+ m->m_flags |= M_MCAST;
  break;
 
  default:
@@ -478,6 +487,50 @@
  if (m == NULL)
  return (0);
 #endif
+ if (hdrcmplt) {
+ /* check if destination is for us */
+ if (memcmp(LLADDR(ifp->if_sadl), edst, ETHER_ADDR_LEN) == 0) {
+ /* just pass it to ether_input, no need to wire. */
+ int s;
+
+ m->m_pkthdr.rcvif = ifp;
+ s = splnet();
+#if NBPFILTER > 0
+ /*
+ * Check if there's a BPF listener on this interface.
+ * If so, hand off the raw packet to BPF.
+ */
+ if (ifp->if_bpf)
+ bpf_mtap(ifp->if_bpf, m);
+#endif
+
+ ether_input(ifp, m);
+ splx(s);
+ return (0);
+ } else if (m->m_flags & (M_BCAST | M_MCAST)) {
+ /*
+ * in this case we have to copy a packet to
+ * ether_input.
+ */
+ int s;
+
+ minput = m_copypacket(m, M_DONTWAIT);
+ if (minput) {
+ minput->m_pkthdr.rcvif = ifp;
+ s = splnet();
+#if NBPFILTER > 0
+ /*
+ * Check if there's a BPF listener on this interface.
+ * If so, hand off the raw packet to BPF.
+ */
+ if (ifp->if_bpf)
+ bpf_mtap(ifp->if_bpf, minput);
+#endif
+ ether_input(ifp, minput);
+ splx(s);
+ }
+ }
+ }
 
 #if NBRIDGE > 0
  /*

[Rhialto]

On Wed 11 Apr 2007 at 23:51:55 +0200, Anders Hjalmarsson wrote:
> This patch is based on something Manuel Bouyer posted some years ago, but
> fixed so that two bpf-using programs on the same host can communicate,
> e.g. simh-vax and mopd.

Yes, that is nice. I assume though that a bpf reader will get its own
packets back though (if they pass the filter).

I wonder, by the way, why this patch is adding code in ether_output() in
if_ethersubr.c. I was thinking to add some code (much like this) in
bpf_write() in bpf.c, near the call to (*ifp->if_output)(). That would
only inspect packets injected into bpf, not all outgoing packets to all
interfaces.

-Olaf.
--
___ Olaf 'Rhialto' Seibert      -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl        -- Cetero censeo "authored" delendum esse.