kern/140326: em0: watchdog timeout when communicating to windows using 9K MTU

View: New views
3 Messages — Rating Filter:   Alert me  

kern/140326: em0: watchdog timeout when communicating to windows using 9K MTU

by Maxim Sobolev-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


>Number:         140326
>Category:       kern
>Synopsis:       em0: watchdog timeout when communicating to windows using 9K MTU
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:      
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Nov 06 01:20:01 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Maksym Sobolyev
>Release:        7.2-p4
>Organization:
Sippy Software, Inc.
>Environment:
FreeBSD pioneer.sippysoft.com 7.2-RELEASE-p4 FreeBSD 7.2-RELEASE-p4 #0: Sun Oct  4 03:08:04 PDT 2009     root@...:/usr/obj/usr/src/sys/PIONEER  amd64
>Description:
My em0 interface repeatedly hangs up with watchdog timeout when communicating to the windows host at MTU 9K.

[sobomax@pioneer ~]$ grep em0 /var/run/dmesg.boot
em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xecc0-0xecdf mem 0xfe6e0000-0xfe6fffff,0xfe6d9000-0xfe6d9fff irq 21 at device 25.0 on pci0
em0: Using MSI interrupt
em0: [FILTER]
em0: Ethernet address: 00:22:19:32:87:2f
[sobomax@pioneer ~]$ uname -a
FreeBSD pioneer.sippysoft.com 7.2-RELEASE-p4 FreeBSD 7.2-RELEASE-p4 #0: Sun Oct  4 03:08:04 PDT 2009     root@...:/usr/obj/usr/src/sys/PIONEER  amd64
[sobomax@pioneer ~]$ ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 00:22:19:32:87:2f
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
        inet6 fec0::1 prefixlen 64
        media: Ethernet autoselect (1000baseTX <full-duplex>)
        status: active
[sobomax@pioneer ~]$ dmesg | grep watchd
em0: watchdog timeout -- resetting
em0: watchdog timeout -- resetting
em0: watchdog timeout -- resetting
em0: watchdog timeout -- resetting
em0: watchdog timeout -- resetting

I have managed to make a packet capture right at the time when hang happens. It appears to be that either "MAC Pause" or "TCP Segment of reassembled PDU" is the last packet that goes through before the interface hangs.

Here is the screenshot, if somebody wants to take closer look at the actual packets please let me know.

http://sobomax.sippysoft.com/~sobomax/ScreenShot527.png

Turning off TSO and TXCSUM/RXCSUM has not helped. Bringing MTU down to 1,500 resolved the issue.

I have had the same problem happening several times in the past (although I initially attributed it to the bad cable or something like that), so it's definitely not on-off issue.

Given popularity of intel/pro chips in today's computers it look like quite serious issue to me. Any help is greatly appreciated.
>How-To-Repeat:

>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted:
_______________________________________________
freebsd-bugs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@..."

Parent Message unknown Re: kern/140326: em0: watchdog timeout when communicating to windows using 9K MTU

by Jack Vogel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

The following reply was made to PR kern/140326; it has been noted by GNATS.

From: Jack Vogel <jfvogel@...>
To: Maksym Sobolyev <sobomax@...>
Cc: freebsd-gnats-submit@...
Subject: Re: kern/140326: em0: watchdog timeout when communicating to windows
        using 9K MTU
Date: Thu, 5 Nov 2009 17:28:50 -0800

 --0016e6d99d6125581f0477a9c469
 Content-Type: text/plain; charset=ISO-8859-1
 
 Can't do much unless you adequately identify hardware, on BOTH sides,
 believe
 it or not "windows" is not a sufficient description :)
 
 I need to know what the E1000 hardware is, using pciconf -l, and I also need
 to
 know what is on the Windows side before having a clue on how to repro or
 help
 you.
 
 Cheers,
 
 Jack
 
 
 On Thu, Nov 5, 2009 at 5:18 PM, Maksym Sobolyev <sobomax@...> wrote:
 
 >
 > >Number:         140326
 > >Category:       kern
 > >Synopsis:       em0: watchdog timeout when communicating to windows using
 > 9K MTU
 > >Confidential:   no
 > >Severity:       serious
 > >Priority:       high
 > >Responsible:    freebsd-bugs
 > >State:          open
 > >Quarter:
 > >Keywords:
 > >Date-Required:
 > >Class:          sw-bug
 > >Submitter-Id:   current-users
 > >Arrival-Date:   Fri Nov 06 01:20:01 UTC 2009
 > >Closed-Date:
 > >Last-Modified:
 > >Originator:     Maksym Sobolyev
 > >Release:        7.2-p4
 > >Organization:
 > Sippy Software, Inc.
 > >Environment:
 > FreeBSD pioneer.sippysoft.com 7.2-RELEASE-p4 FreeBSD 7.2-RELEASE-p4 #0:
 > Sun Oct  4 03:08:04 PDT 2009     root@...:/usr/obj/usr/src/sys/PIONEER
 >  amd64
 > >Description:
 > My em0 interface repeatedly hangs up with watchdog timeout when
 > communicating to the windows host at MTU 9K.
 >
 > [sobomax@pioneer ~]$ grep em0 /var/run/dmesg.boot
 > em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xecc0-0xecdf mem
 > 0xfe6e0000-0xfe6fffff,0xfe6d9000-0xfe6d9fff irq 21 at device 25.0 on pci0
 > em0: Using MSI interrupt
 > em0: [FILTER]
 > em0: Ethernet address: 00:22:19:32:87:2f
 > [sobomax@pioneer ~]$ uname -a
 > FreeBSD pioneer.sippysoft.com 7.2-RELEASE-p4 FreeBSD 7.2-RELEASE-p4 #0:
 > Sun Oct  4 03:08:04 PDT 2009     root@...:/usr/obj/usr/src/sys/PIONEER
 >  amd64
 > [sobomax@pioneer ~]$ ifconfig em0
 > em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
 >        options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
 >        ether 00:22:19:32:87:2f
 >        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
 >        inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
 >        inet6 fec0::1 prefixlen 64
 >        media: Ethernet autoselect (1000baseTX <full-duplex>)
 >        status: active
 > [sobomax@pioneer ~]$ dmesg | grep watchd
 > em0: watchdog timeout -- resetting
 > em0: watchdog timeout -- resetting
 > em0: watchdog timeout -- resetting
 > em0: watchdog timeout -- resetting
 > em0: watchdog timeout -- resetting
 >
 > I have managed to make a packet capture right at the time when hang
 > happens. It appears to be that either "MAC Pause" or "TCP Segment of
 > reassembled PDU" is the last packet that goes through before the interface
 > hangs.
 >
 > Here is the screenshot, if somebody wants to take closer look at the actual
 > packets please let me know.
 >
 > http://sobomax.sippysoft.com/~sobomax/ScreenShot527.png<http://sobomax.sippysoft.com/%7Esobomax/ScreenShot527.png>
 >
 > Turning off TSO and TXCSUM/RXCSUM has not helped. Bringing MTU down to
 > 1,500 resolved the issue.
 >
 > I have had the same problem happening several times in the past (although I
 > initially attributed it to the bad cable or something like that), so it's
 > definitely not on-off issue.
 >
 > Given popularity of intel/pro chips in today's computers it look like quite
 > serious issue to me. Any help is greatly appreciated.
 > >How-To-Repeat:
 >
 > >Fix:
 >
 >
 > >Release-Note:
 > >Audit-Trail:
 > >Unformatted:
 > _______________________________________________
 > freebsd-bugs@... mailing list
 > http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
 > To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@..."
 >
 
 --0016e6d99d6125581f0477a9c469
 Content-Type: text/html; charset=ISO-8859-1
 Content-Transfer-Encoding: quoted-printable
 
 Can't do much unless you adequately identify hardware, on BOTH sides, b=
 elieve<br>it or not "windows" is not a sufficient description :)<=
 br><br>I need to know what the E1000 hardware is, using pciconf -l, and I a=
 lso need to<br>
 know what is on the Windows side before having a clue on how to repro or he=
 lp<br>you.<br><br>Cheers,<br><br>Jack<br><br><br><div class=3D"gmail_quote"=
 >On Thu, Nov 5, 2009 at 5:18 PM, Maksym Sobolyev <span dir=3D"ltr"><<a h=
 ref=3D"mailto:sobomax@...">sobomax@...</a>></span> wrote=
 :<br>
 <blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
 >Number: =A0 =A0 =A0 =A0 140326<br>
 >Category: =A0 =A0 =A0 kern<br>
 >Synopsis: =A0 =A0 =A0 em0: watchdog timeout when communicating to windo=
 ws using 9K MTU<br>
 >Confidential: =A0 no<br>
 >Severity: =A0 =A0 =A0 serious<br>
 >Priority: =A0 =A0 =A0 high<br>
 >Responsible: =A0 =A0freebsd-bugs<br>
 >State: =A0 =A0 =A0 =A0 =A0open<br>
 >Quarter:<br>
 >Keywords:<br>
 >Date-Required:<br>
 >Class: =A0 =A0 =A0 =A0 =A0sw-bug<br>
 >Submitter-Id: =A0 current-users<br>
 >Arrival-Date: =A0 Fri Nov 06 01:20:01 UTC 2009<br>
 >Closed-Date:<br>
 >Last-Modified:<br>
 >Originator: =A0 =A0 Maksym Sobolyev<br>
 >Release: =A0 =A0 =A0 =A07.2-p4<br>
 >Organization:<br>
 Sippy Software, Inc.<br>
 >Environment:<br>
 FreeBSD <a href=3D"http://pioneer.sippysoft.com" target=3D"_blank">pioneer.=
 sippysoft.com</a> 7.2-RELEASE-p4 FreeBSD 7.2-RELEASE-p4 #0: Sun Oct =A04 03=
 :08:04 PDT 2009 =A0 =A0 root@...:/usr/obj/usr/src/sys/PIO=
 NEER =A0amd64<br>
 
 >Description:<br>
 My em0 interface repeatedly hangs up with watchdog timeout when communicati=
 ng to the windows host at MTU 9K.<br>
 <br>
 [sobomax@pioneer ~]$ grep em0 /var/run/dmesg.boot<br>
 em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xecc0-0xecdf =
 mem 0xfe6e0000-0xfe6fffff,0xfe6d9000-0xfe6d9fff irq 21 at device 25.0 on pc=
 i0<br>
 em0: Using MSI interrupt<br>
 em0: [FILTER]<br>
 em0: Ethernet address: 00:22:19:32:87:2f<br>
 [sobomax@pioneer ~]$ uname -a<br>
 FreeBSD <a href=3D"http://pioneer.sippysoft.com" target=3D"_blank">pioneer.=
 sippysoft.com</a> 7.2-RELEASE-p4 FreeBSD 7.2-RELEASE-p4 #0: Sun Oct =A04 03=
 :08:04 PDT 2009 =A0 =A0 root@...:/usr/obj/usr/src/sys/PIO=
 NEER =A0amd64<br>
 
 [sobomax@pioneer ~]$ ifconfig em0<br>
 em0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mt=
 u 9000<br>
  =A0 =A0 =A0 =A0options=3D98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM><br>
  =A0 =A0 =A0 =A0ether 00:22:19:32:87:2f<br>
  =A0 =A0 =A0 =A0inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255=
 <br>
  =A0 =A0 =A0 =A0inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255=
 <br>
  =A0 =A0 =A0 =A0inet6 fec0::1 prefixlen 64<br>
  =A0 =A0 =A0 =A0media: Ethernet autoselect (1000baseTX <full-duplex>)=
 <br>
  =A0 =A0 =A0 =A0status: active<br>
 [sobomax@pioneer ~]$ dmesg | grep watchd<br>
 em0: watchdog timeout -- resetting<br>
 em0: watchdog timeout -- resetting<br>
 em0: watchdog timeout -- resetting<br>
 em0: watchdog timeout -- resetting<br>
 em0: watchdog timeout -- resetting<br>
 <br>
 I have managed to make a packet capture right at the time when hang happens=
 . It appears to be that either "MAC Pause" or "TCP Segment o=
 f reassembled PDU" is the last packet that goes through before the int=
 erface hangs.<br>
 
 <br>
 Here is the screenshot, if somebody wants to take closer look at the actual=
  packets please let me know.<br>
 <br>
 <a href=3D"http://sobomax.sippysoft.com/%7Esobomax/ScreenShot527.png" targe=
 t=3D"_blank">http://sobomax.sippysoft.com/~sobomax/ScreenShot527.png</a><br=
 >
 <br>
 Turning off TSO and TXCSUM/RXCSUM has not helped. Bringing MTU down to 1,50=
 0 resolved the issue.<br>
 <br>
 I have had the same problem happening several times in the past (although I=
  initially attributed it to the bad cable or something like that), so it=
 9;s definitely not on-off issue.<br>
 <br>
 Given popularity of intel/pro chips in today's computers it look like q=
 uite serious issue to me. Any help is greatly appreciated.<br>
 >How-To-Repeat:<br>
 <br>
 >Fix:<br>
 <br>
 <br>
 >Release-Note:<br>
 >Audit-Trail:<br>
 >Unformatted:<br>
 _______________________________________________<br>
 <a href=3D"mailto:freebsd-bugs@...">freebsd-bugs@...</a> ma=
 iling list<br>
 <a href=3D"http://lists.freebsd.org/mailman/listinfo/freebsd-bugs" target=
 =3D"_blank">http://lists.freebsd.org/mailman/listinfo/freebsd-bugs</a><br>
 To unsubscribe, send any mail to "<a href=3D"mailto:freebsd-bugs-unsub=
 scribe@...">freebsd-bugs-unsubscribe@...</a>"<br>
 </blockquote></div><br>
 
 --0016e6d99d6125581f0477a9c469--
_______________________________________________
freebsd-bugs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@..."

Parent Message unknown Re: kern/140326: em0: watchdog timeout when communicating to windows using 9K MTU

by Maxim Sobolev-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

The following reply was made to PR kern/140326; it has been noted by GNATS.

From: Maxim Sobolev <sobomax@...>
To: Jack Vogel <jfvogel@...>
Cc: freebsd-gnats-submit@...
Subject: Re: kern/140326: em0: watchdog timeout when communicating to windows
  using 9K MTU
Date: Thu, 05 Nov 2009 18:28:13 -0800

 Jack Vogel wrote:
 > Can't do much unless you adequately identify hardware, on BOTH sides,
 > believe
 > it or not "windows" is not a sufficient description :)
 >
 > I need to know what the E1000 hardware is, using pciconf -l, and I also
 > need to
 > know what is on the Windows side before having a clue on how to repro or
 > help
 > you.
 
 Jack,
 
 Thank you for the amazingly fast reply.
 
 Sure, FreeBSD side is this:
 
 em0@pci0:0:25:0:        class=0x020000 card=0x02761028 chip=0x10de8086
 rev=0x02 hdr=0x00
      vendor     = 'Intel Corporation'
      class      = network
      subclass   = ethernet
 
 On windows side it's Realtek GiGe card. The system itself is Windows 7
 Ultimate 64-bit edition:
 
 PCI\VEN_10EC&DEV_8168&SUBSYS_02C01028&REV_03
 
 Please let me know if any other information is necessary.
 
 Regards,
 --
 Maksym Sobolyev
 Sippy Software, Inc.
 Internet Telephony (VoIP) Experts
 T/F: +1-646-651-1110
 Web: http://www.sippysoft.com
 MSN: sales@...
 Skype: SippySoft
_______________________________________________
freebsd-bugs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-bugs
To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@..."