« Return to Thread: VIF weirdness

Re: VIF weirdness

by tmac-3 :: Rate this Message:

Reply to Author | View in Thread

Notes aboutof 10GbE TOE NIC limitations : The 10GbE TOE NIC cards
have a number of limitations. They include:
◆ Multimode vif limited to two (2) 10GbE TOE NICs
◆ LACP not supported with 10GbE TOE NICs
◆ TOE functionality disabled on 10GbE NIC in vif


Do you need a
switchport access vlan ###
in the interface lines?

Here is the config I use for vifs on my switch:

interface Port-Channel7
 description netapp-vif
 switchport
 switchport access vlan 239
 switchport mode access
 no ip address
 flowcontrol receive on
 flowcontrol send on
 spanning-tree portfast
!
interface GigabitEthernet7/42
 description filer-e9
 switchport
 switchport access vlan 239
 switchport mode access
 no ip address
 flowcontrol receive on
 flowcontrol send on
 spanning-tree portfast
 channel-group 7 mode active
!

In your case, with 10-gig, the only change should be:
channel-group 7 mode on
(which you are already doing)


Also, make sure your filer is setup correctly (not using LACP for 10Gig)

vif create multi tgif -b mac e2a r3b

On Thu, May 15, 2008 at 3:55 PM, Page, Jeremy <jeremy.page@...> wrote:

> I *think* the problem may be isolated to array1. I've tried setting them
> to single mode instead of multi (LACP is not supposed to work on a
> stacked switch).
>
>
> array01> sysconfig -v 2
>        slot 2: Dual TOE-10G Ethernet Controller (T320E-XFP)
>                Device Type:        CT-31-1
>                Version Number:
> T3-SRAM1.1.0-BR1040-20-C0-FW4.6.0-DR03
>                Serial Number:      PT4107009
>                e2a MAC Address:    00:07:43:05:16:98
> (auto-10g_sr-fd-up)
>                e2b MAC Address:    00:07:43:05:16:99
> (auto-10g_sr-fd-up)
>
>
> **switch config (short form)**
> version 12.2
> no aaa new-model
> switch 1 provision ws-c3750e-24td
> switch 2 provision ws-c3750e-24td
> interface TenGigabitEthernet1/0/1
>  switchport mode access
>  channel-group 1 mode on
> !
> interface TenGigabitEthernet1/0/2
>  switchport mode access
>  channel-group 2 mode on
> interface TenGigabitEthernet2/0/1
>  switchport mode access
>  channel-group 1 mode on
> !
> interface TenGigabitEthernet2/0/2
>  switchport mode access
>  channel-group 2 mode on
> !
> interface Vlan1
>  ip address 172.1.0.250 255.255.0.0
>
> -----Original Message-----
> From: tmac [mailto:tmacmd@...]
> Sent: Thursday, May 15, 2008 3:46 PM
> To: Page, Jeremy
> Cc: toasters@...
> Subject: Re: VIF weirdness
>
> How about a "sysconfig -v 2" from the filers
> and a "show running-config " that shows at a minimum:
> the port config of the 10-gig stuff and the port-channel definitions.
>
> --tmac
>
> On Thu, May 15, 2008 at 2:19 PM, Page, Jeremy <jeremy.page@...>
> wrote:
>> They are 10 gig cards, they are only in  a VIF with themselves (no
> other
>> cards). It's a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g
>> connections. The 2 1000 are on our production network as lan0 VIF for
>> CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on
> the
>> storage only network (no routing etc, private for storage devices and
> the
>> hosts using them only).
>>
>>
>>
>> They are plugged into stacked 3750e boxes, I tried putting them on the
> same
>> switch just to make sure it was not a stacking thing with no real
> change.
>>
>>
>>
>> The oddness is that occasionally a single packet will get through, and
> if I
>> bring the interfaces up and down it may change who can get to what.
>>
>>
>>
>> ________________________________
>>
>> From: Webster, Stetson [mailto:Stetson.Webster@...]
>> Sent: Thursday, May 15, 2008 2:13 PM
>> To: Page, Jeremy; toasters@...
>> Subject: RE: VIF weirdness
>>
>>
>>
>> Are these 10g TOE cards?  Are they in a VIF with other 'non-TOE'
> cards?
>>
>>
>>
>> Stetson M. Webster
>> Onsite Professional Services Engineer
>> PS - North Amer. - East
>>
>> NetApp
>> 919.250.0052 Mobile
>> Stetson.Webster@...
>> www.netapp.com
>>
>>
>>
>>
>>
>> ________________________________
>>
>> From: Page, Jeremy [mailto:jeremy.page@...]
>> Sent: Thursday, May 15, 2008 1:32 PM
>> To: toasters@...
>> Subject: VIF weirdness
>>
>> I am having a great deal of difficulty getting my filer's 10g
> interfaces
>> connected to a pair of 3750E switches I am trying to use for my
> storage
>> network's backbone. I can inconsistently ping other objects on the
> network
>> (there really is nothing there yet, we are just getting  started, but
> I do
>> have a host and an RLM card just for troubleshooting purposes), but
> they
>> cannot ping each other.  >From a remote host (172.1.2.5)  I can ping
>> everything but array1. If I bring down either of the two stacked
> switches
>> (which brings down one port of each VIF member pairs) everything
> works.
>>
>>
>>
>> array01> ping 172.1.0.1
>>
>> 172.1.0.1 is alive
>>
>> array01> ping 172.1.0.2
>>
>> array01> ping 172.1.0.4
>>
>> ping: wrote 172.1.0.4 64 chars, error=Host is down
>>
>> ping: wrote 172.1.0.4 64 chars, error=Host is down
>>
>> array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]:
> Local NIS
>> group update successful.
>>
>> array01> ping 172.1.2.5
>>
>> 172.1.2.5 is alive
>>
>>
>>
>> array02> ping 172.1.0.1
>>
>> array02> ping 172.1.0.4
>>
>> 172.1.0.4 is alive
>>
>> array02> ping 172.1.0.4
>>
>> 172.1.0.4 is alive
>>
>> array02> ping 172.1.0.250
>>
>> 172.1.0.250 is alive
>>
>> array02> ping 172.1.2.5
>>
>> no answer from 172.1.2.5
>>
>>
>>
>>
>>
>> array01> ifconfig -a
>>
>> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>>
>>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>>
>>         ether 00:00:00:00:00:00 (VIA Provider)
>>
>> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255
>>
>>         partner 10.28.17.214 (not in use)
>>
>>         ether 02:a0:98:08:22:b7 (Enabled virtual interface)
>>
>> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
> mtu 1500
>>
>>         inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255
>>
>>         partner 172.1.0.2 (not in use)
>>
>>         ether 02:a0:98:08:22:b6 (Enabled virtual interface)
>>
>>         nfo enabled
>>
>> array02>ifconfig -a
>>
>> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>>
>>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>>
>>         ether 00:00:00:00:00:00 (VIA Provider)
>>
>> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255
>>
>>         partner 10.28.17.213 (not in use)
>>
>>         ether 02:a0:98:0c:1a:37 (Enabled virtual interface)
>>
>> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
> mtu 1500
>>
>>         inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255
>>
>>         partner 172.1.0.1 (not in use)
>>
>>         ether 02:a0:98:0c:1a:36 (Enabled virtual interface)
>>
>>         nfo enabled
>>
>> This message (including any attachments) contains confidential
>> and/or proprietary information intended only for the addressee.
>> Any unauthorized disclosure, copying, distribution or reliance on
>> the contents of this information is strictly prohibited and may
>> constitute a violation of law. If you are not the intended
>> recipient, please notify the sender immediately by responding to
>> this e-mail, and delete the message from your system. If you
>> have any questions about this e-mail please notify the sender
>> immediately.
>>
>> This message (including any attachments) contains confidential
>> and/or proprietary information intended only for the addressee.
>> Any unauthorized disclosure, copying, distribution or reliance on
>> the contents of this information is strictly prohibited and may
>> constitute a violation of law. If you are not the intended
>> recipient, please notify the sender immediately by responding to
>> this e-mail, and delete the message from your system. If you
>> have any questions about this e-mail please notify the sender
>> immediately.
>>
>
>
>
> --
> --tmac
>
> RedHat Certified Engineer #804006984323821 (RHEL4)
> RedHat Certified Engineer #805007643429572 (RHEL5)
>
> Principal Consultant
>
>
> This message (including any attachments) contains confidential
> and/or proprietary information intended only for the addressee.
> Any unauthorized disclosure, copying, distribution or reliance on
> the contents of this information is strictly prohibited and may
> constitute a violation of law.  If you are not the intended
> recipient, please notify the sender immediately by responding to
> this e-mail, and delete the message from your system.  If you
> have any questions about this e-mail please notify the sender
> immediately.
>



--
--tmac

RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)

Principal Consultant

 « Return to Thread: VIF weirdness