Configure N5K System Qos for FCOE and Jumbo MTU

N5K-1# sh pol sys
Type network-qos policy-maps
===============================

policy-map type network-qos default-nq-policy
class type network-qos class-fcoe
match qos-group 1

pause no-drop
mtu 2158
class type network-qos class-default
match qos-group 0

mtu 1500

Service-policy (qos) input: default-in-policy
policy statistics status: enabled

Class-map (qos): class-fcoe (match-any)
Match: cos 3
set qos-group 1
N5K-1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
N5K-1(config)# feature fcoe
FC license checked out successfully
fc_plugin extracted successfully
FC plugin loaded successfully
FCoE manager enabled successfully
FC enabled on all modules successfully
N5K-1(config)# sh pol sys
Type network-qos policy-maps
===============================

policy-map type network-qos default-nq-policy
class type network-qos class-fcoe
match qos-group 1

pause no-drop
mtu 2158
class type network-qos class-default
match qos-group 0

mtu 1500

Service-policy (qos) input: default-in-policy
policy statistics status: enabled

Class-map (qos): class-fcoe (match-any)
Match: cos 3
set qos-group 1

N5K-1(config)# policy-map type network-qos jumbo

N5K-1(config-pmap-nq)# class type network-qos class-fcoe
N5K-1(config-pmap-nq-c)# exit
N5K-1(config-pmap-nq)# class type network class-default
N5K-1(config-pmap-nq-c)# mtu 9000
N5K-1(config-pmap-nq-c)# exit
N5K-1(config-pmap-nq)# exit
N5K-1(config)# system qos
N5K-1(config-sys-qos)# service-policy type network-qos jumbo
N5K-1(config-sys-qos)# sh pol sys
Type network-qos policy-maps
===============================

policy-map type network-qos jumbo
class type network-qos class-fcoe
match qos-group 1

pause no-drop
mtu 2158
class type network-qos class-default
match qos-group 0

mtu 9000

Service-policy (qos) input: default-in-policy
policy statistics status: enabled

Class-map (qos): class-fcoe (match-any)
Match: cos 3
set qos-group 1
N5K-1(config-sys-qos)#

Nexus 5548 and unsupported CNA

Nexus 5548 Troubleshooting

Possible Cause

The type of Converged Network Adapter might not be supported.

Solution

Ensure that the type of adapter is supported. The FCoE interface only supports a Generation-2 Converged Network Adapter.

Possible Cause

The FCoE class-fcoe system class is not enabled in the QoS configuration.

Solution

For a Cisco Nexus 5548 switch, the FCoE class-fcoe system class is not enabled by default in the QoS configuration. Before enabling FCoE, you must include class-fcoe in each of the following policy types:

Network-QoS

Queuing

QoS

The following is an example of a service policy that needs to be configured:

F340.24.10-5548-1

class-map type qos class-fcoe

class-map type queuing class-fcoe

match qos-group 1

class-map type queuing class-all-flood

match qos-group 2

class-map type queuing class-ip-multicast

match qos-group 2

class-map type network-qos class-fcoe

match qos-group 1

class-map type network-qos class-all-flood

match qos-group 2

class-map type network-qos class-ip-multicast

match qos-group 2

system qos

 service-policy type qos input fcoe-default-in-policy

 service-policy type queuing input fcoe-default-in-policy

 service-policy type queuing output fcoe-default-out-policy

 service-policy type network-qos fcoe-default-nq-policy

MTU N5K / FI

The Cisco Nexus 5000 Series switch is a Layer 2 device. This means it does not fragment frames. As a result, the switch cannot have two ports in the same Layer 2 domain with different maximum transmission units (MTUs). A per-physical Ethernet interface MTU is not supported. Instead, the MTU is set according to the QoS classes. You modify the MTU by setting Class and Policy maps.


Note

When you show the interface settings, a default MTU of 1500 is displayed for physical Ethernet interfaces and a receive data field size of 2112 is displayed for Fibre Channel interfaces.

 

policy-map type network-qos system_nq_policy
class type network-qos class-fcoe
pause no-drop
mtu 2158
class type network-qos class-platinum
mtu 9000
pause no-drop
class type network-qos class-gold
mtu 9100
class type network-qos class-default
system qos
service-policy type qos input system_qos_policy
service-policy type queuing input system_q_in_policy
service-policy type queuing output system_q_out_policy
service-policy type network-qos system_nq_policy

MTU over vPC N7K.

Jumbo frames are enabled by default on the vPC peer link.

No MTU size configured, just system jumbomtu 9216 (Which is default)

N7K-N7K-DC1-3(config-if)# ping 10.10.30.1 pack 9000
PING 10.10.30.1 (10.10.30.1): 9000 data bytes
9008 bytes from 10.10.30.1: icmp_seq=0 ttl=254 time=8.365 ms
Request 1 timed out
9008 bytes from 10.10.30.1: icmp_seq=2 ttl=254 time=8.141 ms
Request 3 timed out
9008 bytes from 10.10.30.1: icmp_seq=4 ttl=254 time=8.384 ms

— 10.10.30.1 ping statistics —
5 packets transmitted, 3 packets received, 40.00% packet loss
round-trip min/avg/max = 8.141/8.296/8.384 ms
N7K-N7K-DC1-3(config-if)#

 

MTU 9216 on the Po :
N7K-N7K-DC1-3(config-if)# ping 10.10.30.1 pack 9000
PING 10.10.30.1 (10.10.30.1): 9000 data bytes
9008 bytes from 10.10.30.1: icmp_seq=0 ttl=254 time=8.317 ms
Request 1 timed out
9008 bytes from 10.10.30.1: icmp_seq=2 ttl=254 time=8.589 ms
Request 3 timed out
9008 bytes from 10.10.30.1: icmp_seq=4 ttl=254 time=8.341 ms

 

 

 

UCS in FC mode…

Configuration of a port in FC Switching Mode.

interface fc1/32
switchport mode E
channel-group 100 force
no shutdown

The same port in FC Endhost Mode :

interface fc1/32

switchport mode NP
switchport trunk mode on
channel-group 100 force
no shutdown

Feature fcoe-npv

 

N5K-1(config-if)# feature fcoe-npv
ERROR: Cannot enable feature fcoe-npv because feature fcoe is enabled. Disable fea
ture fcoe, reload the system, and try again.
N5K-1(config)#

 

N5K-1(config)# sh feature | i ena
assoc_mgr 1 enabled
fcoe 1 enabled
lldp 1 enabled
npv 1 enabled
sshServer 1 enabled
telnetServer 1 enabled

OTV Config and Verifications

N7K-1-OTV-2A(config-if-overlay)# sh otv

OTV Overlay Information
Site Identifier 0000.0000.0001

Overlay interface Overlay1

VPN name : Overlay1
VPN state : UP
Extended vlans : 131 151 171 211 (Total:4)
Control group : 239.2.1.1
Data group range(s) : 239.2.2.0/28
Join interface(s) : Eth1/18 (10.1.21.5)
Site vlan : 1005 (up)
AED-Capable : Yes
Capability : Multicast-Reachable
N7K-1-OTV-2A(config-if-overlay)# sh otv ar
OTV ARP/ND L3->L2 Address Mapping Cache
N7K-1-OTV-2A(config-if-overlay)# sh otv route

N7K-1-OTV-2A(config-if-overlay)# sh otv adj
Overlay Adjacency database

Overlay-Interface Overlay1 :
Hostname System-ID Dest Addr Up Time State
N7K-2-OTV-2B 0026.980d.92c3 10.1.23.5 00:05:34 UP
N7K-1-OTV-2A(config-if-overlay)# sh otv vlan

OTV Extended VLANs and Edge Device State Information (* – AED)

VLAN Auth. Edge Device Vlan State Overlay
—- ———————————– ———- ——-
131* N7K-1-OTV-2A active Overlay1
151* N7K-1-OTV-2A active Overlay1
171* N7K-1-OTV-2A active Overlay1
211* N7K-1-OTV-2A active Overlay1

N7K-1-OTV-2A(config-if-overlay)# sh run otv

!Command: show running-config otv
!Time: Fri Apr 12 00:46:23 2013

version 5.2(1)
feature otv

otv site-vlan 1005

interface Overlay1
otv join-interface Ethernet1/18
otv control-group 239.2.1.1
otv data-group 239.2.2.0/28
otv extend-vlan 131, 151, 171, 211
no shutdown
otv site-identifier 0x1

N7K-1-OTV-2A(config-if-overlay)# sh run int e 1/18

!Command: show running-config interface Ethernet1/18
!Time: Fri Apr 12 00:46:30 2013

version 5.2(1)

interface Ethernet1/18
mtu 9042
ip address 10.1.21.5/24
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
ip igmp version 3
no shutdown

N7K-1-OTV-2A(config-if-overlay)#

Fabricpath : How Are Multidestination Roots Selected?

Switch with highest priority value becomes root for the tree
‒ Highest system ID, then highest Switch ID value, used in event of a tie.

Image

vPC : Role and system priority

The recommended practice for configuring role priorities is to ensure that the primary vPC peer is the same device used as the Spanning Tree root bridge and HSRP active peer. This can be achieved by lowering the role priority (default value is 32667), using the following commands:

vpc domain

role priority

The vPC System Priority parameter corresponds to the LACP System Priority value, and is used to guarantee that a peer partner cannot make a decision in terms of aggregation capability and advertisement. In the case of LACP, it is possible to have up to 16 Ethernet ports in a channel, however only eight of these ports can be active at any one time, with the remaining eight operating in standby mode. If there are more than eight ports in the link, the switch at the controlling end of the link will make a decision on which ports are placed into the channel.

It is recommended to manually configure the vPC System Priority to ensure that the vPC peer devices are the ‘primary’ devices in terms of LACP. It is also important that the vPC System Priority is identical on each pair. vPC System Priority is configured using the following commands:

vpc domain

system-priority

vPC / CFS

Do not enter the no cfs eth distribute or the no cfs distribute command. You must enable CFSoE for vPC functionality. If you do enter either of these commands with vPC enabled, the system displays an error message.


When you enter the show cfs application command, the output displays “Physical-eth,” which shows the applications that are using CFSoE