Archive

Archive for August, 2011

Direct Connected Fiber Storage to UCS

August 24, 2011 Leave a comment

So i’ve come across this recently.  I have a client that is direct connecting the Fiber from their NetApp array to the 6120’s of the UCS.

The issue that has been raised is that this is not technically supported.  As is seems Cisco releases with the 1.4.1 firmware release that you can absolutely do this.  However there is a caveat, it’s supported by Cisco as long as the storage vendor will support it.

The biggest problem is that NetApp did support it, but they don’t any longer.  So it seems Cisco was left holding the ball when NetApp walked away.

So if your running a NetApp array that is direct connected to their UCS w/o an MDS or even a 5548 with the FC module, its no longer technically supported and you very well may run into issues if you need Vendor support.

For those not familiar with direct connecting the storage i’ll give a little but of information on it, as well as some of my experiences with it and some tips on making it “work” with UCS.

So inside the 6120 there is effectivly a very very dumb MDS switch.  There is no Zoning, it is all 1 big zone, you do vSANs, but obviously no inter-vSAN routing, no security, no real way of even getting any initiator/target information for troubleshooting purposes.

In order to even use the functionality, you must change the Fiber portion of the switch from “End-Host Mode” to “Switch Mode”.  This is EXTREMELY similar in method and functionality to switching the Network side to “Switch Mode”.

You MUST also make sure to select the default vSAN that is created upon inital set-up, and enable “Default Zoning”

Intersting note you MUST absolutely make sure the HBA name in the Boot Policy is the EXACT same as the HBA name in the HBA Template, or it won’t boot.
So again, in my opinion if you can avoid direct connecting your SAN storage to the 6120, please avoid it, at least until UCS 2.0 comes out  🙂

Enabling Jumbo Frames in a Flexpod Environment

August 24, 2011 3 comments

Update: I have fixed the 5548 section i was missing the last two lines.

This post will help the user enable Jumbo frames on their Flexpod environment. This document will also work for just about any UCS-based environment, however you will have to check on how to enable Jumbo Frames for their storage array.

This post assumes a few things;

Environment is running 5548 Nexus switches
User needs to setup Jumbo-Frames on the NetApp for NFS/CIFS Shares
Netapp has VIF or MMVIF connections for said NFS/CIFS connections.

Cisco UCS Configuration 

-Login to the UCSM, Click on the LAN Tab.
-Expand LANs, & LAN Cloud.
-Click on the QoS System Class, Change the “Best-Effort” MTU to 9216. 

NOTE: You need to just type in the number, it’s not one of the ones that can be selected in the drop-down.

Expand the Policies section on the LAN Tab.  Right-Click on the QoS Polices and click “Create new QoS Policy”.  Call it “Jumbo-Frames” or something similar.
-On the vNIC Template or actual vNIC on the Service Profile, set the “QoS Policy” to the new Policy.

 ESX/ESXi Configuration

-Either SSH or Console into the ESX host.  If your using ESXi you’ll need to ensure local or remote tech support mode is enabled.
-We need to set the vSwitch that the Jumbo-Framed NICs will be on to allow Jumbo-Frames.
          Type esxcfg-vswitch –l   find the vSwitch we need to modify.
          Type esxcfg-vswitch –m 9000 vSwitch# (Replace # with the actual number)
          Type esxcfg-vswitch –l   you should now see the MTU to 9000

-We now need to set the actual VMKernel NICs.

          Type esxcfg-vmknic –l  find the vmk’s that we need to modify
          Type esxcfg-vmknic –m 9000 <portgroup name> (this is the portgroup that the vmk is part of)
          Type esxcfg-vmknic –l   verify that the MTU is now 9000 

Note: If your using dvSwitches, you can set the MTU size through the VI-Client.

5548 Configuration 

Login to the 5548 switch on the “A” side.
-Type the following;

system jumbomtu 9216
policy-map type network-qos jumbo
class type network-qos class-default
mtu 9216
multi-cast-optimize
exit
system qos
service-policy type network-qos jumbo
exit
copy run start

-Repeat on the “B” Side 

NetApp Configuration 

-Login to the Filer.
-Type ifconfig –a  verify which ports we need to make run jumbo frames.
 -Type ifconfig <VIF_NAME> mtusize 9000 

NOTE: You need to make sure you enable jumbo-frames not only on the VLAN’d VIF but also the “root” VIF.