[Lessons Learned] Cisco Nexus 9336C Deployment on 10.3(1)(F) + Convert A300 to Switched

on

Some notes from a recent deployment of Cisco Nexus 9336C cluster switches from NetApp. And converting a switchless AFF A300 cluster to switched, to support addition of new nodes to the cluster.

1) Setting up the Cisco Nexus 9336C switch with latest NX-OS and RCF.

Basic setup, procedure to apply NX-OS and RCF, is pretty much the same as in these old blog posts:

The switches I had shipped on 9.3(5). I upgraded to 9.3(10), then to 10.3(1)F then applied the RCF file (Nexus-9336C-RCF-v1.8-Cluster-HA-Breakout) and all went perfectly 😊.

2) No route to host error seen when pinging from Cisco Nexus 9336C-FX2

Do it like this:

sw1# ping 10.10.10.10 VRF management

3) Return code 0x40450037 while attempting to install NX-OS 10.3(1)F or newer on Cisco switches

You need to upgrade to 9.3(10) before you can upgrade to 10.3(1)F.

4) Migrate from a switchless cluster with direct-attached storage

This did not work:

  • cs1(config)# interface e/1-34
  • cs1(config-if-range)# shutdown

I only had 3 ports to shutdown, so I did this (port 1 is a 4-way breakout):

  • cs1(config)# interface Eth1/1/1-4
  • cs1(config-if-range)# shutdown
  • cs1(config-if-range)# exit
  • cs1(config)# interface Eth1/7-8
  • cs1(config-if-range)# shutdown
  • cs1(config-if-range)# exit

Also the documentation is based on an A400, so when it starts talking about A400 cluster ports, replace with your system's cluster ports. And 'Step 2: Set up the shared switch' was not applicable.

5) Renaming a Cisco IP switch

  • SW1# configure terminal
  • SW1(config)# switchname CLUSTER_SW_1

6) Changing the NX9336C Password (from here)

  • SW1# configure terminal
  • SW1(config)# username admin password NEW_PASSWORD

7) Configuring the Cisco NX-OS Default Gateway Post Basic Setup (see here):

  • sw1# config t
  • sw1(config)# vrf context management
  • sw1(config-vrf)# ip route 0.0.0.0/0 mgmt0 1.1.1.1

Note: Replace 1.1.1.1 with your default gateway.

The output from show running-config of interface mgmt0 looks like this:

  • interface mgmt0
  •   vrf member management
  •   ip address 10.10.10.11/24
If you wanted to change the IP, it is just like this:
  • sw1# config t
  • sw1(config)# interface mgmt0
  • sw1(config-ig)# ip address 20.20.20.22/24
8) Changing Password of logged in user

Is just:

change-password

APPENDIX: Default configuration script

Default RCF = Nexus-9336C-RCF-v1.8-Cluster-HA-Breakout

  • Default configuration script supports:
    • 2x QSFP28 ports configured for 100GbE Inter-switch Link (ISL)
    • 3x QSFP28 ports configured in 4x10GbE
    • 3x QSFP28 ports configured in 4x25GbE
    • 28x QSFP28 ports configured for 40GbE / 100GbE

Comments

Post a Comment