AFF A300 to A400 ARL Headswap: Lessons Learned

Two interesting things encountered whilst doing an AFF A300 to A400 ARL Headswap on 9.10.1P12.

I was using the official procedure (I have a cut down version that I print out and take with me to site - I'll have to share this at some point):

• Manually upgrade controller hardware running ONTAP 9.8 or later
• https://docs.netapp.com/us-en/ontap-systems-upgrade/upgrade-arl-manual-app/index.html

Issue 1) Ports on the X1147A card did not come up

This had me scratching my head. I'd headswapped one node, so we had an A300 and A400 in the cluster, but could not get the temporary cluster connection (from onboard ports on the A300 to X1147A card in slot 1 on the A400) to come up. I tried a few things - different cables, different SFPs - but could not get the temporary cluster network link to come up. I contemplated needing to roll-back the headswap. Then I went and got a cup of tea, got on the wifi and found this in kb.netapp.com:

• Port on the expansion card X1147A do not link up - NetApp Knowledge Base
• NetApp Support Site - BURT - 1347483

Running the "system power cycle" on the new A400 did the trick. And we were able to proceed.

Note: When I did this a second time (for second head swap) I had to get the default BMC credentials from NetApp. The usual BMC credentials did not work. There is an article on KB.NetApp.Com accessible internal only, entitled "What are the Default Passwords for Service Processors and BMCs?"

Issue 2) X1133A and X1135A ports are physically reversed

A300: slot 2 OK: X1133A: Broadcom LPe16004 rev. A0
A400: slot 5 OK: X1135A: HBA,4-Port FCP Target 32Gb PCIe

I dutifully moved the fibre channel connections from e2a/b/c/d to e5a/b/c/d, or I thought I did, but I hadn't actually done that, I'd moved them to to e5d/c/b/a. The X1133A and X1135A ports are physically reversed even though the LC/LC cable tab is pointing downwards on both cards.

I'd been checking the output of -

vserver fcp initiator show

- before downing the node 2 FC LIFs, but I wasn't seeing any initiators on node 1, so didn't want to down the node 2 FC LIFs as this would have caused an outage.

Once I'd corrected cabling, all was good :)

Image: X1133A and X1135A ports are reversed from left to right (2a/b/c/d on A300, 5d/c/b/a on A400):