Friday, 29 January 2021

[ONTAP 9.7] Steps to add 2nd NS224 Shelf to NetApp AFF A400

Setting the Scenario
 
You have a NetApp AFF A400 with 1 external NS224 shelf.
You already have an IO card in slot 5 (say for serving ethernet or fibre channel data).
You need to add a 2nd NS224 shelf.
 
High-Level Overview
 
The NS224 with NSM100 shelf modules has a max stack size of 1 (see HWU - hwu.netapp.com.) This means that in order to add a 2nd NS224 shelf to the A400, you need an X1148A NVMe Storage card (assuming you don’t have one already) and this must go in slot 5 following the ‘Priority Slot Assignment’ on HWU.
 
Starting point is one NS224 shelf cabled as:
 
Shelf|NSM|Shelf|Node |Node
     |   |Port |Port |
-----+---+-----+-----+----
01   | A | e0a | e0c | A
01   | B | e0a | e0c | B
01   | B | e0b | e0d | A
01   | A | e0b | e0d | B
 
Where:
- NSM A and Node A are the topmost NSM/Node.
- NSM B and Node B are the bottommost NSM/Node.
 
Image: NetApp AFF A400 cabled with one NS224 shelf

 
The process to add the 2nd shelf is:
 
1) Take over Node A
2) [Node A] Remove from the chassis
3) [Node A] Swap IO card in slot 5 into slot 1 (check HWU)
4) [Node A] Insert X1148A NVMe Storage card into slot 5
5) [Node A] Unplug e0d storage connection
6) [Node A] Boot and perform partial giveback
7) [Node A] Configure slot 5 storage ports
8) [Node A] Old e0d storage connection connects to e5b
9) [Node A] Perform necessary re-configurations for slot 1 IO card
10) [Node A] Complete giveback
11) Take over Node B
12) [Node B] Remove from the chassis
13) [Node B] Swap IO card in slot 5 into slot 1 (check HWU)
14) [Node B] Insert X1148A NVMe Storage card into slot 5
15) [Node B] Unplug e0d storage connection
16) [Node B] Boot and perform partial giveback
17) [Node B] Configure slot 5 storage ports
18) [Node B] Old e0d storage connection connects to e5b
19) [Node B] Perform necessary re-configurations for slot 1 IO card
20) [Node B] Complete giveback
21) [NS224 Shelf 2] Set shelf ID
22) [NS224 Shelf 2] Hot-add using node ports e5a (path A) and e0d (path B)
 
Finishing point is 2 NS224 shelves, cabled as:
 
Shelf|NSM|Shelf|Node |Node
     |   |Port |Port |
-----+---+-----+-----+----
01   | A | e0a | e0c | A
01   | B | e0a | e0c | B
01   | B | e0b | e5b | A
01   | A | e0b | e5b | B
02   | A | e0a | e5a | A
02   | B | e0a | e5a | B
02   | B | e0b | e0d | A
02   | A | e0b | e0d | B
 
Note: e5b (2) is to the left, and e5a (1) is to the right in the below diagram.
 
Image: NetApp AFF A400 cabled with one NS224 shelf


End-to-end, this process shouldn’t take longer than 4 hours (perhaps even less than 2 hours.)
 
Low-Level Commands
 
The low-level commands used in the operation will be bespoke to the particular system (for instance: the card originally in slot 5 might home SAN LIFs, or it might home NAS LIFs.) The below is a rough guide.
 
1) Take over Node A
 
Send AutoSupport and disable auto-giveback:
 
timeout
timeout modify -timeout 0
autosupport invoke -node * -type all -message "MAINT=4h Moving IO cards 1 to 5 and adding a shelf"
storage failover show -node * -fields auto-giveback
storage failover modify -node * -auto-giveback false
storage failover show -node * -fields auto-giveback
 
Card 5 had NAS LIFs: Temporarily rehome Node A’s LIFs to Node B:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
Card 5 had SAN LIFs: Leave for now.
 
Takeover Node A:
 
storage failover takeover -ofnode NODE-A
 
When the output says ‘Hit [Enter] to boot immediately, or any other key for command prompt.’ press any other key. When the system is at the LOADER> prompt it is safe to continue.
 
2) [Node A] Remove from the chassis
3) [Node A] Swap IO card in slot 5 into slot 1 (check HWU)
4) [Node A] Insert X1148A NVMe Storage card into slot 5
5) [Node A] Unplug e0d storage connection
6) [Node A] Boot and perform partial giveback
 
From Node B, perform a partial giveback:
 
storage failover giveback -ofnode NODE-A -only-cfo-aggregates true
storage failover show-giveback
 
7) [Node A] Configure slot 5 storage ports
 
storage port enable –node NODE-A –port PORT
 
8) [Node A] Old e0d storage connection connects to e5b
9) [Node A] Perform necessary re-configurations for slot 1 IO card
 
For NAS LIFs: Rebuild the IFGRPs/Ports/VLANs and perform clean up.
 
For SAN LIFs:
 
net int modify -vserver VSERVER -lif LIFNAME -status-admin down
net int modify -vserver VSERVER -lif LIFNAME -home-port PORT -home-node NODE-A
net int modify -vserver VSERVER -lif LIFNAME -status-admin up
set adv
net port modify -node NODE-A -port PORT -up-admin down
net port modify -node NODE-A -port PORT -up-admin up
 
10) [Node A] Complete giveback
 
storage failover giveback -ofnode NODE-A
storage failover show-giveback
 
For NAS LIFs: Rehome back to NODE-A:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
11) Take over Node B
 
Card 5 had NAS LIFs: Temporarily rehome Node B’s LIFs to Node A:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-A -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
Card 5 had SAN LIFs: Leave for now.
 
Takeover Node B:
 
storage failover takeover -ofnode NODE-B
 
When the output says ‘Hit [Enter] to boot immediately, or any other key for command prompt.’ press any other key. When the system is at the LOADER> prompt it is safe to continue.
 
12) [Node B] Remove from the chassis
13) [Node B] Swap IO card in slot 5 into slot 1 (check HWU)
14) [Node B] Insert X1148A NVMe Storage card into slot 5
15) [Node B] Unplug e0d storage connection
16) [Node B] Boot and perform partial giveback
 
From Node A, perform a partial giveback:
 
storage failover giveback -ofnode NODE-B -only-cfo-aggregates true
storage failover show-giveback
 
17) [Node B] Configure slot 5 storage ports
 
storage port enable –node NODE-B –port PORT
 
18) [Node B] Old e0d storage connection connects to e5b
19) [Node B] Perform necessary re-configurations for slot 1 IO card
 
For NAS LIFs: Rebuild the IFGRPs/Ports/VLANs and perform clean up.
 
For SAN LIFs:
 
net int modify -vserver VSERVER -lif LIFNAME -status-admin down
net int modify -vserver VSERVER -lif LIFNAME -home-port PORT -home-node NODE-B
net int modify -vserver VSERVER -lif LIFNAME -status-admin up
set adv
net port modify -node NODE-B -port PORT -up-admin down
net port modify -node NODE-B -port PORT -up-admin up
 
20) [Node B] Complete giveback
 
storage failover giveback -ofnode NODE-A
storage failover show-giveback
 
For NAS LIFs: Rehome back to NODE-B:
 
net int modify -vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert -vserver VSERVER -lif LIFNAME
 
21) [NS224 Shelf 2] Set shelf ID
22) [NS224 Shelf 2] Hot-add using node ports e5a (path A) and e0d (path B)
 
node run local storage show disk -p
node run !local storage show disk -p
storage failover modify -node * -auto-giveback true
storage failover show -node * -fields auto-giveback
autosupport invoke -node * -type all -message "MAINT=END"
timeout modify -timeout 30
 
THE END
 
Also see:
https://examcramnotes.blogspot.com/2021/01/netapp-aff-a400-max-ns224-shelves-in.html

No comments:

Post a Comment

Note: only a member of this blog may post a comment.