Setting the Scenario
You have a NetApp AFF A400 with 1 external NS224 shelf.
You already have an IO card in slot 5 (say for serving
ethernet or fibre channel data).
You need to add a 2nd NS224 shelf.
High-Level Overview
The NS224 with NSM100 shelf modules has a max stack size
of 1 (see HWU - hwu.netapp.com.) This means that in order to add a 2nd NS224
shelf to the A400, you need an X1148A NVMe Storage card (assuming you
don’t have one already) and this must go in slot 5 following the ‘Priority
Slot Assignment’ on HWU.
Starting point is one NS224 shelf cabled as:
Shelf|NSM|Shelf|Node
|Node
| |Port
|Port |
-----+---+-----+-----+----
01 | A | e0a | e0c | A
01 | B | e0a | e0c | B
01 | B | e0b | e0d | A
01 | A | e0b | e0d | B
Where:
- NSM A and Node A are the topmost NSM/Node.
- NSM B and Node B are the bottommost NSM/Node.
Image: NetApp AFF A400 cabled with one NS224 shelf
The process to add the 2nd shelf is:
1) Take over Node A
2) [Node A] Remove from the chassis
3) [Node A] Swap IO card in slot 5 into slot 1 (check HWU)
4) [Node A] Insert X1148A NVMe Storage card into slot 5
5) [Node A] Unplug e0d storage connection
6) [Node A] Boot and perform partial giveback
7) [Node A] Configure slot 5 storage ports
8) [Node A] Old e0d storage connection connects to e5b
9) [Node A] Perform necessary re-configurations for slot
1 IO card
10) [Node A] Complete giveback
11) Take over Node B
12) [Node B] Remove from the chassis
13) [Node B] Swap IO card in slot 5 into slot 1 (check HWU)
14) [Node B] Insert X1148A NVMe Storage card into slot 5
15) [Node B] Unplug e0d storage connection
16) [Node B] Boot and perform partial giveback
17) [Node B] Configure slot 5 storage ports
18) [Node B] Old e0d storage connection connects to e5b
19) [Node B] Perform necessary re-configurations for slot
1 IO card
20) [Node B] Complete giveback
21) [NS224 Shelf 2] Set shelf ID
22) [NS224 Shelf 2] Hot-add using node ports e5a (path A)
and e0d (path B)
Finishing point is 2 NS224 shelves, cabled as:
Shelf|NSM|Shelf|Node
|Node
| |Port
|Port |
-----+---+-----+-----+----
01 | A | e0a | e0c | A
01 | B | e0a | e0c | B
01 | B | e0b | e5b | A
01 | A | e0b | e5b | B
02 | A | e0a | e5a | A
02 | B | e0a | e5a | B
02 | B | e0b | e0d | A
02 | A | e0b | e0d | B
Note: e5b (2) is to the left, and e5a (1) is to the
right in the below diagram.
Image: NetApp AFF A400 cabled with one NS224 shelf
End-to-end, this process shouldn’t take longer than 4
hours (perhaps even less than 2 hours.)
Low-Level Commands
The low-level commands used in the operation will be
bespoke to the particular system (for instance: the card originally in slot 5
might home SAN LIFs, or it might home NAS LIFs.) The below is a rough guide.
1) Take over Node A
Send AutoSupport and disable auto-giveback:
timeout
timeout modify
-timeout 0
autosupport
invoke -node * -type all -message "MAINT=4h Moving IO cards 1 to 5 and adding
a shelf"
storage failover
show -node * -fields auto-giveback
storage failover
modify -node * -auto-giveback false
storage failover
show -node * -fields auto-giveback
Card 5 had NAS LIFs: Temporarily rehome Node A’s LIFs to
Node B:
net int modify
-vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert
-vserver VSERVER -lif LIFNAME
Card 5 had SAN LIFs: Leave for now.
Takeover Node A:
storage failover
takeover -ofnode NODE-A
When the output says ‘Hit [Enter] to boot immediately, or
any other key for command prompt.’ press any other key. When the system is at
the LOADER> prompt it is safe to continue.
2) [Node A] Remove from the chassis
3) [Node A] Swap IO card in slot 5 into slot 1 (check HWU)
4) [Node A] Insert X1148A NVMe Storage card into slot 5
5) [Node A] Unplug e0d storage connection
6) [Node A] Boot and perform partial giveback
From Node B, perform a partial giveback:
storage failover
giveback -ofnode NODE-A -only-cfo-aggregates true
storage failover
show-giveback
7) [Node A] Configure slot 5 storage ports
storage port
enable –node NODE-A –port PORT
8) [Node A] Old e0d storage connection connects to e5b
9) [Node A] Perform necessary re-configurations for slot
1 IO card
For NAS LIFs: Rebuild the IFGRPs/Ports/VLANs and perform
clean up.
For SAN LIFs:
net int modify -vserver
VSERVER -lif LIFNAME -status-admin down
net int modify -vserver
VSERVER -lif LIFNAME -home-port PORT -home-node NODE-A
net int modify -vserver
VSERVER -lif LIFNAME -status-admin up
set adv
net port modify -node
NODE-A -port PORT -up-admin down
net port modify -node
NODE-A -port PORT -up-admin up
10) [Node A] Complete giveback
storage failover
giveback -ofnode NODE-A
storage failover
show-giveback
For NAS LIFs: Rehome back to NODE-A:
net int modify
-vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert
-vserver VSERVER -lif LIFNAME
11) Take over Node B
Card 5 had NAS LIFs: Temporarily rehome Node B’s LIFs to
Node A:
net int modify
-vserver VSERVER -lif LIFNAME -home-node NODE-A -home-port PORT
net int revert
-vserver VSERVER -lif LIFNAME
Card 5 had SAN LIFs: Leave for now.
Takeover Node B:
storage failover
takeover -ofnode NODE-B
When the output says ‘Hit [Enter] to boot immediately, or
any other key for command prompt.’ press any other key. When the system is at
the LOADER> prompt it is safe to continue.
12) [Node B] Remove from the chassis
13) [Node B] Swap IO card in slot 5 into slot 1 (check HWU)
14) [Node B] Insert X1148A NVMe Storage card into slot 5
15) [Node B] Unplug e0d storage connection
16) [Node B] Boot and perform partial giveback
From Node A, perform a partial giveback:
storage failover
giveback -ofnode NODE-B -only-cfo-aggregates true
storage failover
show-giveback
17) [Node B] Configure slot 5 storage ports
storage port
enable –node NODE-B –port PORT
18) [Node B] Old e0d storage connection connects to e5b
19) [Node B] Perform necessary re-configurations for slot
1 IO card
For NAS LIFs: Rebuild the IFGRPs/Ports/VLANs and perform
clean up.
For SAN LIFs:
net int modify -vserver
VSERVER -lif LIFNAME -status-admin down
net int modify -vserver
VSERVER -lif LIFNAME -home-port PORT -home-node NODE-B
net int modify -vserver
VSERVER -lif LIFNAME -status-admin up
set adv
net port modify -node
NODE-B -port PORT -up-admin down
net port modify -node
NODE-B -port PORT -up-admin up
20) [Node B] Complete giveback
storage failover
giveback -ofnode NODE-A
storage failover
show-giveback
For NAS LIFs: Rehome back to NODE-B:
net int modify
-vserver VSERVER -lif LIFNAME -home-node NODE-B -home-port PORT
net int revert
-vserver VSERVER -lif LIFNAME
21) [NS224 Shelf 2] Set shelf ID
22) [NS224 Shelf 2] Hot-add using node ports e5a (path A)
and e0d (path B)
node run local
storage show disk -p
node run !local
storage show disk -p
storage failover
modify -node * -auto-giveback true
storage failover
show -node * -fields auto-giveback
autosupport
invoke -node * -type all -message "MAINT=END"
timeout modify
-timeout 30
THE END
Also see:
https://examcramnotes.blogspot.com/2021/01/netapp-aff-a400-max-ns224-shelves-in.html
Comments
Post a Comment