It's been a while (a good 3 years) since I did much StorageGRID, so I thought I'd take a few notes down.
- StorageGRID Recommended Release: A reasonably risk-averse policy is to wait one to three months to upgrade to the next major release (X.X.0) and always stay up to date with hotfixes (X.X.X.hotfix).
- StorageGRID Release Model
- As a best practice, qualify releases in your test environment prior to rollout in your production environments.
- Q: How to I find out about the latest StorageGRID hotfixes?
- Go to: NetApp StorageGRID (Downloads) and click on the Related Release button:
As of writing (13-Apr-2023, hotfix is 11.6.0.8) - Key NetApp StorageGRID links:
- StorageGRID Resources
- StorageGRID 11.6 Documentation
- Includes these sections of specific interest to me:
- SG6000 appliances
- Expand your grid
- Important: When you're adding new nodes to the GRID be sure to check the settings on the old nodes (i.e. defaults - like for RAID type DDP - may not have been used - see: How to identify the storage RAID mode configuration of a StorageGRID storage node appliance)
- GRID expansion is quick but removal very very slow!
- You can add Storage Nodes to more than one site in a single expansion procedure.
- Before starting run: Support > Diagnostics, check ALERTS and verify grid health.
- In general, when you select the ADC Service setting for a new node, you should select Automatic.
- 2+1 Erasure Coding: Using only 3 Storage Nodes is not recommended. An actual production grid should use a minimum of k+m+1 Storage Nodes for redundancy, which equals 4.
- If you cannot add the recommended number of Storage Nodes for the erasure-coding scheme you are using, you might need to run the EC rebalance procedure to allow additional erasure-coded objects to be stored.
- EC rebalancing is a StorageGRID procedure that might be required after a Storage Node expansion. The procedure is run as a command-line script from the primary Admin Node
- When the EC rebalance procedure is running, the performance of ILM operations and S3 and Swift client operations are likely to be impacted. For this reason, you should only perform this procedure in limited cases.
- See: Requirements for EC Re-balancing
- Gather required materials:
- Recovery Package (contains passwords.txt to access grid nodes on the CLI)
- Provisioning passphrase
- Add grid nodes to existing site or add new site:
- Prereqs: You have Root access or Maintenance permission, all existing nodes are up, and no maintenance procedures are in progress.
- Update subnets for Grid Network as required.
- Deploy new grid nodes (Appliance):
- To install the StorageGRID software on an appliance node, you use the StorageGRID Appliance Installer, which is included on the appliance.
- Example: Configure hardware (SG6000)
- Version 3.6 of the StorageGRID Appliance Installer is compatible with version 11.6 of StorageGRID. (If the versions are not compatible, you must upgrade the StorageGRID Appliance Installer firmware.)
- Perform expansion
- Configuration steps after expansion
- Review ILM (specifically storage pools)
- Verify that the Storage Node is ingesting objects
- Rebalance erasure-coded data after adding Storage Nodes
- kb.netapp.com - StorageGRID
- StorageGRID hardware models in NetApp Hardware Universe (13-Apr-2023):
- Models: SG5612, SG5660, SG5712, SG5712X, SG5760, SG5760X, SG6060, SG6060X, SGF6024
- Install hardware (SG6000)
SG6060 to E2800A Connections - Configure hardware (SG6000)
- https://169.254.0.1:8443 (or .2) to access StorageGRID Appliance Installer if no DHCP.
- Services Appliance: SG100, SG1000
- Also, if you have access to Lab on Demand (NetApp) there are some labs:
- Enterprise Object Storage in the Data Fabric with StorageGRID
- Lab for StorageGRID 11.6
- Protecting ONTAP NAS with Rubrik and StorageGRID
- Cloud Tiering with FabricPool for Cost-Efficient Storage
- NetApp Data Backup and Recovery with Commvault v3.0
- Protecting Your Data in the Hybrid Cloud
- Getting Started with BlueXP Backup and Recovery
PS Recommend running the diagnostics before and after the expansions:
Support > Tools > Diagnostics
APPENDIX A: Other Links
- Manage objects with ILM: Overview
- Check data repair jobs
- Monitor and troubleshoot: Overview
- Rebalance erasure-coded data after adding Storage Nodes
- rebalance-data start --site "site-name"
APPENDIX B: More on Deploying SG6060
- Rack and Stack
- Cable
- Power On (SG6000-CN first in the documentation, then E2860)
- SG6000-CN any warning LED? (if so troubleshoot)
- E2860s at status 99? (if not then troubleshoot)
- StorageGRID Appliance Installer (SGAi):
- https://169.254.0.1:8443 (or .2)
- SGAi: Advanced > Upgrade Firmware (is it correct? i.e. 3.6 for StorageGRID 11.6)
- SGAi: Configure Networking > Link Configuration
- SGAi: Configure Networking > IP Configuration
- SGAi: Configure Networking > Ping and MTU Test
- SGAi: Configure Networking > Port Connectivity Test
- SGAi: Advanced Tab to Configure SANtricity (don't need to connect to E-series)
- Configure Hardware > Storage Controller Network Connection
- or E-Series Mgmt Port > Advanced > System Manager > Set Admin Password
- IMPORTANT: Cancel the Wizard (Do not set up E-series in SANtricity!)
- Hardware Alerts
- AutoSupport: Support > Support Center
- SGAi: Configure Hardware > BMC Configuration
- Change root p/w
- Set IP
- Then deploy as a Storage Node using the SGA.
- To get updated license file go to:
- https://register.netapp.com/register/getlicensefile
- Verify:
- Configuration and health (including subnets)
- ActiveIQ (AutoSupport) configuration in SANtricity
APPENDIX C: Default IPs for E-Series without DHCP
Not sure if this is useful information but adding here for completeness.
- 192.168.128.101 / 102 A ports
- 192.168.129.101 / 102 B ports
Note: This might be old information as original posted here back in 2016: My E-Series and EF-Series Notes & Learnings
APPENDIX D: StorageGRID Expansion - with SGA - Checklist
Step # Status # Task # Notes/Questions
####################################################
i # - # Check Grid Version # Does it need upgrade?
ii # - # Check RAID Levels # DDP (default) or DDP-16
iii # - # PGE Upgrade # If required
iv # - # SANtricity Upgrade # If required
v # - # Check SGAi Config # IPs/Names/etc.
vi # - # Provisioning Passphrase # Required
vii # - # Download recovery files # Required
viii # - # Run pre-expand diags #
1 # - # Deploy new nodes #
2 # - # Perform expansion #
ix # - # Run post-expand diags #
APPENDIX E: StorageGRID Expansion - with SGA - Checklist Version 2 (Expanded)
The following later checklist was created for an SG6060 deployment (StorageGRID expansion). Actually, I created a spreadsheet with a column for each node to be added (you add all new nodes at once), so we could verify each node is ready for GRID expansion.
- GRID: Check GRID version. Does the GRID need upgrade (for instance - to fix any known expansion bugs)?
- GRID: Are we using DDP (default) or DDP-16 (DDP-16 gives more capacity - the appliance may have been sold with DDP-16 capacity in mind)?
- Complete rack and stack, cabling, power-on, and configure basic configuration so we can get to the StorageGRID Appliance Installer (SGAi)
- Access SGAi on https://SGA_ADMIN_IP:8443 or https://SGA_GRID_IP:8443
- SGAi: Configure Networking > Link Configuration
- SGAi: Configure Networking > IP Configuration
- SGAi: Configure Networking > Ping and MTU Test (ping Primary Admin Node)
- SGAi: Configure Networking > Port Connectivity Test
- SGAi: Advanced Tabe to Configure SANtricity (do not need to connect to E-Series) > Controller Hardware > Storage Controller Network Connection
- E-Series: Advanced > System Manager > Set Admin Password
- IMPORTANT: Cancel the wizard (DO NOT SET UP E-Series in SANtricity!)
- E-Series: AutoSupport: Support > Support Center
- E-Series: Confirm all connections to StorageGRID controller are set up.
- E-Series: Upgrade SANtricity Version (if required - check in the IMT for supported versions)
- E-Series: Upgrade Disk Firmware (if required)
- SGAi: Check RAID level is correct (DDP by default)
- SGAi: Configure Hardware > BMC Configuration > Change root p/w & set IP
- Verify correct number of disks and no failed disks
- Then ready to deploy as a Storage Node using the SGAi
Once the above is complete:
- Obtain Provisioning Pasphrase
- Dowload recovery files
- Run pre-expand diags in Grid Manager: Support > Diagnostics
- Deploy new nodes (using SGAi)
- Perform expansion (once all nodes deployed and are pending in Grid Manager)
- Run post-expand diags in Grid Manager: Support > Diagnostics
- Obtain updated license: https://register.netapp.com/register/getlicensefile
APPENDIX F: Assorted Links
https://imt.netapp.com/matrix - Check supported SANtricity OS with StorageGRID Appliance
APPENDIX T: Troubleshooting
- Ping result from Installer GUI using Admin network:
- The ping using Admin network will only be successful to subnet(s) that you defined under Admin network subnet list.
- The ping is successful from PGE is because you did not specify Admin network interface br1 and ping is using the node default Gateway (either on Grid network or client network if configured.
- NetApp KB:
Comments
Post a Comment