I was originally
planning to write a ‘StorageGRID Webscale Primer’ blog post, but in doing my
research I found there is an official and very good ~70 page “Grid Primer” (see
StorageGRID
Webscale 10.3 documentation) which you should read; hence this post became
brief notes and links instead. Current (as at 2017.01.31) latest version of
StorageGRID Webscale is 10.3.0.
0) Contents
1) Interoperability
Matrix Tool Results
2) Software and
Documentation Links
3) Brief Notes on
SGW
1) Interoperability Matrix Tool Results
http://mysupport.netapp.com/matrix/#welcome
> Component Search > StorageGRID
StorageGRID
Webscale 10.3 supported on/with:
Hypervisor (OS):
VMware ESXi 6.0u1 / 6.0 / 5.5u2 / 5.5u1 / 5.5 / 5.1u3 /
5.1u2 / 5.1u1 / 5.1 / 5.0u3 / 5.0u2 / 5.0u1 / 5.0 & OpenStack Kilo
TSM Client
(Backup)
IBM Tivoli Storage Manager 6.4.1
Protocol:
SMB 2.1, SMB 2.0, NFS v3.0, HTTPS
Appliance (Storage
Node):
StorageGRID Appliance 5660/5612
StorageGRID NAS
Bridge:
Storage GRID NAS Bridge 2.0.1
API:
Swift 1.0, S3 2006-03-01, CDMI v1.02, CDMI v1.01
2) Software and Documentation Links
StorageGRID
Webscale download link:
StorageGRID
Webscale 10.3.0 download link:
This link contains
links to:
- StorageGRID-Webscale-10.3.0-20160818.2333.94beb49.tgz
- SGW 10.3 Appliance
Installation and Maintenance Guide
- SGW 10.3 Software
Installation Guide for OpenStack Deployments
- SGW 10.3 Software
Installation Guide for VMware Deployments
Note: The download
is 3.4GB
StorageGRID
Webscale documentation link:
StorageGRID
Webscale 10.3 documentation link:
This link contains
includes links to:
- Release Notes
for StorageGRID Webscale 10.3.0
- Release Notes
for StorageGRID Webscale NAS Bridge 2.0.1
- Administrator
Guide
- Appliance
Installation and Maintenance Guide
- Audit Message
Reference
- Cloud Data
Management Interface Implementation Guide
- Expansion Guide
for OpenStack Deployments
- Expansion Guide
for VMware Deployments
- Grid Primer
- Maintenance
Guide for OpenStack Deployments
- Maintenance
Guide for VMware Deployments
- NAS Bridge 2.0
Administration Guide
- NAS Bridge 2.0
Installation and Setup Guide
- NAS Bridge 2.0
Management API Guide
- Simple Storage
Service Implementation Guide
- Software
Installation Guide for OpenStack Deployments
- Software
Installation Guide for VMware Deployments
- Software Upgrade
Guide
- Swift
Implementation Guide
- Troubleshooting
Guide
StorageGRID
Webscale:
Field Portal:
Also, don’t forget to check out NetApp University (learningcenter.netapp.com)
and Lab on Demand (labondemand.netapp.com) if you have access.
3) Brief Notes on SGW
Introduction to
the StorageGRID Webscale System
- SGW is a massively scalable, software-defined object-based
storage solution for media-intensive workloads in the form of video, images,
and PDF documents
- SGW has 10+ years of production object-storage
deployments
- SGW has the industry’s most advanced policy framework
for data lifecycle management
- SGW has a distributed storage architecture
- SGW has true geographically distributed and
geographically selective object placement
- SGW offers the flexibility for customers to choose
their storage hardware
- SGW also allows tape or public cloud as an active tier
(object-granularity retrieval from tape and cloud)
- An object-based storage system does not organize
objects in a hierarchal structure.
- What is object-based storage? Think of the Valet
Parking Analogy...
- Typical use cases:
-- Web Data Repositories
-- Data Archives
-- Media Repositories
Deployment
Topologies, Grid Nodes, and Services
- Single Data Center Site (Grid Nodes and Primary Admin Node
in one Data Center Site)
- Multiple Data Center Sites (Multiple Datacenters, each
with Grid Nodes and Primary Admin Node)
- Grid Nodes:
-- Admin
Node
-- API Gateway Node (Optional)
-- Storage
Node
-- Archive Node (Optional)
- Admin Node
services:
-- NMS:
Network Management System
-- CMN:
Configuration Management Node
-- AMS:
Audit Management System
-- SSM:
Server Status Monitor
- Storage Node
services:
-- LDR:
Local Distribution Router
-- DDS:
Distributed Data Store
-- ADC:
Administrative Domain Controller
-- CMS:
Content Management System
-- SSM:
Server Status Monitor
Note: Each Data
Center Site should have - at a minimum - 3
Storage Nodes.
- API Gateway Node
services:
-- CLB:
Connection Load Balancer
-- SSM:
Server Status Monitor
- Archive Node
services:
-- ARC:
The Archive service
-- SSM:
Server Status Monitor
- For storage and retrieval operations, Archive Nodes can
be configured to interface with either Amazon Simple Storage Service (S3) or
Tivoli Storage Manager (TSM)
Data Management
- A ‘Storage Pool’ is a logical grouping of storage media
- Metadata is the data that describes the object data
that is stored in a StorageGRID Webscale system
- Metadata is used to create the matching criteria for
ILM rule filters
- An ILM rule is not evaluated until it is activated in
the ILM policy configuration
- An Information Lifecycle Management (ILM) rule is a
description of when and where to store objects, how many copies are stored, and
for how long
- ‘Storage Grade’ is the type of storage media
- Erasure coding is a data protection method that is best
applied to objects larger than 1MB
- An erasure-coded copy contains portions of object data
(data fragments) and information for reconstructing object data (parity
fragments)
Data Flow
- SGW supports the APIs Swift, S3 and CDMI
- SGW supports standard RESTful HTTP protocols and interfaces
- SGW delivers NFS and SMB
- When a client application sends a delete request to the
StorageGRID Webscale system, a content handle is released. However, if no ILM
rule exists that requests the purge of the object, the object is not deleted
permanently.
Information
Lifecycle Management: Key Concepts
Storage
Pool = A storage pool is a logical grouping of Storage Nodes or Archive
Nodes. These logical groupings can be located at a single data center site or
spread across multiple sites.
Link
Cost Group = Link cost refers to the relative cost of communication
between data center sites within the StorageGRID Webscale system. Link cost is used
to determine which grid nodes should provide a requested service.
Replication
= A data protection method that duplicates complete instances of object data
and stores the data on distributed storage pools. Replication is best suited
for small or frequently accessed objects.
Erasure
Coding = A data protection method that protects object data from loss by
splitting an object into data fragments. Erasure coding is best suited for
objects that average more than 1 MB in size.
ILM
Rule = A set of instructions for placing objects in the StorageGRID
Webscale system over time. The rules defines when, where, and how to store
objects, how many copies to store, and for how long by evaluating object
metadata.
ILM
Policy = A set of prioritized ILM rules. Multiple ILM rules can be
configured to manage content placements for various types of objects. The
logical order of ILM rules is defined by an ILM policy.
Network Management
System
- CLB
HTTP Port 8080
- CLB
S3 Port 8082
- LDR
HTTP Port 18080
- LDR
S3 Port 18082
- On a grid node, the ‘Resources’ component under the SSM
service provides the IP address of the grid node
- Under Grid Configuration, select the ‘Storage’
component to view the port numbers
- With S3 API, certificates and security partitions is
the mechanism for controlling access to objects
- With CDMI, certificates and security partitions, is the
mechanism for controlling access to objects
- Object metadata is always mapped to object store 0
- The key function of the SSM service is monitoring the
condition of a server and related hardware
- The SSM service is present on every Storage Node
- In the NMS MI, if a service icon is blue,
it indicates that the service is in an unknown state
- In the NMS MI, the Unknown service state overrides the Administratively
Down service state
- A global custom alarm against an attribute does not
override a custom alarm or a default alarm that was configured against this
attribute
- The Audit Management System service on the Admin node
manages audit logs
- If an object does not match the filter criteria of any
ILM rule in the active ILM policy, the content placement instructions defined
by the default ILM rule are applied to the object
- After a new ILM policy is activated, this new active
ILM policy is not applied to all previously ingested objects.
- The function of the ‘Re-evaluate Content’ button that is located on the Proposed Policy
page in the Configuration tab of the ILM Management branch, is to trigger the
SGW system to apply the active ILM policy to all objects ingested before this
ILM policy was activated.
Server Manager
Command Shell (CLI) Notes
- Noteable key presses: Ctrl+Alt+F1 and [Space], Alt+F7
/etc/init.d/servermanager
status
/usr/local/servermanager/reader.rb
/etc/init.d/cms
stop
/etc/init.d/cms
status
/etc/init.d/cms
start
/etc/init.d/ldr
restart
/etc/init.d/ldr
status
storagegrid-status
# Dynamic Status Update (Ctrl+C to escape)
/etc/init.d/servermanager
stop # stop all services (graceful)
shutdown
-h now # shutdown the node (after stopping services)
/etc/init.d/servermanager
start
/etc/init.d/servermanager
restart
Miscellaneous
- If you enable security partitions you cannot disable
them
- As of 10.x, StorageGRID is StorageGRID Webscale.
Earlier versions were just StorageGRID.
Comments
Post a Comment