Monday, 10 September 2012

How Many IPs Do I Need For My Dual Controller HA Compellent iSCSI SAN


Question: How many IP address assignments do I need for my Dual Controller HA Compellent iSCSI SAN?

Scenario: Working with the Dell Enterprise Deployment Team (EDT) on a Compellent iSCSI SAN implementation as a Solutions Integrator, and integrating with a VMware vSphere environment.

The I/O cards we have in each SC8000 controller are: 2 x dual port 10 GbE cards, and 1 x dual port 1 GbE card. The 10 GbE cards are used for iSCSI traffic to VMware hosts and external physical servers accessing the iSCSI SAN. The 1 GbE cards are used for iSCSI replication traffic.

Fig. 1: Connections to a Dual Controller HA Compellent iSCSI SAN

In the diagram below, we split the 10 GbE iSCSI NICs into Fault Domain A, and Fault Domain B. We also have have the 1 GbE iSCSI NICs as Fault Domain R (for Replication.) Also, notice we have circled the Eth0 Management connections.

Fig. 2: HA Compellent iSCSI Controllers with Fault Domains



Controller A will have one iSCSI IP in Domain A (which will “float” across the two available NICs,) one iSCSI IP in Domain B (which will similarly “float”,) and one iSCSI IP in Domain R (again “floats”); also, we have iDRAC and Eth0 Management IPs. Controller B will be similarly configured. Then Fault Domain’s A and B (the 10 GbE iSCSI) will have one Virtual IP, fault domain R (the 1 GbE replication iSCSI) will have one Virtual IP, and there will be one Virtual IP for Management.
Note: iSCSI Fault Domain’s A and B are not different subnets.

Answer: 13
Note: This is using virtual ports mode as opposed to legacy mode.

The 13 IPs required are illustrated with the table below.

Fig 3: Table for IPs required for a Dual-Controller HA Compellent iSCSI SAN solution

Final Word: As always, comments most welcome, especially since it took me a little while to get my head around this!

6 comments:

  1. Did you use a different subnet for your replication traffic? Having some weirdness this end with configuring replication on a Compellent...

    ReplyDelete
    Replies
    1. Hi Anonymous, we had a different subnet for replication traffic. One thing you might want to watch out for is that you should use legacy mode instead of virtual ports mode if there is any NAT-ing going on across you replication link. Cheers!

      Delete
    2. That could be where we're going wrong - we have two subnets at each site, and the replication traffic is on the same subnet, but on a different interface (very similar to your example with 10Gb for iSCSI and 1Gb for replication). There is no NATing going on, but we are using virtual ports mode. I may need to change the IP range we're using for iSCSI, I'll discuss this with the Compellent engineer we're liasing with. Thanks for the reply, there seems to be very little technical info regarding configuration and troubleshooting of Compellent stuff out in the wild!

      Delete
    3. One other thing - obviously your replication subnet should be pingable from both ends of the link, but could you also ping across to the internal iSCSI IPs? Surely these should be isolated?

      Delete
    4. Hi Anonymous,

      The internal iSCSI IPs are isolated - there are two subnets in each site for iSCSI fabric A and B. The replication iSCSI has its own subnet too on each site, and this is routable site-to-site.

      Cheers

      VCosonok

      Delete
    5. Cheers - Compellent dialled in and got it working today, finally I can begin testing prior to going into production with it. Thanks for your help :)

      JD

      Delete