Wednesday, 4 January 2012

How to Configure iSCSI Multipathing (MPIO) in VMware vSphere 5 using ESXCLI

UPDATE: If you want just one iSCSI Switch with multiple iSCSI VMkernels in, please see How to Configure iSCSI Multipathing (MPIO) in VMware vSphere 5 using ESXCLI (UPDATE)

The script below will enable the iSCSI Software Adapter, create two standard switches each with one active network adapter, configure each switch with a configured VMkernel port, bind the iSCSI VMkernels to the iSCSI Software Adapter vmhba, add a send target portal, set the multi-pathing to default to round-robin, and then perform a rescan for storage devices and new volumes. Testing in the lab had a complete VMware Host iSCSI and MPIO configuration applied in less that 20 seconds!
The easiest way to run this script is to establish an SSH connection to the ESXi 5 host to be configured using Putty or similar (first start the SSH service on the host using a vSphere Client connection.) Then simply copy the script and right-click to paste into the Putty window - the script will run.

Notes:

1: In practice you will probably want to run part 1 separately to enable the iSCSI Software Adapter, so can provide the iSCSI Initiator name to input into the SAN management software to allow access to volumes, and also to check the vmhbaXX number.
2: Substitute the following values as per requirements: vmnic4, vmnic5192.168.0.111, 255.255.255.0, 192.168.0.121, 255.255.255.0, vmhba33, 192.168.0.100:3260
3: The line in Part 6 works for HP LeftHand/P4000 iSCSI SANs. If you want the multi-pathing to default to round-robin for devices not detected as VMW_SATP_DEFAULT_AA, then this line will need to be modified.
4: It is not currently possible with ESXCLI to create just one Standard Switch for iSCSI and add multiple VMkernels in, as there is no command to set an adapter as unused (only active and standby can be configured using ESXCLI.)
5: See http://pubs.vmware.com/vsphere-50/index.jsp for the full vSphere Command-Line Interface Reference.
6: The lines beginning with 'echo' are just there for cosmetic reasons to explain what the line underneath does
7: It is pretty straightforward to modify the script for more than 2-way MPIO (vSphere 5 supports up to 8-way MPIO.)

The Script:

echo # START COPYING ON THIS LINE!
echo #########################
echo # PART 1: Enable iSCSI Software Adapter
esxcli iscsi software set -e on
echo #########################
echo # PART 2: Create 1st iSCSI vSwitch and iSCSI VMkernel
echo ## 2.1 Add a new Standard Switch called iSCSIVMk1
esxcli network vswitch standard add -v iSCSIVMk1
echo ## 2.2 Add vmnic4 to the iSCSIVMk1 Standard Switch
esxcli network vswitch standard uplink add -u vmnic4 -v iSCSIVMk1
echo ## 2.3 set vmnic4 as active adapter for the iSCSIVMk1 Standard Switch iSCSIVMk1
esxcli network vswitch standard policy failover set -a vmnic4 -v iSCSIVMk1
echo ## 2.4 Add a portgroup called iSCSIVMk1 to the iSCSIVMk1 Standard Switch
esxcli network vswitch standard portgroup add -p iSCSIVMkernel1 -v iSCSIVMk1
echo ## 2.5 Add a VMkernel interface called vmk11 to the iSCSIVMk1 portgroup
esxcli network ip interface add -i vmk11 -p iSCSIVMkernel1
echo ## 2.6 configure vmk11 with IP Address and Subnet Mask
esxcli network ip interface ipv4 set -i vmk11 -I 192.168.0.111 -N 255.255.255.0 -t static
echo #########################
echo # PART 3: Create 2nd iSCSI vSwitch and iSCSI VMkernel
echo ## 3.1 Add a new Standard Switch called iSCSIVMk2
esxcli network vswitch standard add -v iSCSIVMk2
echo ## 3.2 Add vmnic5 to the iSCSIVMk2 Standard Switch
esxcli network vswitch standard uplink add -u vmnic5 -v iSCSIVMk2
echo ## 3.3 set vmnic5 as active adapter for the iSCSIVMk2 Standard Switch iSCSIVMk2
esxcli network vswitch standard policy failover set -a vmnic5 -v iSCSIVMk2
echo ## 3.4 Add a portgroup called iSCSIVMk2 to the iSCSIVMk2 Standard Switch
esxcli network vswitch standard portgroup add -p iSCSIVMkernel2 -v iSCSIVMk2
echo ## 3.5 Add a VMkernel interface called vmk12 to the iSCSIVMk2 portgroup
esxcli network ip interface add -i vmk12 -p iSCSIVMkernel2
echo ## 3.6 configure vmk12 with IP Address and Subnet Mask
esxcli network ip interface ipv4 set -i vmk12 -I 192.168.0.121 -N 255.255.255.0 -t static
echo #########################
echo # PART 4: Bind the iSCSI VMkernels to the iSCSI Software Adapter vmhba33
esxcli iscsi networkportal add -A vmhba33 -n vmk11
esxcli iscsi networkportal add -A vmhba33 -n vmk12
echo #########################
echo # PART 5: Add send target portal for iSCSI SAN with IP address 192.168.0.100
esxcli iscsi adapter discovery sendtarget add -A vmhba33 -a 192.168.0.100:3260
echo #########################
echo # PART 6: Set the multi-pathing to default to round-robin
esxcli storage nmp satp set -P VMW_PSP_RR -s VMW_SATP_DEFAULT_AA
echo #########################
echo # PART 7: Perform a rescan for storage devices and new volumes
esxcli storage core adapter rescan -a
esxcli storage filesystem rescan
echo # FINISH COPYING ON THIS LINE!

4 comments:

  1. Doesn't this set only one of the pnics as active?:

    esxcli network vswitch standard portgroup policy failover set -p "iSCSIVMkernel1" -a "vmnic4"

    esxcli network vswitch standard portgroup policy failover set -p "iSCSIVMkernel2" -a "vmnic5"

    ReplyDelete
  2. Hello Anonymous, excellent question and yes you are 100% correct. The command -

    esxcli network vswitch standard portgroup policy failover set -p iSCSIVMkernel1 -a vmnic4

    - will indeed set the pnic as active for the VMkernel in question, but since we have just one VMkernel in the vSwitch, we get the desired effect by applying the command -

    esxcli network vswitch standard policy failover set -a vmnic4 -v iSCSIVMk1

    - to the switch, which we would have to do anyway (otherwise, even though the adapter is active for the VMkernel, would only see as standby for the vSwitch in the GUI, which would look a bit odd.)
    There is no command to set an adapter as unused via CLI, only active or standby are possibilities, hence the one active adapter per vSwitch method is used.
    Cheers!

    ReplyDelete
  3. Hi,

    To only mark 1 iface as active on a portgroup, first specify all interfaces as active, than specify only the one you need.
    When you leave out any interface from the active list it will be marked unused (except when it is in standby, so mark both active to emtpy the standby adapters list (if any) )


    to only have vmnic4 active on your iSCSIVMkernel1:

    esxcli network vswitch standard portgroup policy failover set -p iSCSIVMkernel1 -a vmnic4,vmnic5

    esxcli network vswitch standard portgroup policy failover set -p iSCSIVMkernel1 -a vmnic4

    ReplyDelete
    Replies
    1. Hello Stefan, thank you very much for the comment and education. So that is how you do it then. Cheers!

      Delete