Saturday, 31 July 2010

LeftHand/Storageworks SAN/iQ NSM Fails to Boot with Non-System disk error

Scenario: A brand new HP Storageworks P4500 G2 network storage module (NSM) fails to boot out of the box.


The module cycles on


Attempting Boot From CD-ROM
Attempting Boot From Hard Drive (C:)
Attempting Boot From NIC

Non-System disk or disk error
Replace and strike any key when ready

Press “F9” key for ROM-Based Setup Utility
Press “F10” key for System Maintenance Menu

System will automatically reboot in 4_seconds



So you check all the usual suspects:


No disks in drives
Boot controller order is correct
RAID controller is seeing 3 logical drives as per usual
And no disks are showing amber or red indicator lights



The only answer is to re-image the NSM, and this is very easy:



1) Dig out the SAN/iQ Quick Recovery CD which came with the NSM and place in the CD-drive

2) Obtain the Feature Key (this can be obtained via the iLO as it will be the NIC MAC address with the lowest numerical value)

3) Obtain the License Key (if the module is brand new, this is obtained by logging into https://webware.hp.com and generating a license with the entitlement number from the entitlement certificate that came with the NSM, and the Feature Key obtained above; otherwise hopefully a record of it will have been kept, if not - and there is no record of the HP Passport account the License Key was generated with - only option is to speak to HP Licensing)

4) On a USB key create a blank file with the name featurekey_MACADDRESSWITHNOCOLONSIN and inside this file (edit with notepad or whaterver) paste the License Key

Example:

File name = featurekey_1CC1DE02CAEC
File contents = 0426-1219-FC96-8958-DF52-145C-8E7C-EF42-2B74-B2CD-AAD0-D231-5D4D-65C2-227B-F177-BB56-6161-CDCB-F17A-689F-2A19-1EC9-8EE4-3EAE-53EE-814A-FCF0-A779-F4C4-2F3E-D99D-AC

5) Then boot the NSM with the Recovery CD and USB key in and it will go away and re-image itself



If there is data on the module that needs to be recovered, then the above would not be an option, as it wipes the module and places on a new image.


Note 1: Can put the USB key in later after booting off the CD when prompted; also – if do not have a USB key to hand – can manually input the License Key.


Note 2: The above scenario is very unlikely, and has happened to me only once in many LeftHand/Storageworks SAN/iQ installs.


Note 3: This procedure was done for an NSM with SAN/iQ 8.5 Recovery CD.

5 comments:

  1. Hi,

    I have recently attemptd to run through this reinstall/recover procedure after an upgrade gone bad and keep getting invalid hardware type id[9] after it start the installer following it stating valid license found on usb. Do you any ideas?

    Thanks,

    Dave
    david.heyman@kisingercampo.com

    ReplyDelete
  2. Hi David

    The error appears to suggest that perhaps the firmware is out of kilter with the original build, or maybe the NSM has been modified with something like a disk expansion in the past.

    1) Could try the Quick Recovery without any license key if it will let you.

    2) Could try downgrading the firmware – I remember LeftHand is a bit funny about firmware, and the firmware updates get applied after software updates, also normally only the RAID controller firmware ever gets upgraded – start with RAID controller firmware, and then looking at other firmwares.

    3) Could try the latest SAN/iQ Quick Recovery CD from http://www.hp.com/go/P4000downloads if your NSM supports this, or might need to contact LeftHand Support and see if can get the highest level SAN/iQ Quick Recovery CD for the NSM in question ( (0)800 633 3600 option 1, option 2 - then at verbal prompt say ‘left hand solutions’ )

    4) If the NSM has had a disk upgrade, could try finding disks the same size as the originals and go from there.

    5) A bit risky – if you have a known good NSM of the same type that you can use, could take all the disks out of the faulty one, put it in the good one, and run the Quick Recovery onto that (without using the license key if it will let you), then bring the disks back again, and if it works apply the key; or just temporarily swap in the RAID controller from the known good one.

    If nothing works and LeftHand support cannot come up with a fix, there’s always the option to install some other iSCSI storage software on it – like Openfiler, Open-E, Nexenta (can also do CIFS, NFS,) … - or install ESXi and stick a LeftHand VSA on there.

    To be honest I’m just clutching at straws really. If you find a fix please let me know.

    Cheers

    Vidad

    ReplyDelete
  3. DELAYED UPDATE FROM VIDAD
    From: David Heyman

    We have 3 old NSM 160’s that are lab units but nsm 2120’s ,P4500’s and VSA’s in production and the issue at the moment is with one of the NSM 160’s which is out of support that I attempted to upgrade to 8.5 from 8.1 as suggested by the automated CMC upgrade routine. No disk expansion and the raid controller is at the same revision as the rest, I also tried 2 different recovery install cd’s one with version 8 and the other with version 9. I tried swapping disks with another unit but not raid controllers. What I am doing currently but do not have the results yet is I made a disk image of one the working units DOM with ghost and then pushed it to the non-working unit and it does boot to SAN/iQ but I have not tried to reconfigure it yet but will be trying this afternoon. I ran into a mishap where the little power connector that goes from the motherboard to the DOM broke off while being removed and so have been looking around for a new power cable.

    Thanks, Dave

    ReplyDelete
  4. DELAYED UPDATE FROM VIDAD PART 2
    From: Vidad Cosonok

    Hi Dave
    How did you get on with this?
    Regards, Vidad

    ReplyDelete
  5. DELAYED UPDATE FROM VIDAD PART 3
    From: David Heyman

    It went fine as far as imaging a known working DOM and loading the image onto the one that was rendered non-working by the failed 8.5 upgrade. The issue I ran into was that on the secondary DOM I broke the little power connector wire while taking the DOM’s in and out of the ide slot many times while testing and re-imaging multiple times. I first dumped the same image to both DOM’s and then when booting it threw up a panic. So I imaged and loaded the primary and secondary DOM’s individually and that was seeming to work until the little flimsy power cord broke. Now I am searching for somewhere to get a new one. It is a little 3 pin to 2 pin cable and I do not even know what it is referred to as cable type wise. I have also seen some DOM’s that use a molex to 2 pin power cable so I may try to find one of those and see if it works.
    Thanks for the follow-up, Dave

    ReplyDelete