Sunday, 2 February 2014

Rebuilding a CF Card on a FAS3140A that was Out-of-Sorts

This was quite an interesting experience, so - seeing as I like to share things - here’s the story…

Firstly, you might be wondering why in the title the FAS3140A is Out-of-Sorts; well - without going into too much detail - there’d been a slight problem with an upgrade from 7.3.3 to 8.1.3 the previous weekend and one head was on 8.1.3, and the other (the head I was interested in fixing) was failing to boot - NetBoot-ing to 8.1.3 or back to 7.3.3 had been tried without success, so an engineer was required on site to fix it. And if you’re wondering why a week went by with one head in takeover of the other, well, it was a pre-prod system, and the fix is a little disruptive (and there was some other upgrade work to be done after fixing this little issue…)

I arrived on site armed with my Transcend Compact Flash Card Reader.

Image: Transcent P8 15-in-1 Card Reader
I checked version and partner version (from head in takeover) and both said on 8.1.3. Compared output of printenv on the 7.3.3 head’s boot loader and 8.1.3 head (from systemshell% kenv), and the 7.3.3 boot loader still thought it was 7.3.3 and was in a complete mess after the attempts at NetBoot-ing it.

So, I removed the 7.3.3 head’s CF card, copied off all the contents, and followed the instructions:

Replacing the CompactFlash card on a 31xx system running Data ONTAP 8.0

- and the section -

Installing the boot device using a PC or laptop with a card reader/writer on

Image: 1GB Compact Flash Card from a FAS3140
Essentially, the document details how to rebuild the Data ONTAP 8.X CF Card from scratch using the software image file (in this case 8.1.3P3_q_image.tgz) to populate the re-created folder structure:

x86_64/
   freebsd/
      image1/
      image2/
   diag/
   firmware/
common/
   firmware/

There was one part in the procedure that had me stumped though - since the 7.3.3 head had never been 8.1.3 there was no varfs.tgz to copy from its /etc folder (investigated this connected to the in-takeover 8.1.3 controller - which fortunately was licensed for CIFS - and mapping a network drive to \\partner_ip\c$).

I could not use the varfs.tgz from working and in takeover 8.1.3 controller, so, in went the reconstructed Compact Flash Card minus a varfs.tgz, and we booted the head up.

The head boots to 8.1.3 and we get the:

The boot device has changed. Use option (6) to restore the system configuration.

There I was thinking there’d be no backup configuration since the head had never been 8.1.3 and no backup config (varfs.tgz) was on the disks, fortunately, Data ONTAP is clever enough to get around this (not quite sure of the mechanism, still it works …)

So, I hit the:

Option 6 ‘Update flash from backup config’

It reboots again, back to:

The boot device has changed. Use option (6) to restore the system configuration.

 So, I hit - for the second time - the:

Option 6 ‘Update flash from backup config’

And finally, we have our head at:

Waiting for giveback…

On the partner we do:

cf giveback

And watch the repaired head boot up (now on 8.1.3). Here’s where the disruptive bit happens - it gives back, but then the controller I was repairing did a reboot whilst not in takeover (being disruptive didn’t matter in this instance since all hosts were down anyway.)

Finally, test everything is good.

All good and this job done!

No comments:

Post a Comment