Alan Hargreaves' Blog

The ramblings of a former Australian SaND TSC* Principal Field Technologist

Nevada to OpenSolaris Sun Ray on SPARC (part 8 – back to original hardware)

It’s been a long path to get here, including a little experimenting with having an Ultra45 as the final destination box (the fact that it only had 1gb memory in it turned out to be a show stopper for any kind of desktop work).

And yes I know it’s not called OpenSolaris anymore, but I really wanted to stick with the title to keep these articles together.

Last Wednesday I bit the bullet and migrated back to my original hardware which was slightly better specced than what I had been using in the lab.

I did learn some things in this final step which hopefully if anyone ever has to do something like this again will be beneficial.

Cloning the boot disk

While I could have moved the 72gb disk I had in the lab machine directly into the target box, I was reluctant to do so as I did not have another 72gb disk to use as a mirror and I was under the (mistaken – see later) impression that the target had a pair of 36 gb disks in it.

As we had trouble sourcing a pair of 72gb disks, I sourced a pair of 142gb ones and put one of them into the second disk slot in the lab box.

Lesson #1

You cannot hot swap disks in a Sun Blade 2000. There is a microswitch that powers down the machine when you take the side off. I discovered this by watching the fans spin down on side removal. Sigh.

After powering up and booting again we need to add this disk as a mirror. It’s not important that it is larger than the disk I am mirroring, ZFS will only use what it needs on this larger disk to mirror the smaller. I also didn’t want to partition it to match sizes as once I was done I wanted to grow the zpool to the entire available size.

Well actually I did adjust the partition tables, but only to give me the full disk on slice 0 (yes I could have used slice 2, but neatness counts).

OK we add c6t2d0s0 as a mirror to rpool

# zpool attach rpool c6t1d0s0 c6t2d0s0

and then we wait for it to resilver.

I also updated hosts so that it also had the address of the machine that I was going to move the disk to.

I was not sure about whether or not I could boot a detached zpool mirror or if I had to simply pull the disk and move it to the new machine.

Lesson #2

Don’t detach the mirror before removing it from teh source system to the target. You will get a Failed to boot with a message like “Failed to boot detached mirror”.

Move the disk back to the source machine and re-attach:

zpool attach rpool c6t2d0s0 c6t2d0s0

and wait another few hours for resilvering.

This time on putting this disk into slot 1, the machine booted.

Brought it up single user and modified /etc/hostname.eri0 and /etc/nodename. Rebooted to be sure everything took. Why was it still coming up with the source machine name, and why can it not contact the local NIS server?

Lesson #3

Current builds of Solaris 11 development have moved the nodename to be a property in SMF.

Looking at /lib/svc/method/identity-node we see both how to set this AND why /etc/nodename was not helpful.

/etc/nodename is only used if there is no SMF property for config/nodename in svc:/system/identity:node. When it is used here the startup method removes the file after using it. If the property exists, it will never look at that file again. To change this property you need to use svccfg.

# svccfg -s svc:/system/identity:node setpropconfig/nodename = astring: vesvi

Where vesvi was the name of my target machine.

The method also does a:

svcadm refresh svc:/system/identity:node

Which I did and then rebooted again for good measure to make sure the interfaces came up correctly.

Hmmmm, it still isn’t seeing the NIS servers. DOH! In our lab we have our routers advertise themselves. On the normal network, router addresses are handed out with DHCP. As I have a static address, …

Booted back to single user and added the router address to /etc/defaultrouter and things looked much better. Indeed it looks like Sun Ray had come up. I was worried that I would need to dig into the guts of that configuration as well, but it appears not (though at a later time I will go through my notes to verify this).

I mentioned earlier that I thought that the target machine only had a pair of 36gb disks in it. When I took them out I noticed that they were actually 72gb disks. *CLICK* when I originally migrated to this machine from my old Ultra 80 when it died, I had 36gb disks, I must have done the mirror trick there too. What I had forgotten to do was to grow the zpool.

# zpool set autoexpand=on rpool

and we now have a 142gb non-mirrored rpool.

The last major step was to put the other 142gb disk in the machine and set up the mirror. Before I did so I checked the current configuration:

  pool: rpool
 state: ONLINE

	rpool         ONLINE       0     0     0
	  c6t2d0s0    ONLINE       0     0     0

errors: No known data errors

Hang on, I said that I had put the disk into slot 1. Oh yes, c6t2d0s0 is the label on the disk. It just happened to not reflect the actually installed location. This could have made putting the other disk into c6t2d0s0 interesting.

On powering the machine down, I moved that disk into slot 2 and put the new disk into slot 1. It’s nice how ZFS really doesn’t care where you put the disks.

This time I booted from disk2 at OBP and it came up properly. Instead of working standing up at a vdu attached to the serial port of this machine, I went back to my desk and logged into a Sun Ray session on it.

Adding the other side of the mirror:

# zpool attach rpool c6t2d0s0 c6t1d0s0

and wait for the resilvering (which only took 44 minutes this time).

  pool: rpool
 state: ONLINE
 scan: resilvered 34.6G in 0h44m with 0 errors on Wed Mar  9 21:46:45 2011

	rpool         ONLINE       0     0     0
	  mirror-0    ONLINE       0     0     0
	    c6t1d0s0  ONLINE       0     0     0
	    c6t2d0s0  ONLINE       0     0     0

errors: No known data errors

I’m now running on the original hardware with something much lighter than the old nevada build I had on it and it looks like I have all the services that I need.

I will say that after putting up with swapping whenever I wanted to do something on the Ultra45, the SB2000 with 4gb feels so much better.


Written by Alan

March 14, 2011 at 3:46 pm

%d bloggers like this: