Alan Hargreaves' Blog

The ramblings of an Australian SaND TSC* Principal Field Technologist

Progress on non-debug opensolaris builds

function startAnim(anim_to_play)
{
var myAnim = document.embeds[anim_to_play];
if (! myAnim.IsPlaying() )
{
myAnim.Rewind();
myAnim.Play();
}
}

I’ve been pretty quiet on the progress of getting the non-debug
builds working. In the light of Bart’s
blog
about libmicro, and especially the note about OpenSolaris only being built
with debug binaries, I thought it was about time I pulled my finger
out and wrote little more on the progress I’ve been making. Fortunately while working Saturday, I had
a relatively light day (only two callouts), and I’ve spent quite
some time on the train working on this stuff too.

What needed to be done?

There are two parts to getting this going.

  1. Generating the debug and non-debug encumbered tarballs, and
  2. Making sure that the OpenSolaris build scripts know about which
    tarball to use when

    • Building (we need to link with the right encumbered
      libraries),
    • Creating bfu archives, and
    • Doing Cap I Installs

The approach that I’ve taken is to do most of the work in generating
the tarballs and minimising what needs to happen in the opensolaris
tree.

What needs to happen when generating the tarballs?

Building the tarballs is a bit of a chicken and egg process.

In order to build the encumbered binaries, we need to know what
files exist in ${ROOT} in an opensolaris build. We don’t
need to put these in the tarball (with a couple of exceptions that
you will see later). This means we need to do an opensolaris build.

In order to do an opensolaris build there are a few encumbered
libraries that we need from an internal build (we can’t include
these in the list of what files are in an opensolaris build); so we
have to do an internal build before we do the opensolaris build.

Basically we need to work with two distinct build trees.

tonic-clone
This is a snapshot of the nevada tree that we use as the baseline
for opensolaris. All of the encumbered source is still in here.
tonic-split
This is a teamware child of tonic-split, with all of the
encumbered source moved into usr/closed. This tree is
what we create the opensolaris source drop from.

My goal was to script the entire build process with both trees,
thus:

  • Build a debug tonic-clone, creating a ${ROOT}
    that we can use later for the encumbered libraries.
  • Build a debug tonic-split, creating a ${ROOT}
    that we can use to create a list of files from to exclude from the
    encombered binaries. From this $ROOT use a combination of
    find and grep -xvf to create the list of what is in an opensolaris
    build.
  • Run a version of nightly that has some extra code in it to create
    the tarballs given

    1. debug and non-debug build runs, and
    2. the list of binaries from the tonic-split
      $ROOT

I’ve come up with a script called encumbered-bins and a new
environment file called encumbered-binaries.sh. The
environment file differs from the normal ones in that it expects a
few other environment variabls to be set in order to work out which
workspace it is to build.

${WSDIR}
The directory above the workspace
${GATE}
The directory name of the workspace within ${WSDIR}

These are used to define ${GATE} and ${CODEMGR_WS}
in the envirnment file that we will use for both nightly
and bldenv.

Memo to self: When working with environment variables above
builds, choose your names carefully. For example, ${WSDIR}
is a far better name for what I am using it for than
${TOP}. That took a little working out why things stopped
working how I expected them to.

It’s also worth noting that bldenv has this lovely feature
that allows you to run it with a script. In order to keep the number
of new files to a minimum, we create these files using here
documents and very careful quoting of variable to make sure
they get evaluated in the right place (yes, this bit me more than
once too).

Anyway, the result of this run is that we end up with the following
files in tonic-split.

  • closed-bins.${MACH}.tar.gz, and
  • closed-bins-nd.${MACH}.tar.gz

Where ${MACH} is the result of a uname -p on the
architecture we are building on. A this point it will be either
sparc or i386.

So what needs to change in the opensolaris tree

Surprisingly little.

usr/lib/Makefile
A single line change to make sure we copy over the right
encumbered libraries before we build.
usr/src/tools/scripts/bfu.sh
Another small change to make bfu use the correct closed
tree to pre-populate the proto area that we will build the bfu
archives from.
usr/src/tools/scripts/Install.sh
Another small change (can you see a trend here?), again to make
sure that Install uses the correct closed binaries, yada,
yada, yada.
usr/src/env/opensolaris.sh
Remove the -F flag from ${NIGHTLY_OPTIONS} so
nightly will now do both the debug and non-debug builds, along
with creating the two bfu archives. OK, this bit hasn’t gone into
the tree yet, I just put it in to the opensolaris.sh that
I used for my opensolaris builds.

So where are we up to now?

On Saturday afternoon, I managed to have encumbered-bins
run from go to whoa and create both tarballs for x86 (my goodness
those v40z’s are quick). Using the non-debug bfu archives, I
successfully bfu’d the v40z that I used to build it all. See the
movie below.

As a side note, from the time that grub starts to boot
opensolaris, to the time that we have a login prompt is about 50
seconds. This includes doing DHCP and setting up nisplus (as
that’s how we do our local lab).

Note in the login message, the archives where the bfu was done from.
There is a -nd in there. You will also note that while you
were watching the boot, you did not see the string DEBUG
ENABLED
.

So what do I still need to do?

Test Cap I Install
I’ve still not tested this.</dd

Test SPARC builds
That’s probably next, but I don’t anticipate problems.</dd

Clean up nightly
nightly needs to do something a little more sensible with
bindrop‘s output. It’s currently ending up reported in
the noise at the end of the run.
Test the debug bfu archives
I really wanted to test the debug ones first. For completeness I
need to be certain that I haven’t broken anything.
nscd problems
Check if nscd not starting is a problem with what I’ve
done, or something unrelated.
Code Review
After we’ve finished all of the above, I have to get Mike and
Keith to code-review it before it can go into the opensolaris tree
for you all to enjoy, and maybe even run some libmicro (or other)
benchmarks on.

Technorati Tags:

Advertisements

Written by Alan

August 3, 2005 at 5:04 pm

Posted in OpenSolaris

%d bloggers like this: