Alan Hargreaves' Blog

The ramblings of a former Australian SaND TSC* Principal Field Technologist

/bin/sh DTrace Provider

A couple of days ago Brendan was chatting with me on irc and we got to discussing such a beast. Mainly looking at something simple in the way of the python and perl providers that others have worked on.

Well, to make a long story short (ok it will be longer later), I’ve coded up something that appears to work against the nevada clone tree of a couple of days ago, and logged RFE 6591476 to track it.

I’ll be putting the diffs up in the next day or so, but for a teaser, here is the documentation.

sh Provider

The sh provider makes available probes that can be used to observe the behaviour of bourne shell scripts.


The sh provider makes available the following probes:

builtin-entry Probe that fires on entry to a shell builtin command.
builtin-return Probe that fires on return from a shell builtin command.

exec-entry Probe that fires when the shell execs an external command.
exec-return Probe that fires on return from an external command.

function-entry Probe that fires on entry into a shell function.
function-return Probe that fires on return from a shell function.

line Probe that fires before commands on a particular line of code are executed.

subshell-entry Probe that fires when the shell forks a subshell.
subshell-return Probe that fires on return from a forked subshell.

script-begin Probe that fires before any commands in a script are executed.
script-end Probe that fires on script exit.


The argument types to the sh provider are listed in the below table.

Probe args[0] args[1] args[2] args[3]

char * char * int char **

char * char * int

line char * int

script-begin char *

script-end char * int

subshell-entry char * pid_t
subshell-return char * int

arg0 in all probes is the script name.

In the builtin, exec, and function entry probes, and the builtin, exec, and function return probes, arg1 is the name of the function, builtin or program being called. arg2 and arg3 in these entry probes are the argument count and a pointer to the argument list. In these return probes, arg2 is the return code from the function, builtin or program.

In the subshell-entry, arg1 is the pid of the forked subshell and in subshell-return probes, arg1 is the return code from the subshell.

In the line probe, arg1 is the line number.

In the script-end probe, arg1 is the exit code of the script.


The sh provider uses DTrace’s stability mechanism to describe its stabilities, as shown in the following table. For more information on the stability mechanism see Chapter 39 of the Solaris Dynamic Tracing guide.

Element Name Class Data Class Dependancy Class

Provider Unstable Unstable Common
Module Private Private Unknown
Function Private Private Unknown
Name Unstable Unstable Common
Arguments Unstable Unstable Common

The probes that gave me the most trouble were line and subshell-*.

line was tricky as sh only does line numbers when it parses input. It has no concept of line numbers on execution, which is what we need.

So, I needed to add another structure element (line) to trenod, and every other struct that is cast over the top of it. In the first failed attempt, I set this to standin->flin whenever we allocated a new one of these nodes, I set line to this value. The problem with this is that if the parser hits a newline, this number gets incremented before we actually set it, which means that the last command on every line has the line number of the following line. Not quite what I wanted.

What I ended up going with was the creation of another variable in the fileblk structure (comline) that I set just before standin->flin is incremented. This looks like it works.

The subshell-* probes were not initially going to be a part of this, but an assumption I made about com in execute() when coding the exec-* probes ended up causing the shell to coredump on me. It turns out that com is properly defined when we fall through the switch to the TFORK code, but if we go directly into the TFORK case, it’s something completely different (would you believe “1”?). So, I made the probe conditional on the node type. If it was a TFORK, then we do a subshell probe, otherwise we do an exec probe. This also appears to work.

In the meanwhile, I’ve been sending Brendan sh binaries and he’s already started coding more tools for the DTrace Toolkit based on this provider, and I have to say that some of his ideas look pretty cool.

Technorati Tags:


Written by Alan

August 10, 2007 at 4:03 am

Posted in OpenSolaris

7 Responses

Subscribe to comments with RSS.

  1. Way cool. Long been wanting this. set -x doesn’t cut it.


    August 10, 2007 at 4:24 am

  2. On the one hand this is a great idea. On the other one, it is sad that this is being done on the old /bin/sh and not a more modern shell (say ksh93 since it looks popular in the opensolaris community).


    August 10, 2007 at 4:42 am

  3. In what build was this provider be included?

    James Dickens

    August 10, 2007 at 6:15 am

  4. Great work Alan! The line probe is certainly worth the effort — that’s going to be huge for being able to correlate events in the system to the shell code that’s inducing it. Looking forward to seeing this integrated soon!

    Bryan Cantrill

    August 10, 2007 at 6:17 am

  5. @Sean: Thanks.
    @ /bin/sh is the first on a list that a few people will be doing. A lot of scripting is done in it. The other thing to consider about ksh93 is that it’s on track to get integrated, do we really want to throw some experimental code in there and delay it further? I think the probes will come in right fast after we get past that hurdle.
    @James: No build yet. It exists only on my notebook. Once I’ve done a little more code tidy-up I’ll post teh diffs here so folks can play with it and help me find anything I’ve overlooked before I start pushing for integration.
    @Bryan: Thanks mate. Coming from you that really means something to me.

    Alan Hargreaves

    August 10, 2007 at 12:17 pm

  6. that is impressive, as a sysadmin that does a lot of shell programming for a living i can see a couple of great uses for this. i only wish you targeted ksh instead 😛


    August 10, 2007 at 3:29 pm

  7. Great work Alan, and in remarkable time!
    If people would like to see some demos of this, check out what I just posted here:

    Brendan Gregg

    August 10, 2007 at 5:03 pm

Comments are closed.

%d bloggers like this: