For the very long list of additions and fixes in this release, please see CHANGES.txt.

Other interesting things might be:

• create a “multi-supervisor” dashboard, where you could easily see information about many running supervisors (and the processes which they control) on one web page
• create and document common-UNIX-variant rc.d scripts for supervisor start at system boot

If you’re a student that wants to participate in GSoC and you’re interested in UNIX process control and Python, please send me an email. Other feature suggestions welcome.

If you didn’t make it to PyCon, here’s an overview of our talk:

Supervisor is a tool for managing UNIX processes. Supervisor starts arbitrary processes at its own startup and allows otherwise unprivileged users to start, stop and restart its subprocesses and view its subprocess’ logs using a command line or web interface. Other programs exist which do this, but what sets Supervisor apart is that it is written in Python and built with extension points that can be leveraged by Python developers. In this talk, we’ll look at Supervisor as a platform, and how Python programs written to run under Supervisor can use its unique capabilities.

You can now download the slides from our talk in PDF format.

The relevant changes from 3.0a4 are

• Supervisorctl now supports persistent readline history. To
  enable, add history_file = <pathname> to the [supervisorctl]
  section in your supervisord.conf file.
• Multiple commands may now be issued on one supervisorctl command
  line, e.g. restart prog; tail -f prog. Separate commands with a
  single semicolon; they will be executed in order as you would
  expect.

Changes in release 0.2:

Renamed addProcessToGroup() to addProgramToGroup(). The method
now supports program definitions with numprocs > 1 and will add
all resulting processes from the program definition. It also
fixes a bug where the process config was not added to the group
properly. Requested by Roger Hoover.

Added new method log() that writes an arbitrary message to the
main supervisord log. This is useful for recording information
about your twiddling.

You can download the new version and read the updated documentation on the Supervisor Twiddler page of Maintainable Software’s open source project pages.

Notable features in this release include a generic way to restart processes that are using “too much” memory, and an actual honest-to-god manual .

Supervisor now also has its own (currently stark) web presence.

Enjoy!

Supervisor is a tool for managing UNIX processes. Supervisor starts arbitrary processes at its own startup and allows otherwise unprivileged users to start, stop and restart its subprocesses and view its subprocess’ logs using a command line or web interface. Other programs exist which do this, but what sets Supervisor apart is that it is written in Python and built with extension points that can be leveraged by Python developers. In this talk, we’ll look at Supervisor as a platform, and how Python programs written to run under Supervisor can use its unique capabilities.

The talk will cover basic usage but will focus on the more advanced features we’ve added recently, such as adding custom RPC interfaces and event listeners.

PyCon 2008 will in Chicago on March 13-16th. From the list of talks, it looks like this is going to be a great conference.

  - Supervisorctl now reports a better error message when the main
    supervisor XML-RPC namespace is not registered.  Thanks to
    Mike Orr for reporting this. (Mike Naberezny)

  - Create 'scripts' directory within supervisor package, move
    'pidproxy.py' there, and place sample event listener and comm
    event programs within the directory.

  - When an event notification is buffered (either because a listener
    rejected it or because all listeners were busy when we attempted
    to send it originally), we now rebuffer it in a way that will
    result in it being retried earlier than it used to be.

  - When a listener process exits (unexpectedly) before transitioning
    from the BUSY state, rebuffer the event that was being processed.

  - supervisorctl 'tail' command now accepts a trailing specifier:
    'stderr' or 'stdout', which respectively, allow a user to tail the
    stderr or stdout of the named process.  When this specifier is not
    provided, tail defaults to stdout.

  - supervisor 'clear' command now clears both stderr and stdout logs
    for the given process.

  - When a process encounters a spawn error as a result of a failed
    execve or when it cannot setuid to a given uid, it now puts this
    info into the process' stderr log rather than its stdout log.

  - The event listener protocol header now contains the 'server'
    identifier, the 'pool' that the event emanated from, and the
    'poolserial' as well as the values it previously contained
    (version, event name, serial, and length).  The server identifier
    is taken from the config file options value 'identifier', the
    'pool' value is the name of the listener pool that this event
    emanates from, and the 'poolserial' is a serial number assigned to
    the event local to the pool that is processing it.

  - The event listener protocol header is now a sequence of key-value
    pairs rather than a list of positional values.  Previously, a
    representative header looked like:

      SUPERVISOR3.0 PROCESS_COMMUNICATION_STDOUT 30 22\n

    Now it looks like:

      ver:3.0 server:supervisor serial:21 ... 

  - Specific event payload serializations have changed.  All event
    types that deal with processes now include the pid of the process
    that the event is describing.  In event serialization "header"
    values, we've removed the space between the header name and the
    value and headers are now separated by a space instead of a line
    feed.  The names of keys in all event types have had underscores
    removed.

  - Abandon the use of the Python stdlib 'logging' module for speed
    and cleanliness purposes.  We've rolled our own.

  - Fix crash on start if AUTO logging is used with a max_bytes of
    zero for a process.

  - Improve process communication event performance.

  - The process config parameters 'stdout_capturefile' and
    'stderr_capturefile' are no longer valid.  They have been replaced
    with the 'stdout_capture_maxbytes' and 'stderr_capture_maxbytes'
    parameters, which are meant to be suffix-multiplied integers.
    They both default to zero.  When they are zero, process
    communication event capturing is not performed.  When either is
    nonzero, the value represents the maximum number of bytes that
    will be captured between process event start and end tags.  This
    change was to support the fact that we no longer keep capture data
    in a separate file, we just use a FIFO in RAM to maintain capture
    info.  For users whom don't care about process communication
    events, or whom haven't changed the defaults for
    'stdout_capturefile' or 'stderr_capturefile', they needn't do
    anything to their configurations to deal with this change.

  - Log message levels have been normalized.  In particular, process
    stdin/stdout is now logged at 'debug' level rather than at 'trace'
    level ('trace' level is now reserved for output useful typically
    for debugging supervisor itself).  See 'Supervisor Log Levels' in
    README.txt for more info.

  - When an event is rebuffered (because all listeners are busy or a
    listener rejected the event), the rebuffered event is now inserted
    in the head of the listener event queue.  This doesn't guarantee
    event emission in natural ordering, because if a listener rejects
    an event or dies while it's processing an event, it can take an
    arbitrary amount of time for the event to be rebuffered, and other
    events may be processed in the meantime.  But if pool listeners
    never reject an event or don't die while processing an event, this
    guarantees that events will be emitted in the order that they were
    received because if all listeners are busy, the rebuffered event
    will be tried again "first" on the next go-around.

  - Removed EVENT_BUFFER_OVERFLOW event type.

  - The supervisorctl xmlrpc proxy can now communicate with
    supervisord using a persistent HTTP connection.

  - A new module "supervisor.childutils" was added.  This module
    provides utilities for Python scripts which act as children of
    supervisord.  Most notably, it contains an API method
    "getRPCInterface" allows you to obtain an xmlrpxlib ServerProxy
    that is willing to communicate with the parent supervisor.  It
    also contains utility functions that allow for parsing of
    supervisor event listener protocol headers.  A pair of scripts
    (loop_eventgen.py and loop_listener.py) were added to the script
    directory that serve as examples about how to use the childutils
    module.

  - A new envvar is added to child process environments:
    SUPERVISOR_SERVER_URL.  This contains the server URL for the
    supervisord running the child.

  - An 'OK' URL was added at /ok.html which just returns the string
    'OK' (can be used for up checks or speed checks via
    plain-old-HTTP).

  - An additional command-line option '--profile_options' is accepted
    by the supervisord script for developer use.

      supervisord -n -c sample.conf --profile_options=cumulative,calls

    The values are sort_stats options that can be passed to the
    standard Python profiler's PStats sort_stats method.

    When you exit supervisor, it will print Python profiling output to
    stdout.

  - If cElementTree is installed in the Python used to invoke
    supervisor, an alternate (faster, by about 2X) XML parser will be
    used to parse XML-RPC request bodies.  cElementTree was added as
    an "extras_require" option in setup.py.

  - Added the ability to start, stop, and restart process groups to
    supervisorctl.  To start a group, use "start groupname:*".  To
    start multiple groups, use "start groupname1:* groupname2:*".
    Equivalent commands work for "stop" and "restart". You can mix and
    match short processnames, fullly-specified group:process names,
    and groupsplats on the same line for any of these commands.

  - Added 'directory' option to process config.  If you set this
    option, supervisor will chdir to this directory before executing
    the child program (and thus it will be the child's cwd).

  - Added 'umask' option to process config.  If you set this option,
    supervisor will set the umask of the child program.  (Thanks to
    Ian Bicking for the suggestion).

  - A pair of scripts "osx_memmon_eventgen.py" and
    "osx_memmon_listener.py" have been added to the scripts directory.
    If they are used together as described in their comments,
    processes which are consuming "too much" memory will be restarted.
    The 'eventgen' script only works on OSX (my main development
    platform) but it should be trivially generalizable to other
    operating systems.

  - The long form "--configuration" (-c) command line option for
    supervisord was broken.  Reported by Mike Orr.  (Mike Naberezny)

  - New log level: BLAT (blather).  We log all
    supervisor-internal-related debugging info here.  Thanks to Mike
    Orr for the suggestion.

  - We now allow supervisor to listen on both a UNIX domain socket and
    an inet socket instead of making them mutually exclusive.  As a
    result, the options "http_port", "http_username", "http_password",
    "sockchmod" and "sockchown" are no longer part of the
    '[supervisord]' section configuration. These have been supplanted
    by two other sections: '[unix_http_server]' and
    '[inet_http_server'].  You'll need to insert one or the other
    (depending on whether you want to listen on a UNIX domain socket
    or a TCP socket respectively) or both into your supervisord.conf
    file.  These sections have their own options (where applicable)
    for port, username, password, chmod, and chown.  See README.txt
    for more information about these sections.

  - All supervisord command-line options related to "http_port",
    "http_username", "http_password", "sockchmod" and "sockchown" have
    been removed (see above point for rationale).

  - The option that *used* to be 'sockchown' within the
    '[supervisord]' section (and is now named 'chown' within the
    '[unix_http_server]' section) used to accept a dot-separated
    user.group value.  The separator now must be a colon ":",
    e.g. "user:group".  Unices allow for dots in usernames, so this
    change is a bugfix.  Thanks to Ian Bicking for the bug report.

  - If a '-c' option is not specified on the command line, both
    supervisord and supervisorctl will search for one in the paths
    './supervisord.conf' , './etc/supervisord.conf' (relative to the
    current working dir when supervisord or supervisorctl is invoked)
    or in '/etc/supervisord.conf' (the old default path).  These paths
    are searched in order, and supervisord and supervisorctl will use
    the first one found.  If none are found, supervisor will fail to
    start.

  - The Python string expression '%(here)s' (referring to the
    directory in which the the configuration file was found) can be
    used within the following sections/options within the config file:

        unix_http_server:file
        supervisor:directory
        supervisor:logfile
        supervisor:pidfile
        supervisor:childlogdir
        supervisor:environment
        program:environment
        program:stdout_logfile
        program:stderr_logfile
        program:process_name
        program:command

  - The '--environment' aka '-b' option was removed from the list of
    available command-line switches to supervisord (use "A=1 B=2
    bin/supervisord" instead).

  - If the socket filename (the tail-end of the unix:// URL) was
    longer than 64 characters, supervisorctl would fail with an
    encoding error at startup.

  - The 'identifier' command-line argument was not functional.

  - Fixed http://www.plope.com/software/collector/215 (bad error
    message in supervisorctl when program command not found on PATH).

  - Some child processes may not have been shut down properly at
    supervisor shutdown time.

  - Move to ZPL-derived (but not ZPL) license availble from
    http://www.repoze.org/LICENSE.txt; it's slightly less restrictive
    than the ZPL (no servicemark clause).

  - Spurious errors related to unclosed files ("bad file descriptor",
    typically) were evident at supervisord "reload" time (when using
    the "reload" command from supervisorctl).

  - Updated ez_setup.py to one that knows about setuptools 0.6c7.