Running Twisted Daemons with twistd

Twisted ships with a nice daemon runner called “twistd” that can do a lot of different things for your Twisted plugin or application.  It can set the UID/GID of your process, open up a log file and manage a PID file for you.   All of this is configured with command-line options to twistd.

Twisted Tower of the de Young Museum

While each of these options is well documented, the way in which they interact and the order in which these properties are applied to daemon creation are not.  Starting a daemon involves grabbing some ports, changing UID/GID, opening up and managing both a logfile and a PID file.  The management of the logfile and PID file is complicated by the fact that the UID of twistd changes between the time these files are created and when twistd wants to modify them.

This post explains the sequence of operations that twistd performs for starting a daemon defined in a TAC file on Unix.  We will consider only a minimal subset of the options that twistd handles so that we can focus on the interactions due to the daemon’s UID/GID changing.  Our assumptions are that:

  • twistd is started as user root,
  • our daemon will run at reduced privileges,
  • our application is defined in a TAC file and
  • we are discussing Unix only.

Along the way we’ll explain how the order in which twistd performs its operations creates a few “gotchas” that one might run into when setting up logging and a PID file.

An Overview of the twistd Daemon-starter

Although twistd actually has more options than are shown here, the ones that we are interested in for this article are summarized below.

twistd --uid=UID --gid=GID --umask=UMASK --chroot=CHROOT
     --rundir=RUNDIR --pidfile=PIDFILE --logfile=LOGFILE -y myapp.tac

Here is an outline illustrating the steps that twistd performs in order to create the daemon from the TAC file. We’ll describe each of the steps in turn.

  1. preApplication
    1. checkPID
  2. createApplication
    1. readTacFile
    2. startLogger
  3. postApplication
    1. setupEnvironment
      1. chroot
      2. chdir
      3. umask
      4. daemonize
      5. openPIDFile
    2. privilegedStartService
    3. switchUidGid
    4. startApplication
    5. startReactor
    6. removePIDfile

The “preApplication” phase is what is executed before the application is even created.

  • checkPID – this checks for the existence of a prior PID file and removes it if the old PID does not correspond to a currently running process.  This step is performed as user root

The “createApplication” phase deals with the instantiation of the object that defines a Twisted application.  An application is a service object created with a call to the function twisted.application.service.Application.

  • readTacFile – Read the contents of the TAC file “myapp.tac” as Python source code.  Evaluate it in a completely empty namespace. Upon completion return the value of the variable “application” (if there is one) and discard everything else.  Note: this step is performed as user root.  The code in the TAC file is free to import Python modules at will and can interact with the file system, but it is not able to access global Python state and can only return a single value.
  • startLogger – If the Application object returned in the previous step has not defined a logger, then give this application a default rotating logfile.  This step is performed as user root.  The logfile will be created with name LOGFILE and owned by user “root.”

The “postApplication” phase does most of the real work of running the Twisted application.

  • setupEnvironment – Call chroot(CHROOT), chdir(RUNDIR) and set the umask to UMASK.  Daemonize the process by doing the double-fork trick.  Lastly, create a PID file with name PIDFILE.  The PID file will be created by user “root.”
  • privilegedStartService – Grab the ports that are needed.
  • switchUidGid – Change the daemon’s user and group IDs to UID and GID.
  • startApplication – Call startService on our application’s service object.
  • startReactor – Start the Twisted event-reactor in motion.  It is while the reactor is running that the logfile will be rotated.  Log management will be done with the privileges of UID/GID.
  • removePIDFile -  After the reactor has finished, remove PIDFILE.  This will be done with permissions UID/GID.

Implications for Logfile Rotation

Using the configuration described here, the LOGFILE will be created as user “root” and group “root”, but rotated as user UID and group GID.  If you want rotation to work as advertised it is necessary to put the LOGFILE in a directory in which UID/GID has permissions to rename files.

Implications for PID file creation

The PIDFILE will be created as user “root” but when it comes time to remove it, the daemon process will have the permissions UID/GID.  If you want your daemon to be able to remove its PID file, then it would be placed in a directory in which UID/GID has permissions to remove files.

Conclusion

Twisted’s daemon-runner is a useful and well-tested program that has been in use for a long time.  Some of its side-effects are due to the order in which it performs its steps.  This note laid out some of these steps to explain how process permissions interact with logfile rotation and PID file removal.

This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

3 Responses to Running Twisted Daemons with twistd

  1. Glyph says:

    Hi Ted,

    Thanks for writing this up. This is an informative article, but I think these implications just sound like bugs to me :). twistd could do the right thing in these cases, it just isn’t doing it. Would you mind filing some tickets so that some future version of twistd can make the management of pidfiles and logfiles when run as a different UID/GID more automatic – or at the very least, add this stuff to our documentation?

    Thanks!

  2. matteo says:

    About logfile, seems that twisted checks not only for dir access, but also for file access:

    def rotate(self):
    if not (os.access(self.directory, os.W_OK) and os.access(self.path, os.W_OK)):
    return

    So if the file itself is not writable by the UID/GID, even if directory is, you’re out of luck.

    The only way is to change perms to the file itself prior to application start, after logger init. We do that into a custom log observer which uses DailyLogFile after the init of DailyLogFile itself.

  3. Tom Sheffler says:

    That’s a good suggestion. Thanks for sharing it.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>