Using TwistedMatrix

TwistedMatrix  (http://twistedmatrix.com) is an asynchronous framework for building networked applications.  When writing networked applications that open a many ports to other servers and processes, one can end up with a slew of threads and a confusing mess of locks, queues and semaphores.  Asynchronous frameworks eliminate synchronization problems by presenting the user with a single thread of control.

The twisted matrix core “reactor” manages all sockets and OS events. You can think of it as a giant “select” loop that dispatches back to your code.  When your code is running, it is the only thread, so there is no need to use mutexes and semaphores to protect access to your data structures.

Integrating windowing environments with asynchronous network programming presents a challenge.  In a threaded approach, the window environment is usually given its own thread to maintain user responsiveness.  Networking and other activities are usually implemented in other threads.  Managing windows in an asynchronous framework can be complicated, but TwistedMatrix provides standard “reactors” for popular window toolkits like GTK.  With it, the reactor dispatches back to user code for window events, network events and timer events all in a similar fashion.  Very elegant!

I explored using Twisted for a small sensor application with an attached web server.  My application needed a serial port interface, a worker thread, and a webserver.  The Twisted asynchronous approach allowed me to develop each of these in isolation without much concern for concurrency issues.

As an experiment, I ported this small application to a raw Python using threads, locks and queues to see how it compared.  The code got slightly larger, and a little more convoluted with the addition of the necessary concurrency locks.  But being a small application, with only three threads, it was manageable.  Having compared the two approaches I think that if I needed to scale up the application to support hundreds or thousands of connections, I would choose the asynchronous approach and give Twisted the job.