Real-Time Streaming Data Meetup

On November 21 I was pleased to participate in a meetup entitled Real-Time Streaming Data.  The organizer of this meetup assembles a wide variety of presenters and topics under the umbrella topic of “Large-Scale Production Engineering.” Chris (the organizer) does a remarkable job of keeping a pipeline of interesting talks coming.  I’m particularly interested in the January talk humorously entitled “Whatever happened to IPV6?”

Continue reading Real-Time Streaming Data Meetup

Verilog and Python together Again: Cocotb

It has been a few years since there has been a new project embedding the Python interpreter into Verilog. There is a new one on the block, called “Cocotb” built with a definite focus on writing testbenches and running regressions. It is written in a modern dialect of Python and hosted on Git. It is well-documented. I’m impressed with what I’ve seen.

This article gives a little bit of a history of Python and Verilog and describes how Cocotb fits in.

A Brief History

ScriptEDA (2001) [,] was primarily an example of using SWIG to link a Python interpreter into a Verilog simulator using PLI/VPI.  ScriptEDA handled TCK, Perl and Python in roughly the same way.

Continue reading Verilog and Python together Again: Cocotb

AudioContext, noteOn/noteOff, currentTime and time units

The Web Audio API is a high-level JavaScript API for processing sounds in web applications. The specification was released in 2012, and is supported in Chrome and Safari on Mac OSX, and in iOS6. Implementations of the standard I’ve tested are stable and responsive.

Beosystem 2000

AudioContext is very new, and many examples on the net illustrate the basics of starting and stopping a sound. The methods that start and stop sounds are called noteOn and noteOff. To start or stop a sound immediately you can call




The argument of 0 means “immediately” but does not correspond to the audio engine’s notion of “now.” Indeed, I first made the mistake of thinking that that the current audio time was relative to the execution time of the script: it is not – audio time is a property of the AudioContext.

To be completely explicit, I now write all mentions of sound triggers using the currentTime of a global audioContext.

    var now = audioContext.currentTime;
    sound.noteOn(now + 0);


    var now audioContext.currentTime;
    sound.noteOff(now + 0.5);


An AudioContext is an object into which sounds can be rendered, much in the way graphics can be rendered into a Canvas. Continue reading AudioContext, noteOn/noteOff, currentTime and time units

Twisted: txWS and Autobahn and Resource together

The web is organized in terms of “resources”, and many web frameworks make it easy to define a website as a collection of resources managed by an HTTP Resource handler.  Twisted is no exception: it has an excellent Web Server that manages a forest of Resources.  In Twisted terms, its web server is a case of one particular type of Protocol handler: an HTTP server that manages resources.

Sometimes it’s convenient to organize a web-site as a collection of Protocol handlers that each manage a different portion of the Resource forest.  There are many reasons for doing this, but most come down to when the characteristics of some of the resources are better handled by one type of server over another.

Apache installations are configured to do this all the time.  It is customary for static resources (files) to be served by a handler that is optimized for file serving.  Dynamic portions of the website can then be handled by Rails or Django.  In common use, one organizes their web site as a collection of handlers, each mounted at a different URL prefix.

"/static/*"             StaticHandler
"/mywebsite/*"          DynamicHandler

Streaming uploads are another type of traffic that is often better served by its own type of protocol handler.  In Twisted, the default Resource handler buffers an entire request before handling it.  If your application wants to handle upload packets as they arrive (i.e., a streaming upload), you need a custom handler.

"/static/*"             StaticHandler
"/mywebsite/*"          DynamicHandler
"/uploads/*"            UploadHandler


And then there are websockets.  The WebSocket protocol has been evolving and changing over the past few years.  A WebSockets protocol implements a full-duplex channel between the client and server.  Right now, Twisted’s Resources don’t co-mingle with the two popular WebSockets implementations txWS and Autobahn.  Out of the box, Twisted makes it easy to set up a handler for an HTTP Site on different ports, but it doesn’t make it easy to set up two different Protocol handlers at different URI prefixes.

When I have needed to run a standard HTTP website along-side a Twisted WebSocket, I’ve organized my site something like the code below.

reactor.listenTCP(80, HTTPFactory())
reactor.listenTCP(8080, WebSocketFactory())

This isn’t exactly what I wanted to do.  What I wanted was something that dispatched HTTP requests to entirely different protocol handlers based on the URI prefix.

   "/mywebsocket/*" = WebSocketFactory(),
   "/everythingelse/*" = HTTPFactory(),

I wanted to have a little proxy factory that could be configured to route a connection to the proper handler.  This way, I could write specialized Protocol handlers for special applications (streaming media in my case), and still use the proven parts of Twisted’s Resources for everything else.  This way I could drop in either one of the websockets toolkits (txWS or Autobahn) in a Twisted application.

A Proxy Toolkit

I wrote a little toolkit for proxying an incoming TCP connection and dispatching it to one of a collection of Twisted services.  The first few packets of the connection are collected for the dispatcher to make its decision.  These packets are replayed into the chosen service and then the original connection is spliced into the new service.  The service can implement a full-duplex connection like a WebSocket if needed.  A service may be out-of-process as well.  The toolkit can proxy traffic besides HTTP since it operates at Layer 4 of the OSI model.

You can read about the toolkit here at  It comes with a couple of examples that show how to use it.

Update: 2012-11-05

Autobahn recently added a Resource handler. There is a Twisted repository branch adding txWS as a standard resource; it is not released yet.