WebSocket Upgrade Handshake. EventMachine HttpServer and Null characters.

I took advantage of some holiday downtime to experiment with WebSockets a bit.  As of December, Chrome implements HTML5 WebSockets natively.  I think we can expect more implementations shortly.

My experiments were inspired by this blog entry and my own ongoing experiments with EventMachine.  I learned two things from this exercise:

  1. The WebSocket Upgrade Handshake should be sent as a contiguous stream of bytes.
  2. Null characters in WebSockets streams may confuse existing HTTP Protocol handlers.

Upgrade Handshake

The websocket protocol (IETF Spec) is absolutely clear about the fact that the first three lines of the Upgrade Protocol Handshake must match “character-for-character” as laid out in the specification. I wanted to know if these first three lines could be sent as separate frames.  In experimenting with the Chrome implementation I learned that the three lines should be sent as one multi-line string.

Using an EventMachine Connection to listen on a port, I returned the handshake using three separate send_data calls.  Chrome failed to recognize this as a handshake by refusing to report an open connection.

send_data("HTTP/1.1 101 Web Socket Protocol Handshake\r\n")
send_data("Upgrade: WebSocket\r\n")
send_data("Connection: Upgrade\r\n")
send_data("WebSocket-Origin: #{origin}\r\n")
send_data("WebSocket-Location: #{location}\r\n")
send_data("\r\n")

I changed the handshake to the following.  This succeeded in creating an open connection between the browser and server.

upgrade = "HTTP/1.1 101 Web Socket Protocol Handshake\r\n"
upgrade << "Upgrade: WebSocket\r\n"
upgrade << "Connection: Upgrade\r\n"
upgrade << "WebSocket-Origin: #{origin}\r\n"
upgrade << "WebSocket-Location: #{location}\r\n"
upgrade << "\r\n"
send_data(upgrade)

Null Characters

In another experiment, I wanted to subclass EventMachine::HttpServer to modify it to be a WebSocket server. The idea was to detect WebSocket connections and upgrade the connection if a WebSocket is detected. This technique would use the HttpServer header parsing mechanism only and then would get the Request handler out of the way to allow the bytes that are sent in the socket stream to flow freely.

HttpServer has a mode (dont_accumulate_post) that tells it to call method (receive_post_data) on the bytes received in a Request body as they are received, rather than to accumulate the entire body. I reasoned that I could receive bytes in the WebSocket stream by using this option and overriding receive_post_data. This didn’t work. [Yes, I realize all of this is an abuse of HTTP.]

The reason that this didn’t work is that each message in a WebSocket stream starts with a Null (\000) byte. EventMachine::HttpServer is written in C and uses a string function (strpbrk) to do request body parsing. Most of what HttpServer sees is strings that start with Null.

Of course, the WebSockets protocol is not a subset of HTTP exactly, and I shouldn’t expect this to work, but now I know.

I did succeed in subclassing EventMachine::HttpServer to give me the byte streams. Here’s an outline of what I did. This puts a big conditional before the processing of every packet received, but it achives the effect I want. HttpServer parses the headers and I get the byte stream. My existing HTTP header-processing (cookies, sessions) works with my WebSocket stream.

class WsServer < EM::Connection
  include EM::HttpServer

  def post_init
    super
    no_environment_strings
    dont_accumulate_post        # dont read all of the post data
    @header_processing = true
  end

  def process_http_request
    ...
    @header_processing = false
  end

  def receive_data(*args)
    if @header_processing
      super
    else
      receive_post_data(args)
    end
  end

  def receive_post_data(x)
    puts "GOT POST DATA:#{x.inspect}:"
  end
end

Conclusion

The WebSocket protocol superficially looks like a subset of HTTP, and even the terminology “upgrade” suggests that it is only a refinement.  Strictly speaking however, it is a different (but related) protocol.  It will be a while until servers, caches and middleware get the details sorted out.

Leave a Reply

Your email address will not be published. Required fields are marked *