Calendar

November 2014
Mo Tu We Th Fr Sa Su
<< >>
12
3456789
10111213141516
17181920212223
24252627282930

Langs

Comet-style programming with Tora server

Posted on Jul 12 2009

One of the most limiting issues of HTTP is that it's a request/response protocol. This prevent the server to push informations to the Client : instead, the Client needs to make some requests on a regular basis to check if some new informations are available.

While in most of the cases this limitation makes developing a website much more simple than developing a realtime application, it's often desirable to still be able to update some part of the page/game/application in realtime.

In order to do this, there are two main possibilities.

Custom Server

The first one is to entirely bypass HTTP by using a custom server application on which the client can connect to directly. Javascript does not have API for this, but Flash has flash.net.Socket which can be used to connect to such a server and exchange binary data in realtime.

This is often the solution that has been chosen for realtime applications. Flash player even have a native protocol for that called RTMP which is implemented by FlashMediaServer, Red5 and HaxeVideo. Haxe also have its own Haxe Remoting protocol which can be used to exchange data between client/server, but also between client/client (for instance between JS and Flash).

However, it's often hard to design a custom server, especially if you want it to scale well on thousands of users. And even if you have the good server technology, it's quite hard to get everything working nicely.

Comet

Another solution comes from the Javascript world and is called Comet

The idea is to reuse HTTP (since Javascript does not have access to custom sockets) but instead of the server giving an immediate answer, a Comet server will only send the response when some incoming data is available.

There are then two ways of doing which are details on the Comet Wikipedia page :

  • make an XmlHttpRequest call and wait for the incoming data, then when it is received make another request : this require the server to track the user between two different connections in order to ensure the delivery of some data that would need to be sent between the first answer is sent and the second connection is accepted.
  • make an "infinite iframe" and use the HTML streaming browser feature to send <script> tags that will be evaluated by the browser when received : this is more hackish since it rely on browser features that are not part of the protocol and might change depending on the browser vendor/version. However, it practice, it works.

Tora

Tora is a Neko Application Server that we are using to write our website. It's not itself an HTTP server but it uses one as a proxy, using the following scheme :

  • Apache receive the HTTP request and handle it
  • Apache deliver it to mod_tora which connect to the Tora server
  • the Tora server load the neko bytecode, process the request and send the answer back
  • mod_tora send the answer to the client using Apache API

Since the Tora server is a single-process multithreaded generic server which can load any Haxe/Neko web application, one of its advantages is that it can be extended to provide additional APIs for the applications that run on it.

I then gave it a try to implement Comet-style API into Tora, and came with the following two classes.

tora.Share

First, one thing which was easy to add is a way to share data between the applications instances. A typical Tora server run with 32 Threads. When a request comes, it looks for an available instance of the corresponding application. If there is no available instance, it creates a new one. This mean that there can be several independent instances of the same application running at the same time on different threads.

The goal of tora.Share is then to be able to share some data between all instances running on the same Tora server. This is usually done by using a Database, but this API allow for in-memory sharing which is more efficient to write highly interactive services.

class Share<T> {    
    function new( name : String, make : Void -> T ) : Void;
    function get( lock : Bool ) : T;
    function set( data : T ) : Void;
    function commit() : Void;
    function free() : Void;
}

A shared data is identified by a name, and if it's not found then make is called to initialize it. You can then get the corresponding data and eventually lock it to prevent its access by other threads. If you lock the data you will have to commit it before exiting from the application.

You can also set the data and free it : in that case it's simply removed from the global Share table, but another thread will again create it it access it.

Please note that there is no guarantee of concurrent access to the data : a thread that modify a locked data might end up crashing another thread that didn't have the lock but was reading the data. It's up to you to ensure proper atomicity, the best being to use immutable data in order to prevent threading issues.

tora.Queue

Now, in order to be able to implement Comet-like applications, a message-queue API was also added :

class Queue<T> {
    function new( name : String ) : Void;
    function listen( onNotify : T -> Void, ?onStop : Void -> Void ) : Void;
    function notify( message : T ) : Void;
    function count() : Int;
    function stop() : Void;
}

The idea is the following :

  • one client connect to the web application, the application send the current state of the data and then call listen
  • when another client makes a change to the data, he can call notify on the corresponding Queue, with a given message
  • the notify message is delivered to all callbacks which can then ignore it or format it to send it to the client
  • in case a client disconnect, the onStop callback is called which enable proper notification
  • in response to a given message, a client can choose to stop listening on the Queue, in which case the connection is closed
  • finally, it is possible to known the current count of clients listening on this given message queue

The main advantage is that all this API directly integrate into your web application : there is no more need for a separate custom server.

Also, when a client listen on a Queue, the call immediately return and the application exit normally, the corresponding Tora thread is then free and can handle other requests.

However, if your connection goes through mod_tora (which has been slightly modified to handle listen) then an Apache thread or process will still be in use while the client is listening. This does not consume a lot of resources so can be acceptable, but there is another issue which is there is currently no way (?) for mod_tora detect if a client has been disconnected from Apache. You will only know when sending some data to it, and after a long timeout. But unless you want to have realtime notification of disconnected clients, that's fine this way.

tora.Protocol

In case however you want to get around such issues, it's possible to make a direct Flash-Tora connection by using the tora.Protocol API :

class Protocol {
    function new( url : String ) : Void;
    function addHeader( key : String,value : String ) : Void;
    function addParameter( key : String, value : String ) : Void;
    function connect() : Void;
    dynamic function onDisconnect() : Void;
    dynamic function onError( msg : String ) : Void;
    dynamic function onData( data : String ) : Void;
}

This will directly connect to the Tora server by using the same protocol used by communications between mod_tora and the Tora server. You can use it like you would make an HTTP connection, except that you will receive onData everytime the server application makes a print.

Tora Commandline

In order to deal with this, the Tora commandline has been modified, here are the two parameters documentation :

  • -host name : select the host on which you want to run the Tora server (default localhost)
  • -port 6666 : select the port on which you want to run the Tora server (default 6666)
  • -threads 32 : select the number of threads (default 32)
  • -unsafe 6667 : open a port which allow direct connections on the Tora server
  • -config <path> : parse an Apache configuration file to look for VirtualHosts

The two last options are used for direct Flash-Tora connections. First, you will have to configure your firewall to block incoming connections on the port 6666 to prevent clients from directly connecting in safe mode to Tora. This is because the safe mode can bypass any security (htaccess, etc) that is usually handled by the Apache proxy.

Instead, you can set an unsafe port that will only accept an URL, instead of the full path to a neko application. In order to resolve this URL into the corresponding application, you can use -config which will parse an Apache configuration file and look for VirtualHost sections :

<VirtualHost>
    DocumentRoot /path/to/www
    ServerName www.mysite.com
    ServerAlias  other.mysite.com
</VirtualHost>

When Flash connect to the Tora server with an URL, the application loaded will be the DocumentRoot which correspond to the servername + /index.n

Experimental

We are currently experimenting with this new technology at Motion-Twin, but current tests seems to be quite promising, so we are expecting good things from this new technology which integrates very nicely in our current stack.

We are also looking forward for people interested in giving it a try. So far this new Tora and its API can be found in Neko CVS , in neko/libs/mod_tora/server. This require also Haxe CVS to compile properly, but we will make a proper release of a tora Haxe Library

2 comments
  • Jul 12, 2009 at 14:38

    Hey Nicolas,

    Okay, so, I understand the concepts here, and I can see how one might use this, but how would it differ in use or benefit over a custom socket server, perhaps one using ThreadedRemotingServer?

    Best,
    Lee

  • Christoph Degeneres
    Jul 24, 2009 at 01:59

    I understand this kind of Apache module is cool, but what makes this better approach than having a standalone server like StreamHub Comet server, which already implement Comet?

Name : Email : Website : Message :