Calendar

July 2014
Mo Tu We Th Fr Sa Su
<< >>
123456
78910111213
14151617181920
21222324252627
28293031

Langs

Mod-Tora in NekoVM 1.8.0

Posted on Sep 23 2008

NekoVM 1.8.0 has just been released on http://nekovm.org

This release includes several bugfixes and performances improvement. In particular, a lot of fixes related to heaving multithreading usage have been made. Why this ?

The reason is that Neko is now distributed with a new mod_* for Apache : mod_tora

The current mod_neko is similar to mod_php for instance : it embeds the NekoVM in Apache 1.3 (or 2.x for mod_neko2), and for each request creates a VM, load the Neko module bytecode, execute the code, and redirect the output to the client socket.

It works very well, and Neko is very efficient. But at Motion-Twin we have websites which huge traffic (such as MyMiniCity or more recently http://labrute.fr) which get reached by millions of people.

One big performance improvement for that is to use the neko.Web.cacheModule Haxe API, that enables to define an entry point function and cache the module for further web requests : it's a bit the same as different accelerators and bytecode caching for PHP, except that it's not the bytecode but an already initialized Neko module which is cached !

This way all the statics and different initialization that you might do at startup (loading XML configs and datas, filling hashtables, ...) are done only once. That's huge speed improvement, and enable you to design your web applications more efficiently.

The only problem is that this cache takes memory. Not that much - in general 3 to 10 MB per module - but quite a little. And since we are running Apache 1.3, we had something like 200 different Apache processes running, each which their ~3MB module, multiplied by ~10 websites that they are serving. Make the calculus... that's 6 GB of memory !

So before getting CPU limited, our web servers were memory limited. We had to do some server specialization, but it was not always easy to choose the appropriate number of servers for a given website.

That's where mod_tora comes in play. Instead of having 200 different Apache processes each with their own cache, we have one single Tora Server Process, which is a multithreaded Neko application server. Similar to the neko web development server, it perfectly emulates the mod_neko API so you don't have to recompile your bytecode : it's perfectly interoperable with mod_neko.

When a request arrives on Apache and is handled by mod_tora instead of mod_neko, mod_tora connects to the specified Tora server, send the request, the parameters, the headers... and wait for the response.

The Tora server dispatch the request to one of the working threads, which are all sharing the same cache of initialized modules. Several instance of the same module can be put into memory if needed, depending on the number of threads that will need to simultaneously execute a given module.

We deployed Tora today on all our servers :

  • 8 x less memory usage at peak time
  • 2 x less CPU (because less memory is wasted, the Garbage Collector will have less work !)

The good news is that mod_tora is very tiny since it's just implementing a very simple communication protocol to talk with the Tora Server. It means that porting mod_tora for usage with other popular web servers (such as Lighttpd or nGinx) would be very simple !

Oh, and I forgot to say that the Tora server itself is written in Haxe.

If you want to play with it, here's the steps :

  • download and install Neko 1.8.0
  • add mod_tora (or mod_tora2) configuration to your Apache server, like you would do for mod_neko
  • download the Neko 1.8.0 sources
  • compile neko/libs/mod_tora/server/tora.hxml with Haxe
  • you can start your tora server by running neko -interp tora.n (there might still be bugs with JIT mode, so -interp will disable it)

Considering the amount of data that goes between Apache and the Tora server, it's better to have one tora server per Apache server (on localhost). At least that the way we are doing right now.

One last thing that might interest you : after compiling you'll also get a tora_admin.n file, which when accessed will give you a lot of useful informations about the Tora Server status, such as what are doing the current threads, how many memory is used, ... Here's a screenshot for you :

tora.gif

Enjoy !

6 comments
  • Sep 26, 2008 at 15:41

    Hello Nicolas, congrats on the mod_tora release!

    I was thinking, what could possibly be the downside of using mod_tora instead of mod_neko? That you need another process to handle the requests?. It seems to me that mod_neko is still better choice for low requests count.

    Carlos.

  • Sep 28, 2008 at 22:24

    Yes, it still makes perfectly sense to use mod_neko for small or medium traffic. OTOH Tora becomes useful when you want to handle a lot of requests.

  • Marc Draco
    Mar 06, 2011 at 14:58

    Hello Nicolas,

    excellent work on HaXe - I'm loving it! I am a little confused here though - if Tora is so good - and clearly it is - why would we want to use Neko at all?

  • Axel Huizinga
    Apr 09, 2011 at 05:38

    Hi Marc,
    I guess the advantage of mod_neko is not having to configure an run an additional server.

  • James Mc Parlane
    Oct 19, 2011 at 04:09

    Noticed that neko/libs/mod_tora/server/* is not in neko-1.8.2 sources but it is in http://haxe.googlecode.com/svn/other/

    Building that and running with neko -interp tora.n just gets me a "Error while reading request (Blocked)" message for my requests.

    Am I doing something wrong or is that little example deprecated and should I just move onto using it within apache2 running as a mod?

  • Oct 19, 2011 at 05:20

    @James : Tora should be working nicely, but it needs a web server frontend : makes sure you have mod_tora2 setup for Apache.

Name : Email : Website : Message :