NekoVM 1.8.0 has just been released on http://nekovm.org
This release includes several bugfixes and performances improvement. In particular, a lot of fixes related to heaving multithreading usage have been made. Why this ?
The reason is that Neko is now distributed with a new mod_* for Apache : mod_tora
mod_neko is similar to mod_php for instance : it embeds the NekoVM in Apache 1.3 (or 2.x for
mod_neko2), and for each request creates a VM, load the Neko module bytecode, execute the code, and redirect the output to the client socket.
One big performance improvement for that is to use the
neko.Web.cacheModule Haxe API, that enables to define an entry point function and cache the module for further web requests : it's a bit the same as different accelerators and bytecode caching for PHP, except that it's not the bytecode but an already initialized Neko module which is cached !
This way all the statics and different initialization that you might do at startup (loading XML configs and datas, filling hashtables, ...) are done only once. That's huge speed improvement, and enable you to design your web applications more efficiently.
The only problem is that this cache takes memory. Not that much - in general 3 to 10 MB per module - but quite a little. And since we are running Apache 1.3, we had something like 200 different Apache processes running, each which their ~3MB module, multiplied by ~10 websites that they are serving. Make the calculus... that's 6 GB of memory !
So before getting CPU limited, our web servers were memory limited. We had to do some server specialization, but it was not always easy to choose the appropriate number of servers for a given website.
That's where mod_tora comes in play. Instead of having 200 different Apache processes each with their own cache, we have one single Tora Server Process, which is a multithreaded Neko application server. Similar to the neko web development server, it perfectly emulates the mod_neko API so you don't have to recompile your bytecode : it's perfectly interoperable with mod_neko.
When a request arrives on Apache and is handled by mod_tora instead of mod_neko, mod_tora connects to the specified Tora server, send the request, the parameters, the headers... and wait for the response.
The Tora server dispatch the request to one of the working threads, which are all sharing the same cache of initialized modules. Several instance of the same module can be put into memory if needed, depending on the number of threads that will need to simultaneously execute a given module.
We deployed Tora today on all our servers :
- 8 x less memory usage at peak time
- 2 x less CPU (because less memory is wasted, the Garbage Collector will have less work !)
The good news is that mod_tora is very tiny since it's just implementing a very simple communication protocol to talk with the Tora Server. It means that porting mod_tora for usage with other popular web servers (such as Lighttpd or nGinx) would be very simple !
Oh, and I forgot to say that the Tora server itself is written in Haxe.
If you want to play with it, here's the steps :
- download and install Neko 1.8.0
- add mod_tora (or mod_tora2) configuration to your Apache server, like you would do for mod_neko
- download the Neko 1.8.0 sources
- you can start your tora server by running
neko -interp tora.n(there might still be bugs with JIT mode, so
-interpwill disable it)
Considering the amount of data that goes between Apache and the Tora server, it's better to have one tora server per Apache server (on localhost). At least that the way we are doing right now.
One last thing that might interest you : after compiling you'll also get a
tora_admin.n file, which when accessed will give you a lot of useful informations about the Tora Server status, such as what are doing the current threads, how many memory is used, ... Here's a screenshot for you :