A few months ago, Neko 1.0 was released. It was a good starting point : a fully specified language, a compiler for it (written in OCaml), a virtual machine to run the bytecode, and a C FFI API to extend the language and write libraries.
Since then, a lot have been done. Here's some new things in Neko 1.1 which will be released soon :
NekoML : one of the most big feature is that Neko does not need OCaml anymore. A new language have been added, which is similar to OCaml but have a different syntax. It can also generate Neko source code. I started by writing a NekoML "compiler" (parser + typer + neko generator) in OCaml and when that was done, I rewrote first the Neko compiler from OCaml to NekoML and then the NekoML compiler from OCaml to NekoML. Now that both compilers are written in NekoML, the Neko language is bootstrapped.
NekoML is good tool for writing additional languages for Neko, since it includes a Lexer library (which can define Lexers at runtime, not using a separate compilation phase such as Flex or Ocamllex), a Parser library (directly integrate LL(k) Ocaml stream parsers in the language), and a ML type system with pattern matching which is quite a good thing to write a compiler.
The NekoML compiler is trying to generate efficient Neko code, for example in pattern matching to goal is to reduce the number of tests. It was needed to add several features to Neko in order to make it work efficiently, which can then be used for other languages :
- gotos and labels, in order to do non-local jumps, have been added to the language specification.
- switchs are a way to branch on a given value. If all switch cases are small constant integers, then the Neko compiler will directly generate a dispatch table so only one jump will be done.
- a new builtin have been added :
$apply, which enable partial application. In the case the number of parameters needed is reached, the function will be called. This is useful for currying.
Apart from theses few add ins, the Neko language was enough powerful to express all the NekoML data structures and mechanisms, which is good news. NekoML comes with several additions compared to OCaml. For example each type instance carry its own printer to you can actually "print" anything and it will display the tagged constructors names. Useful, very useful.
After I was done with bootstrapping, the next goal was to improve "NekoML compiler compiling itself" speed. The start was quite high, with more than 3 minutes and half needed to recompile the compilers (NekoML + Neko). After some perfs improvements, GC corrections, better Neko code generation and VM optimization, the time is now less than 20 seconds on my box which is reasonable (remember that the compiler is running on dynamicly typed-safe VM).
Other improvements were added to the VM, such as callstack traces and exception traces like in Java so it's a lot more easy to debug a complex program.