Saturday, September 13, 2014

Porting Code to MRuby

If you take a random library from Ruby stdlib & try to use it under mruby, expect failure. If everything seems to work out of the box it's either (a) a miracle or (b) (more likely) you haven't tested the library enough.

The 1st thing I've tried to bring to minirake was FileList. It turned out that FileList uses Dir.glob (glob wasn't implementated in mruby-dir). It turned out that Dir.glob internally uses File.fnmatch (fnmatch wasn't implemented in mruby-io).

Have you ever used File.fnmatch in your code? You usually stumble across its pattern language only as sub-patterns of Dir.glob patterns. For example, Dir.glob adds ** & { } syntax.

In MRI, File.fnmatch is implemented in C. Extracting it to a plain C library w/o Ruby dependency is relatively quick & simple. This is how Rubinius team ported it & so did I. There nothing interesting about the library except maybe the notion that for some reason MRI version returns 0 as a mark of successful match & 1 otherwise.

Dir.glob is a more complex story. Again, in MRI it's implemented in C. At 1st I wanted to do for glob the same job as for fnmatch but glob has too many calls to MRI API that hasn't direct equivalents in mruby. I was lucky not to have to mess with C because Rubinius had its own version of Dir.glob written in Ruby.

It didn't go so smoothly as I hoped because the code isn't a 'pure' Ruby but an Rubinius version of it with annoying calls like Rubinius::LRUCache, Regexp.match_from, String.byteslice. (The last one is from Ruby 1.9+ but mruby still lacks it.)

After the porting struggle I checked the result with unit tests for Dir.glob from MRI & amazingly they worked fine which was a pleasant surprise because I wasn't expecting the good outcome.

Then came FileList turn

As every library that was written by Jim Weirich it's (a) very well documented, (b) uses metaprogramming a lot.

While changing class_eval calls with interpolated strings to a class_eval with blocks & define_method was easy, bugs have started to arrive from unexpected & funny areas. For example:

$ ruby -e "p File.join ['a', 'b', 'c']"


$ mruby -e "p File.join ['a', 'b', 'c']"
["a", "b", "c"]

Or even better:

$ ruby -e 'p [nil] <=> [nil]'
$ mruby -e 'p [nil] <=> [nil]'
        [1] mrblib/array.rb:166:in Array.<=>
        [0] -e:1
mrblib/array.rb:166: undefined method '<=>' for nil (NoMethodError)

The same goes for NilClass & <=>. File.extname behaves differently, File.split is missing, etc.

In many cases it isn't mruby fault but mrbgem libraries, but the whole ecosystem is in a state that isn't suitable for people with weak nerves. Sometimes I thought that 'm' in mruby actually means 'masochistic'.

After the porting struggle with Array methods like | & + I took unit tests from Rake & amazingly they worked almost file (there is no StringIO in mruby) which wasn't a pleasant surprise because at that point I got angry.


Do you know that __FILE__ is a keyword & __dir__ is a method? You can monkey patch __dir__ in any moment, but can do nothing to __FILE__. I didn't know that.

Making an executable with mruby involves producing the bytecode which can be statically linked to the executable & loaded via mrb_read_irep function at the runtime.

Bytecode can be generated with mrbc CL utility that ships with mruby. It sets value for __FILE__ according to its CL arguments. For example:

$ mrbc -b mycode foo/bar/main.rb

will set __FILE__, for bytecoded main.rb, to foo/bar/main.rb. If you have an executable named foobar & use main.rb as an entry point it your Ruby code, the classic trick

do_someting if __FILE__ == $0

won't give the result you've expected.

At 1st I thought of overriding __FILE__ but it turned out that that wasn't possible. Then I thought of setting __FILE__ after the bytecode was generated but wasn't able to figure out how to do it w/o coredumping. At the end I patched mrbc to be able to pass the required value from CL which means, to be compiled, minirake requires now a patched version of mruby. Great. :(


The last missing part of Rake I wanted to have was FileUtils. It may seems like useless & superfluous but we like Ruby for DSLs, thus its more idiomatic to write

mkdir 'foo/bar'


sh "mkdir -p foo/bar"

or even

exit 1 unless system "mkdir -p foo/bar" # [1]

FileUtils has some nice properties like the ability to print on demand what is happening or turn on 'no write' mode. For example, if you

include FileUtils::NoWrite

any 'destructive' command like rm or touch will do nothing.

I've looked into stdlib fileutils.rb & have quickly gave up. It's too much work to port it to mruby. Then I thought of making a thin wrapper around system commands with an FileUtils compatible API.

The idea is to generate a several sets of wrappers around simple methods in some FileUtilsSimple::Commands namespace so that user will never execute them directly but only through pre-generated static wrapper that decide what to do with a command.

Acquiring a list of singleton methods is easy but mruby never makes your life easy enough. The next mruby present was an absence of Kernel.method method. I don't even.

Unit Tests

Don't get tempted to test the ported code under MRI because your favorite test framework runs only under cruby. I've bumped into several occasions where test passes fine under cruby & fail miserably under mruby.

[1]Did I mention that Kernel.system just return a boolean & doesn't set $?? (Make a random guess in which implementation.)