Mruby: Beyond "hello world"

In the last article , I took an initial look at mruby: how to install it using asdf, how to execute a simple code and how to build a standalone binary from a Ruby source. Now it’s time to try to look further and build something more complex.

But first, let me just acknowledge how really basic the mruby is. It supports almost all Ruby syntax (apart from pattern matching), but when it comes to standard library, you’ll find many things missing. Some potentially surprisin omissions include: JSON parsing/building, regular expressions, HTTP client.

> JSON.parse('{"test": [null]}')
uninitialized constant JSON (NameError)

Does this mean that I have to write a JSON parser or regular expressions engine myself? For sure this would be a fun exercise, but luckily it’s not needed. Although in mruby you can use the whole colorful ecosystems of Ruby gems, it has support for its own libraries to extend functionality.

mruby gems

mruby does not have bundler. It has its own mrbgems system instead. The list of available gems can be seen here. As you can see, it’s not very long, but it cover some basic needs in terms of building embedded or system software. There might be also more libraries on Github, that are simply not listed here.

But how do you add one, for example mruby-json?

This took me a while to figure out. And the final realization might be surprising for a Rubyist: you need to compile mruby with support for these gems. In retrospect, it makes sense. We don’t want runtime dependencies in the form of the Ruby code lying around. We want to be able to provide complete building blocks to run the app: either by a self-contained binary or via a mruby interpreter and a Ruby file.

So let’s forget about the installation of mruby via asdf. We will need, however, a regular CRuby installation present. We’ll see that in a moment.

First, let’s clone mruby repo:

$ git clone https://github.com/mruby/mruby.git
$ cd mruby

Now we will need a file defining custom build config. Let’s call it my_config.rb. Now let’s put the following as content:

MRuby::Build.new do |conf|
  conf.toolchain :gcc
  conf.gem :mgem => 'mruby-json'
  conf.gem :core => 'mruby-bin-mirb'
end

The first line of the block, conf.toolchain :gcc, tells mruby to use default gcc to compile. On MacOS you might need :clang instead.

In the second line we add mriby-json gem from the list of mruby gems I shared above. Final third line instructs to build mirb executable. By default mruby just builds mrbc, if you want mirb or mruby itself you need to add it as a core binary gem.

Now we will need CRuby to actually compile this:

$ MRUBY_CONFIG=my_config.rb ruby ./minirake

This takes a while. At the end, we should see something like this:

Build summary:

================================================
      Config Name: host
 Output Directory: build/host
    Included Gems:
             mruby-bin-mirb - mirb command
               - Binaries: mirb
             mruby-compiler - mruby compiler library
             mruby-json
             mruby-metaprog - Meta-programming features for mruby
================================================

================================================
      Config Name: host/mrbc
 Output Directory: build/host/mrbc
         Binaries: mrbc
    Included Gems:
             mruby-bin-mrbc - mruby compiler executable
             mruby-compiler - mruby compiler library
================================================

We should also have bin directory with two executables: mirb and mrbc. Let’s try mirb.

$ bin/mirb
mirb - Embeddable Interactive Ruby Shell

> JSON.parse('{"test": [null]}')
 => {"test"=>[nil]}
> JSON.dump(a: 123, b: [:test])
 => "{\"a\":123,\"b\":[\"test\"]}"

Great! This is exactly what we wanted - an mruby build with JSON support!

Building an app with libraries

Let’s put together everything we’ve learned so far by building a simple application. The app will be called “quickthoughts” (or qtho for short) and it will simply allow me to record ideas from the terminal in a database.

Let’s start with creating a new directory for our project and cloning mruby into it:

$ mkdir ~/projects/quickthoughts
$ cd quickthoughts
$ git clone https://github.com/mruby/mruby.git

With that done, we have to create our build config. I put it in the main directory (parent to mruby) as build_config.rb. This is the content I ended up with:

MRuby::Build.new do |conf|
  conf.toolchain :gcc

  # this is unfortunately needed to compile mruby-sqlite
  conf.cc.flags << ['-Wno-int-conversion']

  conf.gem :mgem => 'mruby-sqlite'
  # implicit dependencies of mruby-sqlite
  conf.gem :core => 'mruby-metaprog'
  conf.gem :core => 'mruby-array-ext'

  # other gems
  conf.gem :core => 'mruby-dir'
  conf.gem :core => 'mruby-print'
  conf.gem :core => 'mruby-time'
  conf.gem :core => 'mruby-bin-mruby'
end

And now for the actual program in main.rb:

class Program
  def self.run(argv) = new.run(argv)
  
  def run(argv)
    context = Dir.getwd
    body = ARGV.join(" ")

    if body.length == 0
      puts "You need to provide a thought"
    else
      db = Database.new
      db.ensure_structure
      db.add_thought(body, context)
    end
  end
end

class Database
  PATH = "/tmp/thoughts.db".freeze

  def initialize
    @db = SQLite3::Database.new(PATH)
  end

  def ensure_structure
    @db.execute <<-SQL
      CREATE TABLE IF NOT EXISTS thoughts (  
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        body TEXT,
        noted_at DATETIME,
        context TEXT
      )
    SQL
  end

  def add_thought(body, context)
    @db.execute(
      "INSERT INTO thoughts (body, context, noted_at) VALUES (?, ?, ?)", 
      [body, context, Time.now.to_s]
    )
  end
end

Program.run(ARGV)

This is very simple. It takes the argument to the application invocation and saves it in the SQLite database in /tmp/thoughts.db with current timestamp and current directory as “context”.

We can check if it works with mruby executable:

$ mruby/bin/mruby main.rb test from mruby
$ sqlite3 /tmp/thoughts.db
sqlite> SELECT * FROM thoughts;
2|test from mruby|2024-10-05 14:10:54 +0200|/home/katafrakt/dev/quickthoughts

Yay, it works!

Building a standalone executable

Let’s now build it and check it as a standalone executable! To begin, we have to compile mruby:

$ cd mruby
$ MRUBY_CONFIG=../build_config.rb ruby ./minirake

It takes a while again. Probably longer than the last time, because mruby-sqlite mrbgem actually contains SQLite sources and compiles it. But after we’re done, we can compile the source code to the bytecode:

$ mruby/bin/mrbc -Bqtho main.rb

And again provide an entry C file for all this:

#include <mruby.h>
#include <mruby/irep.h>
#include "main.c"

int
main(void)
{
  mrb_state *mrb = mrb_open();
  if (!mrb) { /* handle error */ }
  mrb_load_irep(mrb, qtho);
  mrb_close(mrb);
  return 0;
}

Finally, compile it all:

gcc -std=c99 -Imruby/include qtho.c -o qtho mruby/build/host/lib/libmruby.a -lm

And now we have a standalone executable qtho. Let’s copy it to /usr/local/bin and delete the sources, so we are sure it does not depend on them. Finally we can test it:

$ cd /tmp
$ qtho lets test this
$ cd ~
$ qtho I\'m at home now!
$ sqlite3 /tmp/thoughts.db
SQLite version 3.46.1 2024-08-13 09:16:08
Enter ".help" for usage hints.
sqlite> SELECT * FROM thoughts;
2|test from mruby|2024-10-05 14:10:54 +0200|/home/katafrakt/dev/quickthoughts

Trouble time!

Unfortunately, it did not work. I was a bit puzzled by that. However, after quick “puts-debugging”” session, I figured that ARGV was not automatically passed to mruby. With help of this gist I was able to fix it by setting the constant manually.

#include <mruby.h>
#include <mruby/array.h>
#include <mruby/irep.h>
#include "main.c"

int main(int argc, char** argv)
{
  mrb_state *mrb = mrb_open();
  if (!mrb) { /* handle error */ }
  
  /* passing c args */
  mrb_value ARGV = mrb_ary_new_capa(mrb, argc);
  for (int i = 0; i < argc; i++) {
    char* utf8 = mrb_utf8_from_locale(argv[i], -1);
    mrb_ary_push(mrb, ARGV, mrb_str_new_cstr(mrb, utf8));
  }
  mrb_define_global_const(mrb, "ARGV", ARGV);

  mrb_load_irep(mrb, qtho);
  mrb_close(mrb);
  return 0;
}

After building and testing it again we now see the correct results:

$ sqlite3 /tmp/thoughts.db
sqlite> SELECT * FROM thoughts;
2|test from mruby|2024-10-05 14:10:54 +0200|/home/katafrakt/dev/poligon/quickthoughts.moved
3|qtho lets test this|2024-10-05 14:16:31 +0200|/tmp
4|qtho I'm at home now!|2024-10-05 14:16:39 +0200|/home/katafrakt

However, it is a bit troubling that the program simply silently failed, without and notice. For sure this is a topic for further exploration.

UPDATE: It turned out to be quite easy to fix.

Summary

With this exercise I learned a few more things about mruby:

• How to use gems and that you need to compile mruby yourself to get to use them

• That embedding bytecode in the C file (which is the approach I chose) requires additional work in the entry C file

I also came to some realizations about the current state of the ecosystem::

• There are two sqlite mrbems: mruby-sqlite (which I used) and mruby-sqlite3. Both are quite dated. The first one includes sqlite3 sources, produces some compilation warning-errors and has implicit dependencies I had to add; in essence it required quite a lot of additional tinkering. The other gem is described as experimental. It compiles without issues, but does not have documentation and I found some really non-obvious things in the API, so I wanted to ditch it.

• In general, many mrbgems are quite old, with last commits 7-9 years ago. This is probably due to low interest in mruby itself, which might change if its usefulness becomes more apparent.

Next steps

There are more things I want to try with mruby. The first is testing, the second is writing my own mrbgem, and the third, already mentioned before, is error handling when you embed the bytecode in a C file. This means that this series of articles might be continued.