This post is written in org

Recently I found myself doing more and more things using org-mode. Not surprisingly, I also wanted to use it to write articles here. With some time on my hands this weekend, I decided to give it a try and see how hard would it be to add org support to BridgetownWhich this blog uses as an SSG.

Even though Bridgetown supports adding own formats via converters, they have one big limitation:

Bridgetown will only convert files that have a YAML or Ruby Front Matter header at the top, even for converters you add using a plugin.

Of course, I could just add front-matter on top of my org file, but let's be honest: how silly it would look? I had to find a way to force Bridgetown to ignore its own rule and parse org files without the YAML front-matter.

Historically what I described above would not be possible. Check for front-matter presence was hardcoded and I would have to simply overwrite part of Bridgetown's internals. Ruby allows that, but I think you can see why it does not sound like a good idea.

Fortunately, Bridgetown lately released a beta of 2.0 version, which includes a huge improvement in that area. Now front-matter loader are decoupled. And even better! You can quite simply add your own.

In Bridgetown 2.0 front-matter loaders are simply classes that inherit from Bridgetown::FrontMatter::Loaders::Base. So I wrote mine and put it as plugins/org_front_matter_loader.rb:

class OrgFrontMatterLoader < Bridgetown::FrontMatter::Loaders::Base
  def self.register!
    Bridgetown::FrontMatter::Loaders.register(self)
  end

  def self.header?(file)
    File.extname(file) == ".org"
  end

  def read(file_contents, file_path: "")
    # for some reason layouts are run through this function as well and we don't want to attempt
    # to parse them
    return nil unless self.class.header?(file_path)

    Bridgetown::FrontMatter::Loaders::Result.new(
      content: file_contents,
      front_matter: read_front_matter(file_contents),
      line_count: file_contents.lines.size
    )
  end
end

This is where things started to be a little funky.

Ruby has it's org-ruby gem to parse org files. It's old, it's battle-tested. Supposedly Github uses it to display org files. So it should be a solid choice, right?

Wrong.

Being dated and focused on transforming org to different formats (HTML, Markdown, Textile...) org-ruby has few big deficiencies. Among them:

• Poor and secondary support for block properties

• No support for footnotesIn case you haven't noticed, I use them

• No support for some more recent features of org, such as file level properties

Given all that, I had to turn into other implementations (or write my own). I finally settled for orgize, which is written in Rust. I don't know Rust and I don't like Rust, but if that's what it takes...

... I had to get my hands dirtly and write a simple wrapper-gem for orgize. Luckily, it's super simple these days. Bundler gives you a solid foundations, so the main gist of the code I had to write is as follows:

mod html_renderer;
use magnus::{function, prelude::*, Error, Ruby};
use serde_json::to_string;

fn parse(input: String) -> String {
    let org = orgize::Org::parse_string(input);
    format!("{}", to_string(&org).unwrap())
}

fn to_html(input: String) -> String {
    let mut writer = Vec::new();
    let mut handler  = html_renderer::MyHtmlHandler::default();
    let mut org = orgize::Org::parse_string(input);

    let main_title = org.headlines().nth(0).unwrap();
    main_title.set_title_content("", &mut org);

    org.write_html_custom(&mut writer, &mut handler).unwrap();
    format!("{}", String::from_utf8(writer).unwrap())
}

#[magnus::init]
fn init(ruby: &Ruby) -> Result<(), Error> {
    let parent = ruby.define_module("OrgReally")?;
    let parser_module = parent.define_module("OrgizeProxy")?;
    parser_module.define_singleton_method("parse_to_json", function!(parse, 1))?;
    parser_module.define_singleton_method("parse_to_html", function!(to_html, 1))?;
    Ok(())
}

Now you can probably notice that there is also some html_renderer thing here. This is my attempt to make the HTML output a little more aligned with my blog. This part definitely is not that pretty and under heavy development, so I will just link to a gist with a working version.

In the future I will probably want orgize to only parse and traverse the AST, pushing HTML rendering to Ruby. However I'm not there yet.

with the gem in place, I could finally write a front-matter reader.

def read_front_matter(contents)
  json = OrgReally::OrgizeProxy.parse_to_json(contents)

  title_node = json.dig("children", 0, "children", 0)
  title = title_node["raw"]
  pairs = title_node.dig("properties", "pairs")
  properties = pairs.to_h.transform_keys(&:to_sym)

  properties.merge({ title: title, tags: title_node["tags"]})
end

This gives me all metadata from org file available in Bridgetown, so it can nicely render the title, pick correct layout, add tags etc. At this point I should probably also show part of org source of the post. It looks like this:

∗ This post is written in org :bridgetown:org:meta:ruby:
:PROPERTIES:
:layout: post
:date: 2024-09-14
:sidenotes: true
:END:

Recently I found my self doing more and more things using
[[https://orgmode.org/][org-mode]]. Not surprisingly,
I also wanted to use it to write articles here.
With some time on my hands this weekend, I decided
to give it a try and see how hard would it be to add
org su  pport to Bridgetown[fn::Which this blog uses as an SSG].

Yes, if you are not used to org, this probably looks ugly to you. It did for me a year ago or something.

I mentioned before that currently the rendering is done by orgize (so in Rust, with a sprinkle of my customizations on top). The final piece is to tell Bridgetown to use this HTML renderer to actually render the post.

This is done in plugins/org_converter.rb file, which looks like this:

class OrgConverter < Bridgetown::Converter
  input :org

  def convert(content)
    org = OrgReally::OrgizeProxy.parse_to_html(content)
  end
end

That's it? Yes, that's it.

Overall, despite hitting a few dead ends, this ws rather enjoyable little project. Bridgetown, especially in version 2.0, made everything easy for me. For real, I expected more uphill battles against files with no front-matter or rendering with Rust, but it's really solid piece of well-designed software.

The situation with org support not only in Ruby, but generally, seems a bit sad. This is partly due to the fact that org does not really have a formal definition, just a reference implementationThis should sound familiar to a Rubyist ;). And that it's not the most popular format, compared to e.g. markdown, so it's lagging behind a bit.

Orgize is only one of a few Rust parsers. None of them feels finished, but this one seems most complete and has upcoming 0.10 version which is supposed to add support for some missing features.

Some things are still missing, like the code highlighting, but I think I will make them work on par with markdown - and perhaps without having to use Liquid tags, like I had to implement footnotes in markdown.