Tutorial: Interactive code snippets with Hugo and Compiler Explorer

31 May 2021 by Jonathan

I’m currently rewriting the documentation for lexy, my C++ parser combinator library – hey, this is the fourth blog post in a row mentioning it in the introduction! It already has an interactive online playground where you can enter a grammar and input and see the resulting parse tree and/or error messages. This is really helpful, so the new documentation will contain examples that are directly available in the playground, to try it out and see what happens.

While implementing this, I’ve realized that this can also be extended to ordinary code snippets on my blog. Without any Javascript or manual work on my part, I can give the code snippet a “play button” that directly links to Compiler Explorer, like this:

int main()
{
    fmt::print("Hello, {}!", "World");
}

Let’s see how I did it.

If you’re lazy, full code at the bottom.

Compiler Explorer’s API

Much like lexy’s playground, this is also powered by Compiler Explorer’s API. This allows you to compile (and execute!) C++ code just by sending a POST request, which makes the playground work. Compiler Explorer is just amazing.

Here, we’re more interested in the clienstate GET request. With that we can create a URL that simply opens Compiler Explorer with the source and compiler options filled-in, by specifying a ClientState object as base64 encoded JSON.

For example, the following Clientstate results in this messy URL.

{
  "sessions": [
    {
      "id": 1,
      "language": "c++",
      "source": "int main() {}",
      "compilers": [],
      "executors": [
        {
          "compiler": {
            "id": "clang_trunk",
            "libs": [],
            "options": "-std=c++20"
          }
        }
      ]
    }
  ]
}

This opens a CompilerExplorer with a single C++ input with the source int main() {}, no disassembly view ("compilers": []), and one execution view using clang trunk in C++20 mode as compiler without external libraries.

For reference, the ClientState object is defined here, However, its often easier to just create the desired result, generate shortlink and just ask Compiler Explorer for information about the shortlink using /api/shortlinkinfo/<id>: https://godbolt.org/api/shortlinkinfo/aEvxefPsT.

Hugo Shortcodes

This blog is built using Hugo, a static site generator. One of the many great features it has are shortcodes. They are snippets you can insert into your content (like the markdown source for this post) and expand to arbitrary HTML.

Here we want a shortcode that takes C++ code as input, and expands to the highlighted C++ code, but also adds a link to Compiler Explorer. Doing only highlighted C++ code is straightforward: add a layouts/shortcodes/godbolt.html with the following content:

{{ $source := .Inner }}

{{ highlight $source "cpp" "" }}

This simply uses the highlight function to take everything and results in syntax-highlighted HTML. It can be used like so:

{{< godbolt >}}
#include <fmt/format.h>

int main()
{
    fmt::print("Hello, {}!", "World");
}
{{< /godbolt >}}

The result is:

#include <fmt/format.h>

int main()
{
    fmt::print("Hello, {}!", "World");
}

Of course, this is something that is already built-in to Markdown, so let’s extend it to add the play button. For that, we first need to construct the JSON, base64 encode it and add it to a new link. To construct the JSON, we can use the dict function, which constructs a key-value object, and the slice function, which constructs an array. The result can then be converted to a string, and the rest is easy:

{{ $source := .Inner }}

{{ $compiler    := dict "id" "clang_trunk" "libs" (slice) "options" "-std=c++20" }}
{{ $executor    := dict "compiler" $compiler }}
{{ $session     := dict "id" 1 "language" "c++" "source" $source "compilers" (slice) "executors" (slice $executor) }}
{{ $clientstate := dict "sessions" (slice $session) }}

{{ $clientstate_b64 := replace ($clientstate | jsonify | base64Encode) "/" "%2F" }}

<a href="https://godbolt.org/clientstate/{{ $clientstate_b64 }}">Try it online.</a>
{{ highlight $source "cpp" "" }}

Note that one of the characters of the base64 variant used is /, which needs to be percent-encoded before it can be safely used inside a URL. If you forget and ask on the Compiler Explorer discord for help, you’ll confuse at least one maintainer why the feature seems broken. Sorry about that.

This adds a link to the code block that links to a Compiler Explorer session with the same options as before, it just uses the source code from the input instead.

Making it nice

Now we’re essentially done, the rest is just sugar.

For starters, a fully compilable example might include unnecessary boilerplate we don’t want to show inside the blog. So we simply use regexes to extract everything inside of //{ and //} and display that inline, while removing the markers in the Compiler Explorer source:

{{ $source        := .Inner }}
{{ $full_source   := trim ($source | replaceRE "//({|})\\n" "") "\n" }}
{{ $inline_source := trim ($source | replaceRE "(?s:.*//{(.*)//}.*)" "$1") "\n" }}

…

{{ $session := dict … "source" $full_source … }}

…

{{ highlight $inline_source "cpp" "" }}

Don’t ask me to explain the regex. It works.

We also might want to use an actual Font Awesome play button instead of text:

<div class="example-play-button">
  <a href="https://godbolt.org/clientstate/{{ $clientstate_b64 }}">
    <i class="fas fa-play"></i>
  </a>
</div>
{{ highlight $inline_source "cpp" "" }}

For me, this requires the following SCSS to make it look nice:

// play button for interactive examples
.example-play-button {
    position: relative;
    a {
        position: absolute;
        right: 5px;
    }
}

Compiler Explorer supports various external libraries. I probably want to use fmt as well as lexy, so lets enable those:

{{ $lexy        := dict "id" "lexy" "version" "trunk" }}
{{ $fmt         := dict "id" "fmt" "version" "trunk" }}
{{ $compiler    := dict "id" "clang_trunk" "libs" (slice $lexy $fmt) "options" "-std=c++20" }}

Lastly, if I use {{< godbolt >}} … {{< /godbolt >}} instead of markdown code blocks, I loose C++ syntax highlighting, code completion and other language server goodies while editing my C++ code. As a hack, I actually put a markdown code block inside the godbolt shortcode that I just strip away:

{{ $source := .Inner | replaceRE "(?ms:^```cpp\n(.*)```$)" "$1" }}

Again, I wrote this regex yesterday and I’ve forgotten it already.

Conclusion

I hope you’ll find this useful. I will definitely start using it in future posts and maybe update some of my previous ones as well. It saves a lot of manual work of pasting the code into Compiler Explorer, getting a shortlink, remembering to keep the shortlink in-sync etc.

The basic system can be freely extended. Shortcodes can take additional parameters, so you might accept options to switch compilers, show disassembly via {{< godbolt compile >}} or something like that. In the lexy documentation, I don’t actually embed the source code into the documentation. Instead, I use {{< godbolt name >}}, where name is a Hugo asset that is pulled in by the shortcode. As the examples are in external files, I can add them to my testing suite to ensure that they will always compile.

Appendix: Full Code

The layouts/shortcodes/godbolt.html shortcode definition:

{{ $source        := .Inner | replaceRE "(?ms:^```cpp\n(.*)```$)" "$1" }}
{{ $full_source   := trim ($source | replaceRE "//({|})\\n" "") "\n" }}
{{ $inline_source := trim ($source | replaceRE "(?s:.*//{(.*)//}.*)" "$1") "\n" }}

{{ $lexy        := dict "id" "lexy" "version" "trunk" }}
{{ $fmt         := dict "id" "fmt" "version" "trunk" }}
{{ $compiler    := dict "id" "clang_trunk" "libs" (slice $lexy $fmt) "options" "-std=c++20" }}
{{ $executor    := dict "compiler" $compiler }}
{{ $session     := dict "id" 1 "language" "c++" "source" $full_source "compilers" (slice) "executors" (slice $executor) }}
{{ $clientstate := dict "sessions" (slice $session) }}

{{ $clientstate_b64 := replace ($clientstate | jsonify | base64Encode) "/" "%2F" }}

<div class="example-play-button">
<a href="https://godbolt.org/clientstate/{{ $clientstate_b64 }}"><i class="fas fa-play"></i></a>
</div>
{{ highlight $inline_source "cpp" "" }}

Example usage in blog post:

{{< godbolt >}}
#include <fmt/format.h>

//{
int main()
{
    fmt::print("Hello, {}!", "World");
}
//}
{{< /godbolt >}}

Rendered:

int main()
{
    fmt::print("Hello, {}!", "World");
}