Templating with ErlyDTL 1

10 Jan 2016

Last time I needed to implement a RESTful service in Erlang which got a large JSON and depending on the content it needed to react on different events (like a new product registered in a webshop, etc.). Reacting meant that the application should have called services with JSON response bodies. I don’t know if you made even medium sized JSONs (or maps) in your source, it doesn’t look like good.

Let us imagine that we have a product service, which needed to provide various information about products like code, desciption, price. We have some database which can give us maps describing our products. Those maps can be any deep, so we want some chaining access of the 3 or 4 level deep things in the maps. The products look like these

-module(product).

-spec create_json(Product::map()) -> map().
create_product() ->
    #{id = 483,
      product_code = "C7HR-BCH",
      brand = "Schecter",
      description = "Schecter C7HR-BCH",
      category = #{name = "electric guitar",
                   category = #{name = "7 string model"}},
      frets = 24,
      body = "mahogany",
      pickup = "2x EMG 707TW"}.

map_to_json() ->
    jsx:encode(create_product()).

In this module we handle data as maps. Maps can be converted to JSON format and back. I am using jsx library to do that. During decoding we should give hint to jsx that we expect maps by jsx:decode(Binary, [return_maps]).

In this simple example you can see what happens when we have embedded JSON objects in the code. When I needed to get object paths from JSONs the situation was even worse. So let us suppose that we need to create a category path from this product resulting “electric guitar/7 string model” string.

-module(product).

get_category_path(Product) ->
    string:join(get_categories).

get_categories(Product) ->
    case maps:is_key(category, Product) of
        true ->
            C = maps:get(category, Product),
            [map:get(name, C)] ++ get_categories(C);
        false ->
            []
end.

The problem is that we need to take care of safeness of accessing map elements. Otherwise we will face with badkey exceptions. That can make the code complex even if we know that if there is no such field as name and the empty string will be fine for now. The category path will be rendered on the webpage anyway, and someone will fix it. But we don’t want to crash the page generation.

Solve the problem with templates

ErlyDTL implements Django templates in Erlang environment. One should write templates and save them into files with .dtl extensions. The ErlyDTL compiler will compile them to Erlang source files. In the next step they will be compiled to beam files with erlc as usual. A template will be a generated Erlang module, so for example my.dtl will be my_dtl module. That module has a render(Vars) function which renders the template with the context we provide by giving the variables to the render function. As a result we will have an iolist which is good for optimization point of view, but we need to be aware of having an iolist, when we pass that result to our functions (io:format("~s", ...) handles iolists but now every function prepared for iolists).

{
    "id": ,
    "productCode": "",
    "brand": "",
    "description": "",
    "category": "",
    "subCategory": "",
    "frets": 22,
  
    "pickup": 2x EMG 707TW"
}

Templates contain tags, expression and pure text (see ErlyDTL Github page for details). Between double brackets you can write an expression, which evaluates in the variable context which is passed to the render() function of the module. So basically I need to define a variable product and put them in the context.

my_dtl:render([{product, product:create_product()}]).

With tags like if or ifequal we can write control structures, so if guitar body is not specified we don’t write such a property in the JSON. Also with filters I can say that if the number of frets are not specified, let the default be 22. It depends on the business logic, but probably you understand what they are good for.

Custom tags, filters

In most situations the functionality what Django/ErlyDTL gives us is enough, but as always the 10% of the problems make software development complex. So we are almost there, but we need to query the product price from an external database. Or not the price but the availability. How to solve that problem? Since a template can contain only pre-defined variables and expressions, and can contain control structures or things which helps to format texts… so what to do?

The big power what ErlyDTL gives is a possibility to extend the functionality by creating custom tags and filters, or in other words a custom library. To create such a library we need to create a module which implements erlydtl_library behaviour. The module should provides all the filter and tag names it defines. Also it needs to export the functions which implements tags and filters. A custom filter is a one or two parameter function which gets the value of the variable we want to filter. The second optional parameter is the parameter of the filter (like the default value in case of default filter in the example above). Custom tags are two parameter functions which get the variables provided in the parameter list and a rendering option list. A custom tag may return with new variable bindings, so by executing a custom tag we can define variables in the page context. Very powerful tool.

-module(guitar_lib).

-export([version/0, inventory/1]).
-export([get_price/2, frets/2]).

version() -> 1.
inventory(filters) -> [frets];
inventory(tags) -> [get_price].

%% In the lack of a fret number, we can give defaults
%% depending on the guitar brand
frets(undefined, Brand) ->
    case Brand of
        <<"Schecter">> -> 24;
        <<"Jackson">> -> 24;
        _ -> 22
    end;
frets(FretNum, _Brand) ->
    FretNum.

get_price(Vars, _Opts) ->
    case lists:keyfind(id, 1, Vars) of
        {id, Id} ->
            %% Let it crash if service fails
            {ok, Price} = guitar_store:get_price_by_id(Id),
            [{value, Price}];
        false ->
            %% No id specified, we can crash or we can
            %% leave the context variables as they are
            []
    end.

We have the module, so we need to compile it and we need to explain erlydtl compiler that we have a library to be loaded during compilation. We can do it by adding {libraries, [{guitar, guitar_lib}]} to the compiler options. With the guitar name the guitar_lib module will be accessible in the templates. So we can rewrite the template a bit now.

{% load guitar %}
{% get_price id=product.id as price %}
{
    "id": {{ product.id }},
    "productCode": "{{ product.product_code }}",
    "brand": "{{ product.brand }}",
    "description": "{{ product.description }}",
    "price": "{{ price.value|default:"n/a" }}",
    "category": "{{ product.category.name }}",
    "subCategory": "{{ product.category.category.name }}",
    "frets": {{ product.frets|frets:product.brand }},
  {% if product.body %}
    "body": "mahogany",
  {% endif %}
    "pickup": 2x EMG 707TW"
}

We don’t want 0 prices otherwise if somebody manages to buy a product in that price, we need to ship it them. Also, frets filter gets the brand of the product in order that it can get the parameter by which it can give sensible defaults. With the load tag we can load the library, so all the features of that library will be accessible. If we are using a small number of modules whose functionalities don’t collide, we can load the modules in default by specifying the {default_libraries, [guitar]} tuple in the compiler options.

Custom tags, filters

So custom library is a powerful feature of ErlyDTL since we can implement custom business logic in templates. Also, in the lack of set tag now we can implement a specific variable setter (including some business logic). As always, the advice is that don’t make template library overly complex since the goal is to have an easy-to-read template.