Warning: A non-numeric value encountered in /home/ricston2/public_html/blogarchive/wp-content/themes/Divi/functions.php on line 5766

This blog post is an example of using the erl_syntax module in Erlang. Although the functions in this module are documented, it’s not very easy to use, at least not in my experience.

Recently, I’ve had reason to start using the module again, with some success. Here I will start from scratch to make sure we’re all on the same page, however, it makes sense to first provide a brief overview, so that you can decide to move along if it’s not what you’re looking for.

We look at how to take the abstract forms given to parse_transform/2 and selectively change some parts of them. We see how to iterate over the forms and substitute expressions to make use of the send/bang operator (for example; ‘!’) into a block expression containing the send expression as well as a function call. We achieve this by using the erl_syntax module without relying on the abstract form or syntax tree data structure representation. This makes our code more resilient to changes in these representations (assuming of course, that the erl_syntax module is kept up to date).

If you fully understand the previous paragraph, you can skip ahead to erl_syntax usage example. Otherwise, please continue reading from here.

Preliminaries

When compiling Erlang code, it is possible to give the compiler the optional argument of {parse_transform,Module}. This causes the parse_transform/2 function (which takes 2 arguments) in the module whose name is bound by the Module variable, to be called during compilation. The parse_transform/2 function which you define in and export from Module has its first parameter bound to a list of abstract forms (and the second to the compiler options used during compilation, but we’re not interested in that here).

The list of abstract forms is passed by the compiler to your parse_transform/2, and your parse_transform/2 is itself expected to return a list of abstract forms. In other words, you get to manipulate these abstract forms before they are passed to the compiler to generate byte code from them.

The abstract forms passed to your parse_transform/2 are a representation of the Erlang source code that you’re compiling. For example, consider the following Erlang module:

When compiling this module, the abstract forms passed to your Module:parse_transform/2 are shown below:

As you can see, abstract forms encode your source code in a more compiler friendly way. However, if you want to alter this representation, it would require more effort on your part. You would need to decipher the semantics of the encoding… but even worse, if you base the implementation of your abstract form manipulation logic on this representation, you tie your implementation to the abstract form representation used in whichever version of Erlang you happen to be using. I suppose that’s the reason why programmers are advised not to engage in parse transformations in the docs.

However, the erl_syntax module can help out here. This module provides an API to create and manipulate a representation of the abstract forms consumed by the compiler. Effectively, the erl_syntax works on a representation of Erlang source code, known as; ‘syntax trees’.

Quoting the doc:

This module defines an abstract data type for representing Erlang source code as syntax trees, in a way that is backwards compatible with the data structures created by the Erlang standard library parser module erl_parse.

The idea is that, if your code uses erl_syntax to do its source code manipulation, then any changes to the representation of this source code shouldn’t affect your code as long as erl_syntax is kept up to date. As a simple example, if your code assumes that variables are represented as: {var,4,’To’}, then it’s likely to break if this representation changes, whereas if you create the representation using erl_syntax:variable/1, then you delegate the responsibility of determining the correct representation of a variable to the erl_syntax module.

So far so good, but as I mentioned before, erl_syntax can be a bit tricky to use, especially if you’ve never used it before. I have managed to produce quite a few exceptions trying to work with the module. Of course, this is probably due to my lack of understanding, but either way, I think that some step-by-step examples can only improve the situation. I’ll be tackling one such example here, and perhaps some more in the future too.

erl_syntax usage example

We’ll be compiling the hello.erl module, and our goal is to inject a call to io:format/1, passing it the argument “About to send:~n”, before every use of the send operator in Erlang.

Consider the following module:

To make use of the Options passed to parse_transform/2, here we are checking to see if a transformation function was passed as an argument to the compiler. If it was, we use the given function, otherwise, we’ll use blog:replace_send/1 function which we’ll be discussing below. This transformation function is responsible for transforming the individual parts of a syntax tree representing the source code of the module being compiled, leaving anything it’s not interested in intact.

As you can see from the transform/2 function, the process we use to carry out our transformation involves 3 steps:

  • Changing the list of abstract forms (the Forms parameter) into a syntax tree which can be used by the erl_syntax module.
  • Modifying the syntax tree.
  • Changing the syntax tree back into a list of abstract forms for consumption by the compiler.

We change the list of abstract forms to and from syntax trees because although an abstract form is a valid syntax tree, a list of abstract forms is not (for example; ‘a syntaxTree() is not a [syntaxTree()]’).

From the docs:

… all erl_parse trees are valid abstract syntax trees, but the reverse is not true: abstract syntax trees can in general not be used as input to functions expecting an erl_parse tree.

The “erl_parse trees” are the abstract forms passed to your parse_transform/2, while “abstract syntax trees” are representations of erl_parse trees which the erl_syntax module works with.

The postorder/2 function was taken from the documentation of erl_syntax:subtrees/1. It applies the first parameter (which is expected to be a function), to every sub-tree in the given syntax tree (the second parameter), updating the tree along the way. It’s what you do in the supplied function (the first parameter), that does the actual transformation, in our case, it’s replace_send/1.

Node, in replace_send/1, is bound to some kind of syntax tree. While iterating through the supplied syntax tree and all its sub-trees (and all sub-trees of these sub-trees etc…), the first thing we need to determine, is which type of sub-tree are we interested in. We do this via erl_syntax:type/1 and catch the case when the Node is an infix_expr. When it’s an infix_expr, we extract the infix expression’s operator and get it’s name. If it’s the infix operator we’re interested in (the send operator; ‘!’), we create our own node to replace the given one in the whole syntax tree. The node we create is a block expression made up of the io:format/1 expression followed by the previous Node, which is the original expression using the; ‘!’ operator in the source code.

That basically covers it. If you want to pass your own transformation function, you can pass the following option to the compiler:

{trans_fun, funYourModule:YourFunction /Arity}

Below we can see our replace_send transformation in action:

I hope you find this blogpost both useful and interesting!