You are likely one of the many folks (myself included) who have recently learned to love Elixir. I initially started down the wrong road with Elixir, but I learned a valuable lesson as I corrected course. It takes some adjustment to view the world in terms of a functional language, with lightweight processes, running on the Erlang Virtual Machine. My realization was that thinking in terms of visual diagrams enables this important mental shift. So much so that if you’re writing Elixir without drawing diagrams, I’m pretty sure you’re doing it wrong.
My experience
I recently ventured into developing a small internal application with my mentor. We did this in Elixir, using Plug, on a Cowboy server. We replaced a JSON API and ported all our 3rd-party integrations to the new Elixir application. There were some interesting challenges to tackle, and in some cases, we one-upped the incumbent software with new features (and speed).
We wrote a good functional application. As for leveraging the Erlang VM, that’s a different question. Here are some of the features we cared about:
- Fault tolerance
- Concurrency
- Low latency
These are features available because of the OTP (Open Telecom Platform) design philosophy. From Erlang’s Overview:
[OTP] is based on a supervision tree, [which] is a process structuring model based on the idea of workers and supervisors:
- Workers are processes that perform computations, that is, they do the actual work.
- Supervisors are processes that monitor the behaviour of workers. A supervisor can restart a worker if something goes wrong.
- The supervision tree is a hierarchical arrangement of code into supervisors and workers, which makes it possible to design and program fault-tolerant software.
Did our implementation take advantage of the strengths of this runtime? We did implement a GenServer in at least one module, and we implemented a couple of supervisors. To a degree, we did achieve some of these goals.
What I learned, however, was that without a deeper shift in thinking, something is still lacking. We might meet the goals of OTP, but it wouldn’t be intentional. All we really did was write a functional program with a mind to implementing some special features available in the runtime. And that’s not good enough. There is a difference between adding features and thinking in OTP.
Changing mindset: Think about processes
We need a change in perspective to do things right. How do we get there? Here is José Valim giving us a clue about the answer in his recent blog on this very subject:
When designing the Erlang language and the Erlang VM, Joe, Mike and Robert did not aim to implement a functional programming language, they wanted a runtime where they could build distributed, fault-tolerant applications. It just happened that the foundation for writing such systems share many of the functional programming principles. And it reflects in both Erlang and Elixir.
According to José, there's more to think about, beyond writing functional programs. It's really about implementing a certain type of robust application. And our job is to comprehend what it means for our software to demonstrate that robustness.
If we want to think like José Valim and Chris McCord and other great Elixir developers, we must realize that processes do not enable OTP in some vague philosophical sense. Thinking about processes is actually the first step to writing Elixir and in planning the design of your software. But what are processes? From Getting Started:
In Elixir, all code runs inside processes. Processes are isolated from each other, run concurrent to one another and communicate via message passing. Processes are not only the basis for concurrency in Elixir, but they also provide the means for building distributed and fault-tolerant programs.
Putting processes first means we shouldn’t start by thinking about how to do the work (which is strictly algorithmic). Instead, we should think about the processes in which the work gets done. Processes are not logical by nature. They are operational. Here's what I mean: They are supervised. They turn on. They turn off. They send messages. They have mailboxes. They receive messages. They collectively form a network.
It is not just easier to reason about Elixir when you view it as a network of processes, it's the only way to reason about Elixir. If the question is, "What is the Elixir way?", then the answer has to start with a momentary (but definite) abandonment of logical nuts and bolts. And it has to begin with the operational nature of running on the Erlang VM.
Once we start to look at problems, at a high level, in terms of the network interaction of independent processes, then we can better see the truth of working with Elixir. The upshot: Doing things the Elixir way will ensure that we take full advantage of the Erlang VM and all of its strengths, rather than merely adding features to our functional code.
Let’s see what that might look like.
In practice
For example, say we have a web application with a "sign up" feature. Let's break it down with a view to the strengths of the Erlang VM and let's think operationally. This will be the Elixir way of pseudo-coding.
A process should bring some form data to the server and start a few subordinate processes. Among them: One process should make sure the new user can be created with the data given and persist the new user. Another process should give the user a helpful web response. Another process should collect some analytics for the product owner. Another process should send a welcome email. All of these processes should shut down when they've done their job.
Some of these processes need to talk to each other and work synchronously. Namely, once the form data is received, we need to ensure that the new user is created before spawning some of our other processes, like a welcome email. Some of the processes can be concurrent. Namely, the web response, the analytics and the welcome mailer.
The general lesson is that we're thinking in terms of discrete processes that do their job and then go away. Of course, inside the process we'll do the functional programming that we're used to doing. But, to start off, we have envisioned the network that facilitates the flow of data from start to finish.
To program the Elixir way, we need to know how to implement those processes and properly configure their interaction to properly map over the pseudo-coding we did above.
If we want fault tolerance, we implement supervisors to restart processes in the correct hierarchy and with the appropriate strategy. If we want concurrency, we compose certain processes to work asynchronously. And none of these things are glued on. Since we started by describing the operation as a network of processes, we are already in the mindset of a robust OTP application. We don't have to make anything fit.
Draw Diagrams Or Go Home
And that’s where diagramming comes in. What we did in the "sign up" example above is the written form of a visual diagram. James Edward Grey II made an inspiring suggestion to attendees of his SimAlchemy Training at ElixirConf 2016. He recommended we start drawing diagrams of the processes in our Elixir applications.
This is not merely a great way to communicate the operations of an application to another developer. JEG2's suggestion was the start of an epiphany for me about the Elixir way of thinking. His representation of processes, with arrows, and hierarchy, was an effortless demonstration of wielding the tool we're using, the way it should be wielded. Here is how he illustrated one of his example applications:
Drawing a diagram also challenges each depicted process to prove whether it needs to exist. Do you need to send a message, receive a message, come to life, do a job, be terminated, be restarted? It's a built-in way of thinking critically and letting the concept break down before it becomes bad code. This is a good exercise. In short, we have achieved basic proficiency as Elixir developers when we can bring a sound diagram to life in code.
Perspective
We've taken for granted that drawing an entity relationship diagram is one of the first steps in understanding our problem domain and our data. I have never worked on a project with a relational database without a visual data model. Similarly, the operations of our Elixir processes are uniquely suited to a visual representation that illustrates the operating network that our application will create.
For me, that change in mindset is the beginning of my understanding of the Elixir way, and I hope it has helped you, too. Let me know what you think in the comments.
PS: Do you have an Elixir project in the pipeline? Check out our Elixir development services to learn more about our development engagements.