Elixir GenServers for Web Developers

Photo by Florian Krumm on Unsplash

I want to share some thoughts and opinions about Elixir GenServers for developers writing web applications (probably with Phoenix).

This isn’t a technical deep dive on GenServers, it’s just some of my thoughts about them from writing web apps in Elixir / Phoenix over the last couple years.

There’s really just one type of GenServer that’s ever been a great fit for a problem I’m facing, and I come across the problem in almost every app I build: the need for recurring background tasks.

For example, in an app I worked on recently I needed to run a job every 10 minutes to pull data from a system of record through one API, and then transform and update some records through another API.

Fortunately, any time this “recurring background task” problem comes up, there’s a great reference directly in the Elixir docs on GenServers for how to solve it. I pretty much copy and paste this example code from the docs, change the module name and tweak it with whatever logic I need:

defmodule MyApp.Periodically do
  use GenServer
  
  def start_link(_) do
    GenServer.start_link(__MODULE__, %{})
  end
  def init(state) do
    # Schedule work to be performed on start
    schedule_work()
    {:ok, state}  
  end
  def handle_info(:work, state) do
    # Do the desired work here
    # ...
    # Reschedule once more
    schedule_work()     
    {:noreply, state}
  end
  defp schedule_work do
    # In 2 hours     
    Process.send_after(self(), :work, 2 * 60 * 60 * 1000)   
  end 
end

What’s great about this is that you barely need to know anything about GenServers to get this working, you just replace the # Do the desired work here line with a call into whatever part of your app runs the job and you’re done.

For completeness, in case you want to test it out, the only other step is that you need to add MyApp.Periodically to the list of children in application.ex like so:

children = [
  MyApp.Repo,
  MyAppWeb.Endpoint,
  MyApp.Periodically
]

The function at the end of the example code, schedule_work/0, implements a job that recurs every two hours, but the timer starts after your code executes. For example, if the code you put in place of # Do the desired work here took 30 minutes to run, your job would actually start every 2 and a half hours because the 2 hour timer would begin once the 30 minute job was completed. This is called “drift” and depending on your requirements it could be a deal breaker.

Here’s a simple way I’ve dealt with drift, and it works as long as your job will never run longer than the interval. If we need the job to run every hour to the second, and we’re certain the job will never take more than an hour to complete, we can implement schedule_work/0 like this:

defp schedule_work() do
  now = DateTime.utc_now()
  # Every hour
  next =
    now
    |> DateTime.add(-now.minute * 60, :second)
    |> DateTime.add(-now.second, :second)
    |> DateTime.add(-elem(now.microsecond, 0), :microsecond)
    |> DateTime.add(60 * 60, :second)
  milliseconds_till_next = DateTime.diff(next, now, :millisecond)
  
  Process.send_after(self(), :work, milliseconds_till_next)
end

When the job finishes running, the schedule_work/0 function will figure out when the next hour is based on the current time, and then schedule the next job to start at exactly that hour mark. Not too bad!

Having such an easy solution for recurring background jobs makes me love Elixir. No external dependencies, no complicated logic, and great documentation to refer to.

Warning

GenServers are a pretty powerful abstraction, and in the hands of experienced developers who are comfortable with their nuances they can be leveraged to do amazing things.

However, if you try to use a GenServer to solve a problem where it’s not the right tool, you’ll quickly find yourself in a world of strange error messages, memory problems, performance bottlenecks, and limited visibility while debugging.

One of the amazing things about Elixir is that as web developers we get to import packages like Phoenix that leverage OTP, GenServers, concurrency, and all the raw power that Elixir inherits from Erlang, but we don’t have to understand how it all works under the hood to benefit from it. I love that, because it means I can focus on solving problems at the domain layer.

Here are a few quick rules of thumb I have:

1. If it’s a recurring background job, use a GenServer like the one above unless some requirement makes it more difficult than using a queuing library or external dependency.
2. If it’s not a recurring job, then a GenServer probably isn’t your best option
3. If you need a quick cache, start with :ets
4. Your web app probably doesn’t need macros, GenServers, custom supervision trees, etc. etc. beyond what you get for free in libraries and the few lines of code their README’s will ask you to put in application.ex.

In short, there’s a great use case for GenServers to make life easy with recurring background jobs, but otherwise just leave the hard OTP stuff to the library maintainers and you’ll build fast, scalable, and most of all simple web apps with Elixir and Phoenix that you’ll enjoy working in for years to come. Thanks for reading!