Brain Yoga with Elixir

Some say that life is too short to do everything, on the contrary, it’s also too short to stay in one place and not explore. Especially for software developers, there are so many ecosystems to explore, like node (aka JavaScript), JVM (aka. Java, Groovy, and friends), and CLI (aka, .net, C#) to name a few… from the mainstream.
But when we zoom out, all of them are kind of the same! All use C-like syntax, all are stack-based, all are primarily mutable and for the most part, solve problems similarly. So even if you explore one of them you more or less stay in the same mainstream paradigm. On the one hand, that’s good as you get up to speed faster and you can tap into your experience from the other ecosystem.

Arguably, the most growth lies beyond the mainstream, where you must bend your mind and sometimes even question your previous experience.

The question here is if you want to bend and squeeze the tools you know (like Facebook did with PHP)… Or would you rather prefer to challenge your brain and learn the proper tool for the task? You can answer this for yourself, in your own time. In this blog post, I’ll give you a few highlights from my mind-bending journey into the land of Elixir.

Let’s start with “why”… Why Elixir?

Have you ever heard Java’s initial catchphrase? “Write once, run everywhere”, neither it says anything about “running forever” nor about “running efficiently” or “run on a cluster” either. But it’s now used for all of the possible use cases, from desktop apps to cloud systems.
Whereas Elixir runs on BEAM VM, which also powers Erlang. Which was created by Ericsson with the sole purpose of running telephone centers. It was designed to run forever in clustered mode, with failovers built-in and task separation to limit the scope of failures (that will inevitably happen).
What Elixir adds on top is a “syntactic sugar” with Ruby-like syntax that is easier to learn and understand compared to Erlang. Additionally, it also removes a lot of boilerplate code compared to Erlang.

When do you want to consider Elixir then? When the thing you’re going to build runs 24/7/365 with error isolation, online upgrades, scalability, and performance… Sounds familiar? Or is my explanation still too cloudy?

Pose 1: Pattern matching

Let’s start from the basics: assignment operation. You may wonder how this could be mind-bending… Stay with me 😉

Probably the first thing we learn when we start our journey with coding is assigning variables. After initial struggles, it becomes second nature to assign a variable. In JavaScript, Java, C#, or any other C-like language it’s always:

const myVariable = 6

or

final String hello = "hello"

No surprise here, in this (simple) case it’s also how it works in Elixir, and it’s even less typing:

myVariable = 6
hello = "hello"

Easy, right?

Let’s raise the bar a bit higher and move to destruction. If you wrote any (modern) React code, for sure you used hooks. So to declare a mutable state you’d do:

const [value, setValue] = React.useState(42)

Let’s jump for a moment to Go-lang, as you may know, it doesn’t have exceptions, instead, each function will return a tuple with the first element being a result (or nil) and the second, being an error (or nil).

Elixir has tuples also, but they usually use :ok, and :error as the first elements to distinguish between success and error. With this in mind…

{:ok, file} = File.read("./existing_file")

above code should be self-explanatory. We’re reading file content from the disk and storing it in file variable. But what will happen when the file doesn’t exist? You may recall that File.read will return :error as the first element of the tuple. But what happens then? Let’s try

{:ok, file} = File.read("./NOT_existing_file")
** (MatchError) no match of right hand side value: {:error, :enoent}
    (stdlib 4.2) erl_eval.erl:496: :erl_eval.expr/6
    iex:1: (file)

MatchError, “assignment” has failed! As this is not an assignment it’s a pattern match!  We can “easily fix this” with:

{:error, reason} = File.read("./NOT_existing_file")

Now, this code will not produce the MatchError.

What’s more, you can use pattern matching and destruction in the function parameters! That’s even the preferred way of writing “conditional” logic. Let’s examine this by implementing a sign function. To recap, the sign function returns -1 for all arguments lower than 0, 0 when an argument is 0, and 1 when it’s greater than zero. So in JS, it would look like this

const sign = (arg) => {
  if (arg > 0) {
    return 1
  else if (arg < 0) {
    return -1
  } else {
    return 0
  }
}

Easy! Let’s now rewrite it in Elixir

defmodule Example do
  def sign(0), do: 0
  def sign(arg) when arg > 0, do: 1
  def sign(arg) when arg < 0, do: -1
end
 
> Example.sign(11)
1
> Example.sign(-89)
-1
> Example.sign(0)
0

Mind blown? We’re “overriding” or “shadowing” sign function with different pattern matches and guards (the “when” keyword is called a guard).

This mechanism is really powerful, as you can pattern-match and destruct arguments in one go. You don’t need to have a complicated if-elseif-else block, instead, you get (powerful) pattern matches. It takes some time to get used to it… but when you do, you miss that in other languages, as the function body (with all that matching) is short and easy to comprehend.

Pose 2: Let’s talk about scopes!

We all know how variable scoping works. Even with fu*ked-up scoping in JS, over time we learned how to use it to our advantage. Let’s engine that simple JS code then:

const sumUp = (input) => {
   var acc = 0
   input.forEach((i) => acc = acc + i)
   return acc
}

Now let’s rewrite it in Elixir

defmodule Example do
  def sum_up(input) do
    acc = 0
    Enum.each(input, fn i -> acc = acc + i end)
    acc
  end
end
 
> Example.sum_up([1, 2, 3])
0

If we try to use the JS version it will gladly sum up all of the elements of the array. But the Elixir version will always return 0… Why?!

Elixir is a bit special with variable scoping. Meaning that you can read variables from parent scopes, but if you try to override any of them, the overridden value will be only visible from that point below. Meaning it will never override values in parent scopes! That’s why we always get 0 from our Elixir implementation. BTW will you be able to rewrite the Elixir version in a way that it will behave like JS? Could you post your version in the comments?

Pose 3: Managing immutable state

Elixir is immutable by default. All data structures are immutable, the only way to change them is to return a new value from a function. Although all that we do in programming is just data processing, meaning data is coming in we do our magic (aka mutate it) and data is coming out. But we still need to have some mutable state, for instance, to compute and hold a cart value for users in an e-commerce app. But when everything is immutable we can’t do that, right?
Yes, that’s right, but there is a GenServer interface that allows us to mutate immutable data (as counterintuitive as it sounds).

Let’s see how we can implement a counter with GenServer

defmodule Counter do
  use GenServer
 
  def start_link() do
    GenServer.start_link(__MODULE__, 0, name: :counter_example)
  end
 
  @impl true
  def handle_call(:value, _from, state) do
    {:reply, state, state}
  end
 
  @impl true
  def handle_call(:increment, _from, state) do
    state = state + 1
    {:reply, state, state}
  end
 
  @impl true
  def handle_cast(:increment, state) do
    # state = state + 1
    {:noreply, state + 1}
  end
 
  def value() do
    GenServer.call(:counter_example, :value)
  end
 
  def increment_sync() do
    GenServer.call(:counter_example, :increment)
  end
 
  def increment_async() do
    GenServer.cast(:counter_example, :increment)
  end
end
 
> Counter.start_link()
{:ok, #PID<0.150.0>}
> Counter.value()
0
> Counter.increment_sync()
1
> Counter.increment_async()
:ok
> Counter.increment_async()
:ok
> Counter.value()
3

We used an integer to keep track of the state. But it can be any data structure like a list, map, struct etc. From the @impl annotation, you may guess that those are implementations of GenServer API and you’re right. The value, increment_sync, and increment_async are so-called interface functions to hide the usage of GenServer. The GenServer.call and GenServer.cast functions take two arguments, the first being a process PID or its name and the second is a message that is being sent to that process. So we’re sending messages (aka actions) to our implementation of handle_call and handle_cast. Both function implementations are pure, meaning that they don’t modify anything, they just return the next valid state for that process.

Talking about the state management, actions, and maps… Doesn’t that GenServer look somehow familiar? Have you heard about a library that uses actions (plus reduces), and a map to keep track of state? … Redux?

Pose 4: Everything is a process

I’ve already mentioned processes while explaining the Counter example. But before we dive deeper, let’s have a quick recap.

We all know that each application in the system is backed by (usually) one or more processes. You may have heard of “fork”s, which would create a copy of the current process to run concurrently, the thing with forks is that they are slow to create and also expensive, as it is a full clone of a process, that includes the memory, file descriptors, etc. What’s more, there is no simple way to communicate between forks.

Then we have threads, which are an improvement over the fork model, as they share the memory with the parent and are created within the process itself. But as the memory is shared, it’s also easier to communicate between threads, as they can just modify the shared memory… and that brings us to the famous concurrency bugs that are hard to reproduce and fix.

Can we do better?

The inventors of BEAM VM thought that we could… Similarly to JVM the BEAM VM will allocate a single chunk of memory for itself, and also it will manage its own internal processes and communication between them.

To make things “simpler”, an internal process in the BEAM VM is called… process. Each process takes ~2 KB of memory (on a 64-bit system). The only way to communicate between processes is to send immutable messages. That means that no memory is shared and each process can consume messages (and send responses) in its own time and not worry that its state is being modified by another process.

Sounds easy and pretty much standard? Wait, there is a twist!

Processes are the main building blocks of Elixir (and Erlang) applications, they are also organized in so-called “supervision trees”. There are two types of nodes in a supervision tree: worker and supervisor. The worker will do the work, but the responsibility of the supervisor is to keep track of the workers. When a worker fails (crashes etc) then (depending on a policy) the supervisor can restart that single process or restart all processes in that node.

This means that a single problematic input data will not bring down your whole app! Errors are scoped to that single process and its supervisor. If the process keeps failing after a few restarts, the supervisor will give up and won’t even try to start it again.

Summary

If any of the above looked odd or unintuitive to you, I would highly recommend giving Elixir a try! Why do you ask? As with the similarities between GenServer and Redux, you may find similar ideas being used in different ecosystems. It then makes it easier to grasp them. A good starting point is the Getting started guide and the book Elixir in Action by Saša Jurić.

In case I’ve got your interest in Elixir, you’ll probably be glad that there’s much much more awesomeness in it and BEAM. Like it is by design cluster-able, there is also a “telemetry” module to monitor your instance(s) and integrate with, hot code deployments, REPL (or interactive shell), mix (build tool and dependency manager) etc.

Hope you learned something interesting.

Bring Your Own Reducer

Lets talk redux for a brief moment. As you may know there are three main ingredients in redux:

  • action,
  • reducer,
  • store.

From the first look this kinda makes sense. You have your store with is actually your app current state. Reducers are pure functions, as such they are disconnected form everything, just take current state, and action to produce new state. Finally, actions, those are like messengers, caring around data for a reducer (transformer).

Since all actions are (or at least should be) easily serializable, you get this amazing feature of time traveling within your app.

On a down side. It makes it bit harder to reason about the code. Since actions and reducers may be defined in separate files. Also, sometimes the same action may be used with different reducer in potentially many different places. A lot things can happen since you have a freedom to mix-and-match actions and reducer!

Can we do better?

I think we can! Some time ago I have found async_redux library and since then cannot live without it! In HabitChallenge app I went from Provider to rx_command to async_redux and I do not plan to move again!

What is all the fuss with your own reducer? In async_redux, each action is a subclass of ReduxAction, that have an abstract reduce() method. So each action, defines its own parameters and have bundled in reducer. Then you are dispatching it as normal redux action.

This way action and reducer are always together, in one file, even more, in one class!

Inside of ReduxAction you have access to whole state object. So you can use it for computation (if needed) or to get additional data for your REST endpoints.

reduce() method of course returns whole new state, so you can modify different parts of your app/state at one go, in a single action.

ReduxAction have useful before() and after() methods. In case you need to do something before and/or after the action. Like show and hide progress indicator somewhere in the UI.

If you need to process errors there is wrapError() method. And global error listener!

Whats more, each of before(), reduce() and after() can be asynchronous! Can also dispatch other actions using dispatch() call!

Instead of middleware, there is an ActionObserver interface so that you can do some fancy side effects! If you want to.

Also StoreConnector widget forces you to use View pattern for all of your widgets! This is also the place where your UI and state meets! Where you extract data from state to show to the user and where you initially dispatch actions.

If you care about your app performance, you should definitely give a try for async_redux. Since BaseModel class (one that is use to bind your UI to the store) lets you define list of properties that when changed will re-render the widget. Otherwise widget will stay as it is. This is so awesome, because you can use values that were computed base on current state, no need to include all the parts from your store!

Have I mentioned about test-ability? There is a dedicated StoreTester class, that lets you easily define initial state, then execute an action(s) and verify resulting state. It also lets you check all the actions that were fired (if any) and also verity order and parameters of actions. Finally you can of course verify the state it self.

The key takeaway from all of this, is that you should at least scan through async_redux README.md file and give it a try in a free moment.

Connected Widget Pattern

This is something that I have found myself doing really often in my Flutter code base lately. If you are you are familiar with react-redux patterns on the web (or react-native) this would be probably obvious for you.

Most of my Dart files (with Flutter widgets) right now have at least two widgets.

First is the widget that I actually need! It is either stateless or state-full, has all the necessary presentation components, like lists, buttons, texts, images etc. But it does not have ANY logic and it does NOT access outside data.

Of course as with everything in programming there are exceptions. I do access theme data on this level and also do some simple logic like decide when things should be visible or active.

Other than those exception everything comes in as parameters! From… you guess it… The second widget in that file that has “Connected” suffix and always is stateless. On this level I am connecting everything together. This is the place when I am accessing my state management layer. Also this is the place where I do access my translations and custom theme data.

Wire Up!

For most part Connected widgets do not have any parameters, since most of the data needed comes from the state or context. When you intend to create a Connected list item, then it makes sense to take item index or even item it self as a parameter 😉

This way we get a pure widget that is library quality, it is easily extract-able to a different project or to a custom library project. This also means that it is easier to test, you don’t need to provide all of the external dependencies in order to write tests for it. From my perspective it also encourages you to put your business logic somewhere else, not in the UI component. This also encourages to do not hardcode labels, but extract them into parameters and eventually to translate them.

Secondly, with Connected widgets you have a clear layer where you connect you widget to data. This is a simple data binding in form get this information from state (or context) and put it on this widget. It is simple extract-and-assign layer that IMO not necessarily needs to be tested.

Show me the code!

Lets assume you have to list people invited for the party. The task is to implement a widget that would show their first and last name and have a button to remove given person form the list.

First lets implement pure widget for it:

Continue reading

Size Matters!? Shaving Size of Your Flutter App

Probably APK (or app bundle) and IPA size is not something that is causing you sleepless nights. At least it isn’t for me.

We know that there are some tradeoffs when going cross-platform route. Most of people will name performance or look-and-feel as main concerns. Maybe some will mention the deliverable size. This is what I will tackle in this blog post. The size of your app for the end user.

Of course, going the cross-platform route we accept that our deliverable would be bigger. The main reason is that platform does not provide the runtime for our out app. We always need to ship everything to start and run our app.

Why should you care about your download size? In most cases there are two, three, or thirty other apps that will do exactly the same thing as yours. If your potential user need to wait one more minute to install yours app, this is potential one minute when he/she can give a try for a competitor app. This is also more time when something may go wrong, more bytes to transfer means higher probably that something will go wrong with the internet connection etc.

Also smaller app size may be a factor for the Play/AppStore listing. As Apple/Google needs to push all of that data to the user and they need to pay for the upload 😀 (just kidding ;)). But anyway, at least Play Console shows how your APK size compares to the app category median.

Some basic things you can find in this blog post, I will cover all the rest ;). Would recommend you go over this blog first, configure progurad etc. and get back here for more!

R8

This is Android specific. It is a successor to proguard, that can also remove not used resources. You can enable it by adding android.enableR8=true to android/gradle.properties (although it may be already enabled). One thing to note is that it may remove not used resources, like eg. your notification icon, please follow instructions from flutter_local_notifications in order to keep your notification icon.

Fonts

If your app uses custom fonts, I would recommend finding alternatives in Google Fonts, and then adding google_fonts to your dependencies. You may wonder how additional dependency may reduce your APK size? Well, first of all you may remove all other fonts from your project. Secondly google_fonts will dynamically download font when it is used and it will download it in a specific style and language, so that you don’t need to ship all of the variation with your project. Yes, this is another tradeoff, most probably fonts will be downloaded on app initial start… but it is less bytes to push when app is installed.

Icons

You probably use a few icons either from Icons or CupertinoIcons class. Normally this means that all of the icons are embedded into your Flutter bundle. But since flutter 1.17, you can add --tree-shake-icons option to flutter build command, that would remove all of the not used icons from the bundle. Few more bytes shaved!

Split Debug Info

You probably don’t want to include Dart debug information in your production release. --split-debug-info is another option you can add to flutter build command to store debug information separately. This option takes additional argument that is a directory where debug information should be stored.

In case of my HabitChallenge app, I have created a git submodule, where I automatically store, and tag, debug information after the release.

Then when you need to add debug information to your stacktrace, you can use flutter symbolize command and point to appropriate version of debug-info.

Putting All Together

In my case, I have created bash scripts that handle the release process (let me know if you are interested to know how I do releases for HabitChallenge).

Android

For android things are fairly easy, since you can manually upload APK(s) or App Bundle to Play Console (or point fastline to do it for you automatically).

Continue reading

Random messages in Dart intl

As people, we are always craving novelty, things that we are familiar with makes us feel cozy but also do not attract as as much they were right on the beginning. This why your app should constantly engage the user. How to achieve this? You can for example have multiple messages for same operations.

In HabitChallenge there are two features that have multiple localized messages:

  • local notifications,
  • streak detection.

Each notification will have different message in form of a question, asking if you already did your habit.

For streak detection it is a bit more complex. Each streak message depends on the streak length and has not only random message, but also header and text that you can share with your friends.

Nevertheless both require a way of providing pseudo random message in one of the supported languages. Plus some of the languages have more options than others (yes, Polish users are less likely to see the same message in row).

In react-native times, this was easy! Since i18n-js relays on JSON configuration file, I was able to get extract translated raw list, then it was easy to check its size and finally generate random number within given range.

It is not as easy in Flutter. Mainly because initially I decided to go with the standard localization library intl, and learned half way through that I cannot use the same approach as I had in react-native with it.

You may ask why? First of all intl does not use JSON to store translations, but specialized ARB files. Main advantage of ARB is that it is supported by software that is used by professional translators.

The downside, is that only data structure that it supports is a map and you can’t access it directly. This makes things a bit harder… but there is more!

intl, does not use ARB files directly, they are only used as a transportation layer. With intl you can extract all messages from your app into ARB file, give this file to translation team (if you are lucky to have one). Then get back one ARB file per language and import them back to your app as dart source files.

Yes, there is some black voodoo magic with code generation in between. When you get ARB files, you need to run them through intl command line tools, to generate dart file with translation.

With code generation there are also some limitations. You can use placeholders, but you can’t call any methods on them. Instead of passing an object and extracting its properties, you need to pass each property separately as a parameter 😐

Enough talking, lets jump into the code. Or at least lets start talking code 😉

As I mentioned before, intl have support for maps… not exactly map data structure, but at least we can think about it as a map. I am talking here about Intl.select() where you can select one option base on provided key. Lets try to use it:

Continue reading