July 17, 2017 — functional, programming and reactive

At first look, mobx seems easy, sounds like easy, quacks like easy.

Its design, however, requires some reconsideration of how the state should be organized in an application.

I love the fact that it is the recommended pattern for beginners these days, but most people use it because it’s magic. I’m here to convince you that there is much more to mobx than that.

First of all, it’s not magic, there is nothing paranormal with it, and I will come to that in a moment. But, first...

... Rotterdam!

Rotterdam is a wonderful city, you should really visit here.

Back to mobx.

I admit, when I first heard of it, in late 2015, I wasn’t really quite impressed with the operational details of it. To my knowledge, there were already many observable implementations around and most of them were not even going this deep; they were just light-weight implementations, and mobx was not.

I thought it was just a sweet syntactic sugar with a sweet real-world application called mobx-react where it proved to be really useful. But even so, it stayed on my mind as something over-simplified or over-complicated.

Now you could say I was not the brightest kid in the class when it comes to functional programming :) I also didn’t join the workshops at Mendix organized by Hans L’hoëst which was aiming to introduce new-comers to FP using Scala.

So, that was my exact thoughts on mobx when I was on the plane to Amsterdam, from there, to Rotterdam, to work on the project that manifested it, Mendix Modeler.

Of course, that was not my exact thoughts when I was on a plane to AMSTERDAM!

I was like... OMGGG !! OMMMGG! I’m moving to the NL, are we there yet? Are we there yet? Are we? ARE we there? But that’s another story.

Then, after a week or so, mobx kicked in.

Now, besides the theory, to this day, I still don’t think it is a very practical tool to use as any other light-weight component you put in an average nodejs application. That’s my experience.

However, when it comes to UIs, it excels.

And we didn’t know that. It’s funny how everything is unfolding now! When I landed in Amsterdam, mobx (then mobservable) had around 120 stars, today I checked it once again and it was almost 10,000. Massive kudos to Michel and everybody who has contributed! A great achievement!

Not so obvious things

Like I said, some things are not really obvious at first sight, like everything should be in state.

When all is left to state, React only becomes a tree renderer. Michel wrote about this, we never used setState again. Oh, wait, I will come to that. But first...

...magic!

Don’t let those smart kids make you think it is magic, it’s not. It’s an illusion.

Some people freak out when they hear the term mutable, or see something that even remotely resembles a mutable is being used for state management. It’s like how Dutch people love Hagelslag, and how the rest of the world thinks: “Why not Nutella?”

Well, they are not the same things. Indeed, they are not the same things, a Hagelslag is never a Nutella. It can taste like a Nutella, quack like a Nutella, but it won’t be a Nutella. For me, if it works, it works, whatever. It really depends on the current context that I am working in. “Is there butter around?” “How big is the bread?” “What should be the ratio of cacao sprinkles to the rest of the other stuff?” But, regardless of that, it’s not Nutella.

I was coming to that. The coolest thing about mobx is... Are you ready? It’s native types. It’s just mimicking... native types. The ones that the mustache spec is mentioning or the lodash API is being used for. The good old native types! Your high school bros!

Yes, of course, in mobx, it looks and quacks like you are dealing with them, but actually, you’re not. But, whatever...

The familiarity creates this illusion that you’re working with native types. But behind that green curtain, there is a wizard of Oz, putting a great deal of effort for creating this illusion. I mean, seriously, I’ve seen Michel bisecting the Arrays, trisecting the Maps into the tiniest bits and putting so much effort into making mobx support all the native types. So, it’s not magic, it’s mostly hard work.

With observable natives, also comes great power w.r.t. interoperability and of course composability. Don’t ignore that.

About a week ago, I dusted off an old mustache parser I wrote years ago, and turned into a reactive templating engine on top of mobx, just like that.

So, it’s not only the ease of use of data types or how fancy their names are, it’s also their familiarity.

state.js

Listen up, yo!

So, you did actually. Thanks!

You put your observables inside a class or whatever, and commented it nicely, then sealed off carefully and saved it to a file called state.js.

That’s why the state container concept should not be a new thing to you. In fact, your state.js file was actually a state container from the beginning, albeit not a very popular one.

Perhaps, so far this can only be considered a convenient namespace for tucking new observables in. You efficiently emit tiny change events without even noticing, and with a little help from mobx-react, voila! Everything is in motion!

Nevertheless, that’s all. When people say that “mobx is un-opinionated”, what they really mean is, there is no opinionated way to simply interact with your observables directly.

You can do anything to start a chain reaction, and you know, people do all the peculiar things in life. Early this year, I wrote these two modules: kek and kek. kek is a state container that proxies a mobx observable to lodash and kek is a state container and a duplex stream which emits & applies JSON-patches (rfc6902). (mobx-state-tree has built-in JSON-patch support so that kek is kapot I guess.) What I mean is: I don’t think people do that, I don’t think people use mobx like this, I don’t even think people use the same names for two different modules.

You see, everybody has their own opinions of leveraging what mobx does well. This causes a lot of confusions and bugs when the project you’re working on is growing.

Scalability knocks the door telling you that you need a way more opinionated approach, maybe even a limited one.

mobx-state-tree does that, it provides a generally accepted approach for defining how a state should look like. And it’s a very cool project, you should check it out.

Ok, there’s one last thing that I want to put emphasize on before closing up the mobx section.

Your state is not only a dataset, it’s a state; and by the same token, it is not only a phase of existence, it’s alive.

This is important. When working with mobx, you don’t have data or store, or whatever. You have an initial state. So, the object/the store you created is gone, mobx has changed it into something else and tries to convince you that it is still the same object, but it’s long gone. Think of it as a piece of paper that you had shown to someone, then thrown into thrash. What is left is a memory of what’s written on it.

Templates

Let’s talk about templates before talking about something even better.

As previously mentioned, in mobx realm React is just a tree-renderer, nothing else. You should really not put that much thought on it. The special thing about React is, those things take time to write, experiment and understand. There are billions of virtual DOM implementations on the internet now, if not trillions. React is just a very simple, very light-weight implementation, battle-tested on one of the most advanced UIs in the history of Internet, TheFacebook.

But seriously, you don’t really have to think about it. Ideally, you only want to test the state; the rest should be solid.

Eventually, this brought me to the idea of a templating language on top of mobx/mobx-state-tree. I think my ideal application has only a state definition of some sorts, an index.js to glue things up, and a bunch of layout templates.

So, xus is not much different than a usual template engine. The only difference is, it is specifically designed to work with mobx observables.

There weren’t many haikus about xus yet, so I wrote one:

A React.Component is a tree

a mobx-state-tree is a tree

So, go ahead and give xus a try!

“A state definition language”

Finally.

I was sitting there and thinking why simple things should be limited and how I can finish this write-up, I realized that data-types are also limited. There is only so much you can do with them.

Can we go further from there and create a minimalistic functional programming language for defining the state?

Let’s start with the simplest. The simplest is, of course, a boolean type. So how would you describe a boolean’s relationship to another boolean in a single sentence? You could say they are symmetric, or you could say,

A is inverse of B

And, here you go! You defined a reaction! Or I think so, I am not really sure.

How would that work indeed? And what about other types?

What about:

A filters B

or

C sorts A by B.x

In the coming weeks, I will experiment with this idea and let you know the results :) I think, we can form such a familiar and expressive-enough language for defining a state which could replace a runtime in this context.

What do you think?

Final words en shout-outs

So, this post pretty much sums up my experience in the Netherlands since the day I stepped foot in the low lands (except the day I came to work after taking Ayahuasca) (ok, just kidding 😅 this is not serious!)

These days, I think I may have taken a great risk, and I only have 10-weeks left to find that out *sigh*

Regardless of that, I had great time at Mendix. A world-class team of developers, I must say. Besides, they gave me shelter and soup and put me on a desk where I sat next to the creator of mobx, Michel Weststrate, and opposite to one of the co-authors of the UML standard, Jos Warmer. Surreal 🏄

And the final shout-out goes to Meinte Boersma, (ex) Team Lead of Mendix Web Modeler. Meinte is an exceptional software engineer and lead. And he also gave me the Dragon Book as a present. ️Do I need to tell more? :)

Meinte and his wife recently welcomed their second son, Jamie! So, I want to congratulate them also from here! 🎉 👶 ✨

So, that’s it. Hope you enjoyed!

July 17, 2017 — programming, reactive and xus

This post will introduce you to xus, the templating language I’ve been working on recently which brings the simplicity of mustache and the expressiveness of mobx-state-tree together, and makes laying out reactive user interfaces incredibly easy.

Recently I came up with an idea I think could save the world. It’s a regular mustache-y template engine which has a lexer, parser and all of that (streaming though) but instead of transforming HTML into regular old HTML again, it compiles them into an observer React component tree. That is to say, given a state object (constructed by mobx or mobx-state-tree or whatever state container implementation that you have with similar characteristics) it will re-render the tree efficiently when something has changed.

Not the most original idea, I know, but it was so tempting to implement something like this on top of mobx, and I am kind of curious about how this could change the way I write UIs.

Here’s an example of a partially finished todo app:

It starts with a template written in xus/mustache which shows a list of todo items and the number of completed ones:

<div>
  <p>You have completed <b>{completedCount}</b> of your tasks.</p>
  <p><b>Click on more tasks to finish them!</b></p>
  <ul>
    {#todos}
        <li class="{#done}finished{/done}" onclick="toggle">{title}</li>
    {/todos}
  </ul>
<div>

If you have worked with mustache or any other templating language before, this should be really straightforward.

xus is so simple, there aren’t many original features yet :) It supports mustache spec with the exception of lambdas and partials.

In mustache, this is a section: {#todos} … {/todos}. The behaviour of the section is determined by the current context. According to the spec, if todos value is of type boolean, then the HTML between the pound and slash will either be displayed or not. If it is a non-empty Array, then it is rendered one or more times. And finally, when it is an object, the value of it is used as the context for a single rendering of the block.

Nested inside the todos section, there is a li tag with title variable in it, and there is also the done section on class attribute which wraps a class name; and then another string, toggle on onclick.

Furthermore, we have completedCount on top where a string will be expanded (just like title) if the key name is found in the current context.

Looking at this template, you can more or less tell what will happen. For example, you can say, finished class will only be assigned if done is set to true. But there is no information here that tells our compiler the type of done, whether it is a boolean or not. So this will only get you so far if you are not running inside a very smart VM, many VMs shouldn’t be that smart anyway. And as for toggle, it could be anything. But you know that it’s just a reference to something else. A reference to an event, or maybe an action name.

So, we need something else that tells us about the data, how to render it and interact with it as well. We need a context.

What is missing? Types of course! Let’s start with them. Let’s focus on a single Todo item first, it should have a title and a done value. So when typed, it looks something like this:

{
  title: string
  done : boolean
}

Next, we need that toggle action which will negate the value of done:

toggle: function() {
    this.done = !this.done
}

And, congratulations! You have just created a mobx-state-tree model:

import { types } from "mobx-state-tree"

const Todo = types.model("Todo", {
  title: types.string,
  done: types.boolean
}, {
  toggle: function() {
    this.done = !this.done
  }
})

Let’s create another one.

This time, call it State with a single value of type Array which will hold a list of Todos. Ah, and before I forget, we are creating this model for the purpose of getting the number of completed tasks; so, we also need an accessor for computing that value:

const State = types.model("State", {
  todos: types.array(Todo),
  get completedCount() {
    return this.todos.reduce(function(count, todo) {
      return todo.done ? count + 1 : count
    }, 0)
  }
})

Finally, let’s create a State instance with an initial value:

const state = State.create({
  todos: [
    { title: "Get coffee", done: false },
    { title: "Wake up", done: true }
  ]
})

That’s it! We now have enough information for rendering the layout. With xus, it is super easy!

xus exposes a default function which expects a template string and a state object. Using xusify transform you can just require() your template files and they will be returned as pre-compiled render functions with this signature: function render(state, options)

Using xusify is faster because you don’t need to parse your templates again. xusify already does it for you during compilation. This is also the reason why it’s possible to emit very tiny bundles with it, because you don’t have to include parser in the final build. (In fact, it is currently mostly optimized for that!)

See xus and xusify readmes for more information about the usage.

Rendering with xus

Assuming you have saved your layout as layout.html, this is how you require() it as a stand-alone render function:

const render = require("./layout.html")

This looks very similar to a functional React.Component, and actually it is. It expects a state and an options argument which are both required.

Since we don’t use setState anymore, you should translate the state argument to props in your head.

Ok, let’s finish this up!

Given our state, following code block shows how you can create a React-ive component tree and render it into DOM using ReactDOM.render. xus does not ship with React or mobx, that’s why you need to provide React.createElement and mobx.observer methods in the options object as well:

import { createElement } from "react"
import { observer } from "mobx-react"

const tree = render(state, {
  createElement: createElement,
  observer: observer
})

ReactDOM.render(tree, document.getElementById("main"))

And, that’s it! Your partial todo application is ready.

Furthermore, it’s not magic! There is so much information there.

See the full example code here.

See the example on CodePen.

See API documentation.

Status

I’d say, at this point, xus is somewhat web-turing complete. Of course, there is still so much else to do! Optimizations, more tests, adding new language constructs and such. One new addition could be a simple range operator for example, e.g. {#todos:10–20}. Have a look at the Issues page for the planned features.

So, yes, please go ahead and create cool things with it. This is not a competitor to JSX or whatever, but it is improving :)

If you are curious why I’ve started this project and how this could play a role within a larger framework, take a look at “Templates, state trees, a school of ‘whatever’ and a state definition language”.