From Flux to Redux: Async Actions the easy way

The Flux architecture is based on some great ideas (check out our visual cheatsheet if you want to know more), but we found a big flaw when implementing it through Facebook’s own Dispatcher implementation. This brought us to Redux, a flux-inspired library that received high praise from the Flux creator herself.

Redux is very young and really tiny at 2kB, but already incredibly stable and feature rich. It is also a very non-opinionated library, and the just completed (almost) docs are very keen to point out how you can do things in many different ways. This is great if you already have your strong preferences, but doesn’t help the newcomer.

One thing some of us found particularly hard to digest was the documentation for Async Actions, so I wrote a simplified version to use with my team. You can find it below; it is based on the Todo List example in the “Basics” section of the docs: if you didn’t read that already, you should do it now.

Async Actions the easy way

As opposed to Flux, In Redux action creators must be pure functions with no side effects. This means that they can’t directly execute any async code, as the Flux architecture instead suggests, because async code has side effects by definition.

Fortunately, Redux comes with a powerful middleware system, and one specific middleware library, redux-thunk, that allows us to integrate async calls into our action creators in a clean and easy way.

First of all, we need to integrate the redux-thunk middleware into our store. We can do this in the Todo List example by changing a few lines inside index.js; from this:

import React from 'react';
import { createStore } from 'redux';
import { Provider } from 'react-redux';
import App from './containers/App';
import todoApp from './reducers';

let store = createStore(todoApp);

To this:

import React from 'react';
import { createStore, applyMiddleware } from 'redux';
import thunk from 'redux-thunk';
import { Provider } from 'react-redux';
import App from './containers/App';
import todoApp from './reducers';

let store =  applyMiddleware(thunk)(createStore)(todoApp);

Before this change, store.dispatch could only accept simple Action objects. With redux-thunk, dispatch can also accept a thunk function.

  • If dispatch receives an Action object, redux-thunk will do nothing, and the reducers will get called with the action as usual, and change the state synchronously.
  • If instead dispatch receives a thunk function, redux-thunk will execute it. It will be the thunk function’s responsibility to actually dispatch an action object. More importantly, the thunk function doesn’t need to be pure, so it can contain async calls, and can dispatch some (synchronous) action only after the async call has finished, also using the data received from the call. Let’s see this with a simple example for the Todo app.

In a real app, we’ll want to keep our Todo list synchronized with our server. For now we’ll only focus on the addTodo action creator; the one from the Basics section will only update the state in the client. Instead, we want it to:

  • Optimistically update the local state and UI
  • Send the data to the server, so it can update the DB
  • Manage the server response

To achieve this, we still need our “optimistic” addTodo action creator, but we also need an async thunk action creator to handle the whole async process. To keep the changes to the code at a minimum, we’ll rename the optimistic action creator, but not the action type that is sent to the reducers (and used in the switch statement to act). This code goes into actions.js:

// renamed optimistic action creator - this won't be called directly 
// by the React components anymore, but from our async thunk function
export function addTodoOptimistic(text) {
  return { type: ADD_TODO, text };
}

// the async action creator uses the name of the old action creator, so 
// it will get called by the existing code when a new todo item should 
//  be added
export function addTodo(text) {
  // we return a thunk function, not an action object!
  // the thunk function needs to dispatch some actions to change 
  // the Store status, so it receives the "dispatch" function as its
  // >first parameter

  return function(dispatch) {
    // here starts the code that actually gets executed when the 
    //  addTodo action creator is dispatched

    // first of all, let's do the optimistic UI update - we need to 
    // dispatch the old synchronous action object, using the renamed 
    // action creator
    dispatch(addTodoOptimistic(text));

    // now that the Store has been notified of the new todo item, we 
    // should also notify our server - we'll use here ES6 fetch 
    // function to post the data
    fetch('/add_todo', {
      method: 'post',
      body: JSON.stringify({
        text
      })
    }).then(response => {
      // you should probably get a real id for your new todo item here, 
      // and update your store, but we'll leave that to you
    }).catch(err => {
    // Error: handle it the way you like, undoing the optimistic update,
    //  showing a "out of sync" message, etc.
    });
  // what you return here gets returned by the dispatch function that 
  // used this action creator
  return null; 
  }
}

The above code and its comments show how you can do async calls, mixing them with optimistic updates, while still using the usual syntax: dispatch(actionCreator(...)). The action creator itself is still a pure function, but the thunk function it returns doesn’t need to be, and it can do our async calls for us. If needed, the thunk can also access the current store state, as it receives getState as its second argument: return function(dispatch, getState) { ...

That’s all! If you want to learn more about Redux, you can find the full docs here: http://rackt.github.io/redux/

Flux Architecture Visual Cheatsheet

At hashtagify.me we recently started experimenting with the React library, and, right from the start, I really liked what I saw.

After that, I also went to study the Flux architecture, and, honestly, I found the official overview and tutorials rather confusing.

After studying those better, and after some more reading around, I think I understood where the confusion came from. Flux is itself – just like the whole Javascript ecosystem – in a period of very intense flux, and the docs aren’t really keeping up with that.

I realized that the Todo List tutorial – the first one you’re supposed to learn from, as I dutifully did – illustrates an older version of Flux, and isn’t even up to date to the recent changes to the Todo List example code on github. I’m pretty sure the Facebook team will fix this soon, but for now this really doesn’t help.

Anyway, in the end, I understood that Flux is much simpler than many wordy explanations make it look. To spare my team my initial pains in getting it, I created a diagram – a visual cheatsheet – to better illustrate how it works, based on the latest code examples by Facebook. The team found it useful, so I decided to give back to the community and publish it here.

flux-architecture-cheatsheet-thumbnail

It’s meant to be a companion to the official docs and tutorials, not a complete explanation. But most of the core concepts are there. The code snippets are from the Chat example app.

Enjoy!

EDIT: some people are having problems with the mural.ly link or want to download a simple image. This will be uglier when zoomed and without links, but here you go with a simple PNG export: Flux Architecture Visual Cheatsheet (png)

UPDATE: after trying to use Flux with Facebook’s own Dispatch library, we decided to instead use Redux. You can read some background in my new post about Redux Async actions the easy way.

Redis: Migrate Keys From One Instance To Another Based On Pattern Matching [node.js script]

If you work with Redis and a fairly big dataset it’s pretty likely that, at some point, you’re going to fill up all your RAM. This already happened to me more than once with hashtagify.me; the easy fix, if the growth is slow enough, is to just add more memory.

Simpsons_Duff_Lite_DryWhen that doesn’t work anymore, the next move – while waiting for Redis cluster – is quite likely going to be using some manual sharding technique. When you do, you’ll probably going to need to split your dataset among more instances.

This is exactly what just happened to me, and I didn’t find any ready recipe or solution to easily split my rather large dataset. So I ended up creating a small node.js script which doesn’t use too much memory and that you can find here: https://github.com/hashtagify/redis-migrate-keys

Enjoy!

Hacker News: Mark All Read Source Code Released

TL;DR: I just released under an MIT license the source code of the Hacker News: Mark All Read Chrome extension. You can find the code on github.

Background: I often thought about open sourcing many of the little projects I create from time to time, but until now I never went on with it. The reason: I’ve got pretty strong perfectionist tendencies against which I’m always struggling, and releasing unpolished code that I’ve quickly hacked up to solve some practical little problem doesn’t bode well with those tendencies.

But with this one I got many feature requests that I already know I won’t have time to implement. So I decided to publish the code anyway and see what happens. Just consider it what it is: A quick hack I created in one afternoon, and polished just a little bit the day after. Have fun with it!

PS: I thought about it and decided not to release the “artwork”, because having more extensions on the Chrome market with the same look would be confusing. The extension works anyway as it is, but if you add a button icon in an “images” folder it will look better.

To discuss on HN: link

OpenVZ howto addendum: Migrating your virtual machines to a new server

As I wrote some time ago, creating your own OpenVZ infrastructure is very easy, and it can save you a lot of money compared to the usual VPS providers. But it also has other advantages.

Hashtagify uses redis to store all its data in memory, which is great for performance, but… it requires a lot of RAM. So it happened that I almost finished my 24GB, right when I learned about a new great offer for a 32GB server which costs 33% less than my 24GB one. Moving to it means that I’m going to get back my 150€ of activation costs in just 5 months, while at the same time having 8GB more. Great!

This is why I bought a new server and decided to move all my virtual machines to it from the old one. It turned out to be quite easy, but I had to find the instructions in different places, so I’ll post here how I did it for both my future reference and for anybody else having the same problem.

First of all, you have to install OpenVZ: I did it just following my previous howto.

Then, you can migrate your machines. I thought I would need to back them up and then restore them in the new server, but I found out there is a more direct and much faster way to do it: The vzmigrate command.

To use it, you first need to enable password-less ssh login from the old server to the new one (this is because vzmigrate uses the ssh protocol to move the files). First, generate your public and private keys if you didn’t already:

ssh-keygen -t rsa

It is important that you create it with no passphrase. Then, move the public key to your new server, like this:

scp /root/.ssh/id_rsa.pub root@XX.XX.XX.XX:./id_rsa.pub

Now log into the new server and do

cd .ssh/
touch authorized_keys2
chmod 600 authorized_keys2
cat ../id_rsa.pub >> authorized_keys2
rm ../id_rsa.pub

After this, in the old server check that everything is ok with:

ssh -2 -v root@XX.XX.XX.XX

If everything is ok, you whould be logged in without being asked for a password. You can log out

exit

and start migrating your machines. For each machine, for example the one with id 101, you do:

vzmigrate -r no XX.XX.XX.XX 101

This will take a pretty long time, depending on how big the virtual machine and how fast the net between the two servers are. There is no progress indicator, but you can get an idea how it is proceeding by checking the original size of the machine and the size of the copied one, by using the command

du -hs /vz/private/101

on the old and the new server respectively (this of course only if you mapped the openvz data directory to /vz as shown in the original howto.

When the command finishes, the new machine is ready… but only if you could move the VM IP from the old to the new server. In my case that wasn’t possible, so I had to ask for new IPs on the new server. You can assign the new IP, and remove the old one, like this:

vzctl set 101 –ipadd NEW.NEW.NEW.NEW –save
vzctl set 101 –ipdel OLD.OLD.OLD.OLD –save

So, now you’re really good to go:

vzctl start 101

And that’s it!

Cheers

Update CSS rules with 5 lines of CoffeeScript

I recently (ie, today) had to find a way to update a CSS rule dynamically in javascript. jQuery, my library of choice, doesn’t offer this functionality – and I really wonder why – so I looked around a little and found how to do it with plain javascript.

As I’m actually using CoffeeScript, I decided to rewrite it in that language and make it a generic function; it could be useful to others, so, there you go:

@updateCSS = (selector, styles) ->
  for sheet in document.styleSheets
      for rule in sheet.cssRules || sheet.rules || []
        if rule.selectorText == selector
          for style, value of styles
            rule.style[style] = value

You can also find an usage example (and play with it yourself) on jsfiddle. Enjoy :)

Javascript async testing tutorial: Jasmine + Rails 3.1 + Coffeescript

I recently wanted to create automated testing for coffeescript code in Rails 3.1, including some async (ajax) code. I had to dig around quite a lot, so I’m writing here what I ended up doing for future reference for myself and others who might need it.

There are many test frameworks for javascript, and the one I chose is Jasmine. Reason why: It is one of the two most used frameworks, and the other one is dependent on jQuery. I’m using jQuery in this project, but I’ll be able to use Jasmine for future projects without jQuery too, so I chose it – I also like its syntax.

You can integrate Jasmin in Rails 3.1 yourself just using the Jasmine gem, but to simplify things with Coffeescript I decided to use a specific gem for the latest version of Rails and its assets pipeline. I couldn’t really choose between test_track and jasminerice, which are both very young, so I went with the first one just based on intuition. Using the other one shouldn’t be very different.

So, I added to my gemfile:

gem "jasmine"

gem "test_track"

and installed the new gems with “bundle”

I then created a spec/javascripts and spec/javascripts/helpers folder in the project tree. As I wanted to be able to easily test jQuery, I downloaded jasmine-jquery.js from github and copied it into spec/javascripts/helpers.

At this point I created the following jasmine.js file om spec/javascripts:

// SUITE :: Jasmine
// SUITE MANIFEST :: spec/javascripts/jasmine.js
// TEST SRC DIR :: spec/javascripts
//
//= require helpers/jasmine-jquery
//
//= require JsonDataSpec

JsonDataSpec is the coffeescript spec file that I was going to create and that will be executed in the test. See the following example

spec/javascripts/JsonDataSpec.js.coffee:

describe "JsonData", ->
  it 'should should have the correct format', ->
    answer = null
    $.getJSON(
      "/controller/get_my_data/",
      (data) ->
        answer = data
    )

    waitsFor ->
      answer

    runs ->
      expect(typeof answer.string).toEqual('string')
      expect(typeof answer.number).toEqual('number')

The secret here is waitsFor -> answer, which will wait until the answer variable gets a non null value – that is, when the ajax call is complete. Only after that the actual test is run.

Last thing, I configured the test_track engine adding this code to routes.rb:

mount TestTrack::Engine => "test"

And that’s it! To use my tests, I just run the server using rails s and access the tests opening http://localhost:3000/test/jasmine in the browser. Smooth and easy.

Happy testing!

Saved by the magical powers of GIT

Yesterday evening I was very happy: My first mobile app, after a little more than a week, was practically finished. It just needed some little finishing touches.

At that point I decided to take a look at it on my phone, after a few days during which I had only tested it on the Android emulator, which allows much faster debugging cycles. So I connected my phone, started deploying the app, waited, started the app… and… crash! It didn’t even open the first screen.

At 12:10 AM and with a long day at my day job waiting for me the next day, I tried to redeploy the app; then, to restart the development environment and redeploy the app; and, last try, to restart the phone and my pc and then redoploy the app. To no avail: It worked perfectly well on the emulator, but on my phone it just wouldn’t start.

So today, after coming home from work, I prepared myself for a LONG debugging session. My plan was to find the latest version on GIT that worked on the phone, and understand what went wrong after that – hoping that there was only one thing that went wrong.

First, I started going just a couple of commits back – I commit very often – to check that the problem was there. It was. Then I tried to remember when I last tried the app on my phone – it had always worked – and checked a version from around that time. I was a little apprehensive by now… but it worked!

I then started the methodical search for the time when the problem came out, so I tried the commit at the middle of the road between the working version and the not-working one. It worked. Then, I again found the middle of the road, and tested. OK. And again: OK. And again: OK! This was just two commits before the not-working one.

At this point, I went up one commit only. It worked. Another one, the last one before my first test (which failed). It worked. Was it possible that, just by chance, I had tested first exactly the commit where the problem arose? Well, I tested it again, to be sure and… surprise, surprise… it worked!! I’m pretty sure that, however much Dijkstra might have liked them, I didn’t use any non-deterministic programming techniques, so I was a little flabbergasted.

Next try, I checked out the master; deployed it; and it worked!

How can I explain it? I’m sorry, but I just can’t. And, as tomorrow I’m going to leave for a short vacation, I don’t even have the time to find out now. I suppose (and hope) that it was just some weird problem in the Titanium compilator, and not in Android itself. If it isn’t either of the, well, I guess I’ll have to suggest adding “Magic” to the list of GIT features.

App Development: Appcelerator vs (Phonegap + (JQuery Mobile vs Sencha))

Mobile apps are all the rage now, but to create cutting edge solutions on mobile platforms, at least for now, you need to adopt native development. So, as I have an idiosyncrasy  for Java and never developed on (not even used) an Apple platform, and lacking a real motivation to start, until now I kept myself away from this field.

Last week though, after reading once again a couple of inspiring article on Hacker News, I started thinking that trying out one of the various cross platform, javascript based solutions for mobile development on a lightweight app could be a pleasant distraction from my current main (night) projects. I even started fantasizing that, if it was really possible to create a web app using just “html5″ technologies, I could have been able to create a complete app (without too much polish) over just one weekend.

A fairly simple yet useful candidate for a lightweight app was to create a mobile front end for my adjustable rate mortgage calculator. Since all the complicated financial stuff is done on the server, and I already created a simplified web service for the free embeddable version (widget), if I could really use the usual web technologies on the phones my “aggressive” (ie, crazy) deadline seemed feasible.

So this is why, on a Friday evening after my day job, I started studying in depth all those solutions that I had just lazily kept an eye on during the last months. I think that what I learned can be interesting to others, so I’ll expose my findings here.

The contenders

My requirements for the ideal development platform were simple: To be as close as possible to usual web development, good enough for the job at hand, able to work on both Android and the iPhone, and possibly free or at least very cheap (as this was just a side show).

After some hours doing a literature review (ie, googling around), I came up with a short list of three candidates:

  1. Phonegap + JQuery Mobile (JQM)
  2. Phonegap + Sencha Touch
  3. [Appcelerator Titanium]

The order of the above list isn’t random. The first combination (Phonegap + JQM) was, on paper, my perfect solution: Based on JQuery, it really is web development for the phone, and it would have allowed me to even reuse part of my current code base.

Sencha Touch is another “html5″ solution (meaning that it creates html GUI objects, not native ones), and from what I read it is quite faster than JQM, but on the downside GUIs have to be created procedurally – something I started hating with the MFC, and never got myself to like.

Last, Appcelerator Titanium is javascript based, but it uses native controls. This should guarantee the best performance of the three, but, in addition to having only procedural GUI programming like Sencha, this also means that things sometimes have to be different on Android and on iOS, so this isn’t cross-platform of the “write once-run anywhere” kind. And, last but not least, I had found an alarming article about some nasty memory leaks that seemed almost a show stopper (that’s why I used the square brackets on the list – at one point Appcelerator was more out of the list than in).

There were also other interesting propositions (like rhomobile – I love Ruby), but they looked less viable for one reason or another and I won’t go over them here.

Testing

What I wrote until now is only based on my “literature review”. With my preferences made up, I decided to try the Phonegap/JQM solution hands on.

Setting everything up was easy enough. The problems came when I tried the “kitchen sink” test app on my Android phone (HTC Wildfire). The performance was terrible, and the rendering often incorrect. The same happened with a showcase app downloaded from the Android Market.

So, that was it for me. As much as I wanted this solution to work, my conclusion is that it is just not production ready yet, at least not for my goals and not for the current generation of not high-end phones. Good bye, reuse of existing code…

I then went on to Sencha. This time, I wanted to check the performance before doing any installation. So I looked for any success story for Sencha on Android, and luckily I found a very recent and very relevant article on techcrunch. When I read that “… it wasn’t easy, performance was a huge issue [and] at times it took days to figure out, and simple things like too much logging and poorly constructed ‘for’ loops actually made our app unusable during our journey.” I didn’t feel very optimistic.

So I installed on my phone the Diary app from the android market. Better than JQM, but still sluggish and with layout problems on my screen size too. And this after a long and painful optimization work, according to the article… not for me, not now. So, Sencha too wouldn’t fit my requirements.

At this point I was pretty disappointed, as I saw my fantasy of being able to complete an app over the weekend slipping away from my grasp. But, not wanting to give up, I decided to look better into Appcelerator: At stake was not only my simple app, but the possibility to develop more interesting ones in the near future!

First of all, I decided to take a look at the existing Titanium-based apps. Among their showcases there are many more iOS apps than Android ones, but I was able to install a couple of good examples on my phone; GetGlue, in particular, looked and felt really good.

But what about the memory leaks? After some more search, I found out that you can avoid them following some guidelines (see the comments in the alarming article) and, if still necessary, using some workarounds that would be perfectly acceptable for my current needs. I’m also pretty confident that in time Appcelerator will fix these problems (if they want to stay in business).

Conclusion

In the end, I downloaded the Titanium Studio and started coding. So far things went pretty smoothly, and, even if I didn’t create my full app over one weekend, I already solved 70% of the technical obstacles – it can already get the needed input from the user, send it to the server, get the response, and show some data in simple pure html bar graphs. What is still missing is a lightweight enough solution to show the payments time series, but I’ll start on that as soon as I have time.

What I learned from this very industrious weekend is that the news about the impending death of native app coding are greatly exaggerated, and that (surprise, surprise!) a LOT of what you read all over the internet about the marvels of html5 development is BS. But I also learned that, even if there are no silver bullets flying around, developing for mobile in a pretty much cross platform way with just one programming language is indeed possible, and that procedural GUI programming, on the simple UIs that you can create on phones, isn’t as painful as I remembered from developing fully fledged desktop applications.