Adventures in client-side routing with re‑frame and OAuth

It’s been a while since my last post; one of the few disadvantages of working at a company you like a lot* is that as your work projects get more interesting, the urge to hack on external stuff in the interim diminishes. But having had some downtime and considerable anxiety to burn off in the last week or two, I revived one of my old open-source ideas: Haunting Refrain, a little single-page app that gets your Foursquare check-in history and then builds a random Spotify playlist out of random data from the places you’ve been to.

This is still very much a work in progress, but I’m pretty pleased with its direction and it uses a lot of intriguing tech. It’s all written in ClojureScript, and it uses re-frame for the basic single-page app control flow and UI bits. re-frame was written by Mike Thompson, probably Clojure’s greatest living essayist, and is an excellent package somewhat in the elm / redux / FRP-if-you-squint vein. I’m also now using datascript to retain more domain focused data and posh to wire it up to the UI elements, about which more later.

In a previous post, well over a year ago now, I talked about some difficulties I was having with dealing with external OAuth-style authentication in single-page apps. Having obtained a great deal more experience with client-side routing since then, I managed to solve this fairly quickly in Haunting Refrain (if you don’t count the time I spent bashing my head against similar problems while at work as trying to figure it out, I guess).

My current set-up uses pushy to handle the HTML5 history setup and sibiro as a routing library, largely because it’s the only client-side routing library whose README I can read without the risk of developing a migraine (honestly, client-side routing should not be very complex, what’s the deal?). I define a big route table which uses keywords for every route; each entry has a URL-matching pattern and a reference to a reagent component which will be used to render the given page.

On the re-frame side I keep a value in the app-db called :route/current-page which keeps track of the keyword matching the current URL. Whenever the URL has been changed, whether by the user landing on the page for the first time or from a link being clicked or the app itself redirecting the user, pushy will dispatch a new re-frame event of the form [:route/changed route-keyword route-params-if-any]. The handler for that route just persists those two values in the app-db, and then in the view code there’s a subscription which pulls out the two values, checks the big route table for the component matching the current route keyword, and renders the component, passing it the page parameters in case it needs them.

So that bit is pretty straightforward, and I’m pleased with how declarative the routing table winds up. For the spotify and foursquare OAuth callback pages, I’m cheating a little bit. When the user needs to authenticate, he or she will be redirected to Foursquare, will hit the “allow access” button, and will then be redirected back to a specific callback URL on Haunting Refrain. Back on the site, pushy parses the callback URL as :foursquare/hello, and the component which is associated with that route renders a blank page and dispatches an event when it is mounted into the DOM. The handler for this event parses the OAuth access token out of the URL, saves it in the app-db, persists it into HTML5 LocalStorage, and then redirects the user back to the home page at :main/index.

The LocalStorage bit is an interesting side track. Since the user winds up seeing a full page refresh whenever he or she is redirected for authorization, the application essentially has three entry points where it needs to construct the user’s state from scratch: on the home page, and then on the authorization callback pages for both Spotify and Foursquare. Since, in theory, we need to store access token for both services, the volatile re-frame app-db is not going to cut it. In an earlier version of the application, you could authenticate to Foursquare, get an access token, then authenticate to Spotify, at which point a full page refresh wiped out the Foursquare token.

Persisting stuff to LocalStorage wound up being pretty easy, and relies on the new re-frame effect and coeffect system. I have a re-frame effect :persist! which, when returned from an event handler, will write a value to the browser via the hodgepodge library. A matching :local-storage coeffect pulls it out of the browser, and I’m using this during app initialization to seed the database with any previously-retrieved access tokens. This works great, and should make it quite easy to persist arbitrary data across application invocations.

Overall I’m quite happy with re-frame. I’ve found that it can be a little difficult to track the control flow once a system gets reasonably complex, since you’re essentially encoding most of it by means of constructing a large bespoke FSM, but it does a fantastic job of keeping the control flow and display logic separate, and it is fairly easy to tinker with.

I’ll have more to say about Haunting Refrain in the days to come—the routing stuff isn’t actually one of the more interesting parts of it, the datascript and posh bits are. The thing is still buggy as heck and not deployed anywhere, but it’s runnable locally. Check it out!


* Workframe – we’re hiring!


I wrote a thing: cljs-datepicker

I spent a little time today messing with date pickers for my eventual single-page app, and wound up with something general enough that it seemed worthwhile packaging it as a library.

To wit, I published cljs-pikaday, a ClojureScript interface to the Pikaday JavaScript date-picker. So far it just has a reagent interface, but I still plan to add an Om interface and a re-frame interface, at least. You can see an online demo at github.

After having worked with a lot of websites spawned from lein new templates in the recent past, I found the eventual project.clj file for the library to be shockingly small. It winds up being very easy to publish a library on clojars (as a caveat, I still need to test the published artifact in a separate project to make sure I didn’t mess everything up – I know my published jar has an extraneous directory in it, for one thing). I found the easiest way to develop the library was by starting with a lein new reagent template, getting the basic functionality to work, and then moving the generated stuff into its own template, save the actual library files.

The interface itself seems to be adequate for my needs, if not perfect. You can basically pass arbitrary atoms in to the pikaday/date-selector component and have the date-picker synchronized with the atom, so that user selection updates the atom and atom updates change the selected date of the picker.

I’m still puzzling over what a humane re-frame interface would look like. One could easily dispatch methods using the existing callback interface, but I’m not really sure what the best way to express subscriptions that the picker should listen to is.

Authentication, state and single-page apps in ClojureScript

I’ve been spending a lot of time recently working on a single-page app in ClojureScript, most recently using the recently-released, and impressive, re-frame project, which builds an FRPish unidirectional data flow on top of the ClojureScript react interface reagent. By a single page app, I mean one with almost no server-side code, which could theoretically be served from a static HTML page, and which only lives at a single URL (modifying the #fragment part of the URL to navigate between logical “pages” in the app, as is the vogue these days).

I’ve tried out a few different approaches to writing this app, including with Om and plain reagent, and re-frame seems to have just the right combination of simplicity and abstraction for my taste. With reagent I sometimes get the feeling I’ve had in backbone (JavaScript) projects, that there’s not really very much structure in the framework and I have to make a lot of stuff up on my own, which is fine but raises the question of why I’m using a framework at all. Om seems pretty neat, and definitely has a structure to it, but I’m not crazy about the ways it relies on async channels to communicate things between components. (In fairness, I’ve probably spent the least amount of time with Om, partly because the tutorials for it seem a little esoteric.)

At any rate, the app as it currently exists is pretty simple. There is a home page, from which the user is prompted to log into an OAuth service (in my case, foursquare). When the user hits the link, he or she will be redirected to foursquare to authorize my app, after which he or she will arrive back at my site with an authorization token in the URL (this is the “callback URL”, cf foursquare). I need the token in order to pass it along to foursquare for API requests I subsequently make to get the user’s check-in history and so forth.

With a full-stack application this would be pretty easy – my server-side code could look for the callback URL, and when it’s found it could grab the token, redirect the user to a known “thanks for logging in” page, and pass the token along to the user’s browser in a cookie, or embedded on the page, or though any number of other well-known server/browser mechanisms.

With a purely client-side app, however, the situation is trickier. In general, the user is expected to stay on a single web page the entire time he or she is using the app, without refreshing; application state sort of accumulates over time in the DOM and JavaScript object models on the page. But since the user needs to redirect to foursquare’s site in order to authenticate, my application now effectively has two entry-points: one when the user first navigates there, and one when the user returns from authentication. Indeed, since in the long term I want my app to authenticate against at least one more OAuth service, and probably several (spotify, twitter, etc), it will have an arbitrary number of entry points.

This complicates managing user state, since the user’s previous state (as reflected in the page’s object model) will be completely destroyed when he or she leaves the site for authentication. Upon the user’s return, the sum total of his or her state will essentially consist of the callback URL, including the authentication token. This is somewhat manageable with a single OAuth provider; when the user hits the callback URL, our client-side code can stash the token in the page’s object model somewhere and then use something like history.replaceState() to modify the URL back to the landing page. With more than one OAuth provider this approach is problematic, since the state will disappear when the user hits the second provider.

So we need a way to persist information between page refreshes. The two most obvious methods are cookies and HTML5 localStorage, and I will be using localStorage (or sessionStorage) because it is new and shiny. With that said, there are still two viable approaches I can see to this.

  1. User chooses to authenticate to foursquare, is redirected to foursquare, and then arrives back at the site with an empty app state and an auth token in the URL. The client-side code stashes the auth state in localStorage and uses replaceState() to navigate back to the home page.
  2. When the user hits the “log in” button, we open a popup. In the popup, the user is redirected to foursquare. When the user comes back to the popup page after authentication, the popup sets the token value in localStorage and closes itself. When localStorage is set, this will trigger a "storage" event in the parent window, which can then react by updating its state (to say “thanks for logging in” or the like).

There are a lot of things I like about the second approach. Because the original page persists while the popup is open, it can just trundle along as it had before, waiting for the storage event to fire; this simplifies state management in the parent window. However, it has a two considerable drawbacks:

Firstly, desktop browsers tend to block popups invoked from javascript these days, and honestly, thank god they do. There is probably a way to work around this, maybe by using an <a target="_blank"> tag or the like, but most of my experiments so far have triggered the popup blocker in my browser.

Secondly, popups kind of suck in mobile browsers. They work, more or less, but they don’t feel native to the mobile experience.

In passing, I’ll note that using an iframe might also be a technical solution to the two above problems, but I don’t really want to because (a) the user should see the address bar in an OAuth situation to validate that they’re not on a phishing site, and (b) ick, iframes.

So it seems like a straightforward redirect is the way to go. In my next installment, I’ll dig a little deeper into what this means for state management in the app, how that works with re-frame, and into client-side routing in ClojureScript generally.

The random album cover project

I find that having a hands-on project is very helpful for me when I’m trying to learn to things in programming, and I’ve needed a fairly simple project to help me learn Scala for some time now.  Some of my earliest ideas for toy projects were a little too ambitious.

I was trying at one time to write a small Android app in Scala, but Android itself is a fairly complex new system that I was also learning simultaneously, and having poked around at it for a bit I’ve decided that it will probably be best to learn Android in its native Java before spilling chocolatey Scala into its rich peanut butter APIs (in addition to that, I don’t relish the prospect of adding a ProGuard step to my compilation cycle).

My second idea was to produce a framework for solving some classic sets of computer sciencey problems, to wit the 99 Prolog Problems and possibly (though my math skills are sadly lacking) the Project Euler problems.  This is still something I’m interested in doing (for my own reference, the Python wiki also has some promising problem sets).  I quickly ran into some trouble with this, as well.  It might be possible to come up with a set of specs / ScalaTest / etc tests which could then just be run as a suite, with the user filling in the implementations for particular problems, but I came to realize that what I really wanted was a sort of unit-test like framework that would detect test implementations and automatically run them over given sets of verified input and output data.

This is a worthwhile goal, and I still would like to do it (FWIW, I’ve also dabbled in writing such a framework for clojure, though it’s been collecting a bit of dust recently).  The problem is that it’s a fairly advanced problem; doing it the way I envision it involves a level of metaprogramming and reflection that isn’t really suited for a first project.  Compounding that, in Scala figuring out the proper type definitions of the individual tests might well be an interesting part of finding solutions to the exercises; if I define it myself in unit test definitions the problem sets may well lose a bit of their didactic panache.

While I was ruminating about these matters I chanced upon a sort of low-level internet meme which had never quite broken through to the big leagues: the random album cover algorithm (here’s a partial history).  Basically, you combine a random wikipedia page, a random quotation, and a random flickr image  to produce a random album cover.  The original flickr pool seems to be dead now, but examples abound.

As a meme, this frankly does not appear to have the legs of, say, Nyan Cat or Rick Astley.  As a project for learning Scala, though, it’s well-nigh perfect:

  • It can be structured as a classic three-tiered web application.
  • It involves using publicly available HTTP APIs, plus some HTML scraping and file downloading.
  • The project is simple and small enough that it should be easy to quickly try out different back-end stores as well as different web frameworks and toolkits; this will hopefully also get me to learn the Cake Pattern and other DI equivalents.
  • The application itself is easy to scale up in complexity, with the first version just getting the three data items, the second combining them to generate a boilerplate cover, and then adding UI and image processing to allow the user to style the result somewhat.
  • The image processing bit could involve Scala / Java interoperability (assuming there are more Java image-processing libraries than Scala ones, which frankly I have no idea about).
  • Eventually the thing could get ported and deployed to a GAE-like cloud environment.
  • It could potentially be ported over to an Android app as well.

So.  Here is the github repository.  What I have got so far is a very basic Scalatra web application which keeps an in-memory album cover list.  It doesn’t currently actually do any HTTP calls right now, about which more later, but it’s a decent starting point, and I feel that I’m well on my way to actually doing something somewhat fun in Scala.