Initial Jasmine impressions

Ok, I must confess that midway into my half-baked project of comparing Javascript testing frameworks, I’m getting a little bored with the entire project. Largely this is due to the fact that of the three major BDD-oriented frameworks I’ve seen so far, the syntax and semantics are roughly the same. Really the only major difference I’ve seen so far is that Mocha warns me about global namespace pollution, and the other frameworks do not.

As a point of reference, I was able to take the nascent set of tests I’d come up with for BusterJS and port them over virtually unchanged to Jasmine, with a few simple syntax changes relating to the setup / teardown methods. I’ll probably finish out the tests in one framework or the other and then move on to more interesting things, like figuring out the whole module mess that seems to be the current state of the art.

Advertisements

BusterJS vs Chai/Mocha

BusterJS, a node.js-oriented test framework, is notable for having a lot of ways to run tests, including one where it fires up an HTTP server that your browser connects to, as well as a way, seemingly, to attach several browsers to a single running test harness. Having opted against exploring the whole node.js / browser module dichotomy up until now, I skipped all of that stuff and went straight to the instructions for just running the thing in a browser. (I should note in passing that the test-runner Mocha, covered earlier, also has a lot of non-browser-focussed running modes which I didn’t cover, among which the most intriguing is the Nyan Runner).

Anyways, the browser stuff was quite easy to set up, and I returned my focus to writing tests in the BusterJS idiom.  As it turned out, there wasn’t all that much different about it.  BusterJS comes with a BDD-oriented test mode which functions almost exactly like Mocha’s does, and it also sports a BDD-flavored assertion style which is very similar, if a bit less tricksy in its syntax.

To illustrate, here’s a snippet of the Mocha/Chai tests:

describe('WeightedList', function() {

  describe('Constructor', function() {

    it('should succeed with no arguments', function() {
      expect(new WeightedList()).to.be.ok;
    });

    it('should throw errors on bad inputs', function() {
      var badConstructor = function() {
        return new WeightedList( {'wrong': 'field names'} );
      };
      expect(badConstructor).to.Throw(Error);
    });
  });
});

And here is their BusterJS equivalent:

var spec = describe("Weighted Lists", function() {

  describe('Constructor', function() {

    it('should succeed with no arguments', function() {
      expect(new WeightedList()).toBeDefined();
    });

    it('should throw errors on bad inputs', function() {
      var badConstructor = function() {
        return new WeightedList( {'wrong': 'field names'} );
      };
      expect(badConstructor).toThrow();
    });

  });
});

As you can see, the broad strokes are virtually identical. The main difference I’ve noticed is that Chai uses a little more trickery to get its assertions into a quasi-DSL style, with statements like result.should.have.length(1); and new WeightedList().should.be.ok; as opposed to BusterJS’s more conventional expect(result.length).toEqual(1); and expect(new WeightedList()).toBeDefined();.  While I appreciate the vigor with which the Chai folks were able to bend Javascript’s syntax to their will, I actually find the more conventional BusterJS syntax to be a little easier to read, since it looks basically like I expect Javascript to look.

Overall, these assertion frameworks both seem adequate for testing Javascript code. I’m really not thrilled about the way one tests code for exceptions in either language, where it’s necessary to pass in a function which is expected to throw an error, but I can’t really think of a better way to do it in Javascript. I’m going to continue porting the tests over to BusterJS, but I don’t know that I’ll have all that much more that’s interesting to say about it. If you’re interested in the nitty-gritty, the details can be found by contrasting the Chai and BusterJS assertion documentation.

I should probably also note in passing that the particular uses I’m putting the testing frameworks through are a little unusual. Most people writing tests won’t be testing a pure-javascript library with no user-interface whatsoever, and a lot of the appeal of the various frameworks seems to be the integration they offer with various other pieces of the Javascript ecosystem.

With that in mind, my next task is to finally figure out what the whole Javascript module system is all about, and likewise to figure out how I can package the library I’ve got into something usable in a node.js context. Meanwhile, based on a preliminary reading of the AMD vs CommonJS pedagogy in the blogosphere, my idea of just including the library code on a web page via a <script type="text/javascript" src="js-weighted-list.js"/> tag is laughably naive, and I’ll need to add a compilation step in there, because what is Javascript if not an Ada-style B&D language which benefits from the intense scrutiny of static program analysis?

In any event, adding a compilation step will probably give me a ready excuse to jump over to CoffeeScript.

Forging on with javascript testing frameworks

I’ve got what I consider to be a decent set of unit tests in Chai, though I’m certainly a good deal away from a complete set. (As an aside, the inherently non-deterministic nature of js-weighted-list presents something of a testing challenge in itself, totally apart from the particular framework in use; I’ll have more to say about this in future.)  My impressions of it aren’t radically different from how I felt about it last time – I’m still not crazy about the syntax, but it has offered me one or two pleasant surprises when it found scope bugs I had missed.

There’s another  aspect of the framework I’ve enjoyed as well. As some background, I’m mostly investigating BDD frameworks, but I have to confess that I don’t really see how BDD is all that different from unit-test based testing, apart from the terminology in use; frankly the difference between a test suite and a spec doesn’t seem all that large to me, and I’m dubious about the vague claims of readability to non-programmers that seem to haunt the rhetoric of some BDD sites I’ve seen. But anyways, according to the classic BDD (or TDD) approach, I’m going about things backwards: I’m writing a suite of tests to cover code which already exists instead of starting with the tests and then writing code to satisfy them.

Certainly I’m not the first programmer in the history of the world to approach testing this way, but I did enjoy adding in some undefined specifications for a few missing features for the library and seeing them show up as elements in the list:

    it('should increase the length of the list');
    it('should validate its weights');

Mocha testrunner with pending specs

Seeing the tasks come up with little blue dots in the test-runner output, and then change to red and then green as I implemented first the test code and then the implementing code in the library was a pretty satisfying process, and I can definitely see its appeal. This is a whole aspect of BDD I’ll probably miss out on in this comparison, since I’m not planning on reimplementing the library itself over and over again. In recognition of this, though, I’m hoping to think up some features and hold them in abeyance so I can try more specification-first type development.

At this point I have a decent feel for Chai, and I will move on to the next framework; I’m thinking I’ll try out BusterJS, since from a cursory examination of Jasmine it looks to be very similar to Chai. Another thing which is bubbling up towards the top of my priority list is trying out the coveraje library, which measures test coverage for javascript unit tests. However, as a node.js only project, this will require me to figure out how to get the same test suite and module system running on the command-line and the browser, and I’d just as soon keep my focus on the browser for the moment.

First few stabs at Chai

Continuing on my investigation of what’s what in the state of Javascript testing frameworks, I downloaded Chai and ported several of my existing Qunit tests over to it. Chai is more or less a plugin for Mocha, which is a test harness framework that allows you to organize your tests, while leaving the meaty assertion bits of testing to outside libraries. The decoupling of these two aspects of a testing framework seems like a pretty neat approach, and I may investigate some of the other assertion plugins available for Mocha besides Chai in the future, assuming I’m not completely burnt out on testing frameworks by then. Meanwhile, Chai itself has a plugin infrastructure, with hooks available that expand its vocabulary or integrate it with other libraries such as jQuery and Backbone.

I haven’t got too far yet, but I think I can safely say that I’m not in love with the syntax. Chai exposes two main verbs, which do about the same thing. expect(foo).to.be(5); is the more obvious syntax; optionally Chai can instead attach the function should() to the global Object prototype, which allows assertions like this: foo.should.be(5);. Overall combined with Mocha’s BDD test style, you wind up with code like this:

describe('WeightedList', function() {

  describe('Constructor', function() {
    it('should succeed with no arguments', function() {
      expect(new WeightedList()).to.be.ok;
    });
  });

  describe('Empty List', function() {
    var wl = new WeightedList();

    it('should have length zero', function() {
      wl.should.have.length(0);
    });

    it('should give an empty list on shuffle', function() {
      wl.shuffle().should.deep.equal([]);
    });
  });
});

I suppose it’s readable enough, maybe I just still haven’t gotten over Javascript’s relentless use of function(){} blocks.   I will say that seeing some similar tests (but for Jasmine, a competing framework) written in CoffeeScript is certainly easier on the eyes.

Error reportIn any event, despite my trepidation at the syntax, Chai has already helped me in one way I hadn’t expected: it found a bug in my code.  Running the test that calls shuffle on an empty list, the test runner produced the error “Error: global leaks detected: heap, result.”  Sure enough, looking at my code, I’d left off the var declarations from two variables in one of the inner methods there.  Adding in the var statements caused the test to pass again.  Thanks Chai!  As I experiment with other frameworks I might try taking the declarations out again to see whether anyone else will catch the same thing.

 

Javascript unit testing

(I’m still here. During my time away from the blog I got a job. Hopefully things are about to be calmer and I’ll be updating more frequently.)

I spent a while burnishing up the weighted-list javascript library I wrote last spring, in the process catching up with the latest goings-on in Javascript. I have a real love-hate relationship with Javascript the language and its software ecosystem, but that’s probably a topic for another day; for now I guess I’ll just snarkily mention hasOwnProperty() and leave it at that.  In any event, the ascendency of Node.js seems to have sparked a mini-renaissance in software packages written in javascript. Things are moving pretty rapidly and there aren’t necessarily clear favorites in a lot of areas, so you wind up with a great variety of alternatives for various types of useful libraries.

The resulting competition is probably good for the overall quality of javascript open-source software, but it can be a little difficult to determine which packages are good to use right now. All of which is a lead-up to say that I’ve been spending some time looking into Javascript unit-testing frameworks.

js-weighted-list currently has a light suite of unit tests written in Qunit.  Qunit seems like an adequate framework overall, and being able to run my tests in a browser is pretty convenient, but it has a few aspects I’m not crazy about: the documentation is a little thin on the ground and I’m not crazy about the actual syntax of the tests, especially for tests where I’m expecting the code under test to throw an exception. I’m also not sure that Qunit is the new hotness any longer, which is of course of paramount importance in Javascript these days.

With that in mind, I’m going to try, as an experiment, to reproduce the minimal suite of unit tests I’ve got in a few other frameworks and see which ones I prefer.  The candidates I’ve got in mind so far are MochaChai, BusterJS, Jasmine, and Pavlov.  I’m not completely sure I’ll have the attention-span necessary to do a full-on shootout between them, but hopefully I can at least try a few out and see.  The weighted-list library is currently small enough that it’s pretty easy and fast to test, and as a bonus, I will hopefully shake some bugs out of it along the way.

While I’m on the subject, another idea I’ve got is to fix up a page that will run selections thousands of times and graph the results, so that I can have some idea as the whether the library works correctly or not.  But that’s another topic for a different day.

Mysterious 404 responses with foursquare OAuth2 + google-app-engine

I’ve done a lot of work on my aforementioned foursquare / last.fm mashup project, which I’m still not exactly ready to start crowing about, but whose source is, as ever, available.

I’m currently hitting a bunch of problems in getting my Python code to work while it’s deployed out to Google’s appspot.com production environments.  I’ve been able to authenticate successfully against foursquare from my local environment, following their straightforward instructions, but every time I try to run the same code out in Google’s cloud, once I hit the point where I retrieve an access code from foursquare and make a request for a legit session token, this is what I wind up with:

    HTTP blarg, feh
    Traceback (most recent call last):
    File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
      return method(*args, **kwargs)
    File "/base/data/home/apps/s~how-you-been/0-2-4.357616581640073459/whence.py", line 97, in get
      accessCode = self.getFoursquareAccessToken(code)
    File "/base/data/home/apps/s~how-you-been/0-2-4.357616581640073459/howyoubeen/Foursquare.py", line 49, in getFoursquareAccessToken
      httpResponse = urllib2.urlopen(url)
    File "/base/python27_runtime/python27_dist/lib/python2.7/urllib2.py", line 126, in urlopen
      return _opener.open(url, data, timeout)
    File "/base/python27_runtime/python27_dist/lib/python2.7/urllib2.py", line 400, in open
      response = meth(req, response)
    File "/base/python27_runtime/python27_dist/lib/python2.7/urllib2.py", line 513, in http_response
      'http', request, response, code, msg, hdrs)
    File "/base/python27_runtime/python27_dist/lib/python2.7/urllib2.py", line 438, in error
      return self._call_chain(*args)
    File "/base/python27_runtime/python27_dist/lib/python2.7/urllib2.py", line 372, in _call_chain
      result = func(*args)
    File "/base/python27_runtime/python27_dist/lib/python2.7/urllib2.py", line 521, in http_error_default
      raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

Dang it!  My initial instinct is that I’m hitting a timeout in Google’s cloud environment, but are they honestly passing a 404 back to my stack at that point?  (I suspect, as an alternate explanation, something about my error-handling handlers are deficient.)

While looking into this I’ve stumbled upon a few Python OAuth2 libraries, about which OAuth2 isn’t exactly rocket science, but if the libraries have a decent interface and can overcome this issue, maybe I’ll give them a shake.  (On the js side of the world I’ve been having a few similar issues, where I’d like to move to an open-source library but the ones I’ve seen make odd assumptions about the execution environment.)

Another possibility I’ve considered is that there’s some sort of SSL wildcard issue at play, as suggested by this stack overflow answer which foursquare has bravely elected as their replacement for tech support.  But for now, it is a mystery.

Generating weighted-probability lists in Javascript

js-weighted-list test page screenshot

I’ve made a good deal of progress on my Google App Engine application, about which more later.  During the course of it, I started to keenly feel the need for a weighted random list.  That is, I have a list and want to select a random sample from it, but it has an unequal (weighted) probability distribution; I want some of the items to be selected more frequently than others.  I also need to modify the weights depending on what else is going on in the program (eg, in response to user input or web service results.  Also, this is all happening on the front-end, so it needs to work in javascript.

Poking around for a while I couldn’t find anything obviously relevant to this, but I did find a pretty good breakdown of the problem and the algorithm to follow in this Stack Overflow answer.  This implementation makes use of a heap in order to enable a binary search on the probability space.  While digging through to understand it, I wound up reimplementing the thing in Javascript, and then made a bunch of improvements to the API for it.  It’s pretty basic and doesn’t have any particular dependencies, so I’m releasing it as its own library, js-weighted-list.

It is available on github under the MIT license.  There are still several things I need to improve with it, in particular it’s severely under-tested, but it’s decently documented and immediately usable for what I need it to do.

Using pyjade with webapp2 on Google App Engine

And now for something completely different. I got a little distracted again by an idea for a simple social networking mashup I’ve had and decided that the best place to deploy it would probably on Google App Engine.  App Engine supports Scala and I’ve seen one or two positive reports about using Scalatra on it, so I thought it wouldn’t be too hard to spin something up quickly using my existing code.

It turned out not to be as easy as that, unfortunately; I may try to elaborate on the reasons in a later post, but it seems to come down to Scalate always checking the filesystem for its template routines.  In any event, I found that I was spending a lot of time trying to glue weird bits of infrastructure to one another rather than working on my idea.

With that in mind, I decided to start over in Python, and of course as my first task I spent a long time glueing random bits of infrastructure together.  In this case I’ve been successful, though, so here’s a brief writeup on getting Jade templates to work with Google App Engine and Google’s webapp2 in python.

1. Add dependencies to include webapp2’s jinja2 module in your project.

This is as simple as adding this bit of code to app.yaml:

libraries:
- name: jinja2
  version: latest

2. Install pyjade in your environment.

I’ve been using a layout similar to the one mentioned in this Stack Overflow answer.  It’s puzzling that Google doesn’t really mention anything about a standard way to get libraries and stuff from the wider Python ecosystem installed in a development environment, but essentially I set up a virtualenv, ran pip install pyjade, and then symlinked the pyjade directory from the virtualenv lib directory into the project’s src directory.

3. Add a custom Jinja2 factory.

This is where the magic happens.  We write a factory function, jinja2_factory, whose purpose is to add pyjade’s included Jinja2 extension into the Jinja2 instance’s Environment.

  def jade_factory(app):
    j = jinja2.Jinja2(app)
    j.environment.add_extension('pyjade.ext.jinja.PyJadeExtension')
    return j

Then, in our handler’s jinja2 method, we pass along the factory method to it:

  @webapp2.cached_property
  def jinja2(self):
    return jinja2.get_jinja2(app=self.app, factory=jade_factory)

(This is based on moraes’s Stack Overflow answer here.)

4. Use the factory and get a template.

Here’s the full source code for a simple “hello world” app. I’m following the examples on Google’s jinja2 page and creating a JadeHandler subclass of RequestHandler.

import os
import webapp2
from webapp2_extras import jinja2

class JadeHandler(webapp2.RequestHandler):
  # Per http://stackoverflow.com/a/7081653/87990
  @staticmethod
  def jade_factory(app):
    j = jinja2.Jinja2(app)
    j.environment.add_extension('pyjade.ext.jinja.PyJadeExtension')
    return j
  
  @webapp2.cached_property
  def jinja2(self):
    return jinja2.get_jinja2(app=self.app, factory=JadeHandler.jade_factory)

  def render_response(self, _template, **context):
    # Renders a template and writes the result to the response.
    rv = self.jinja2.render_template(_template, **context)
    self.response.write(rv)

class MainPage(JadeHandler):
  def get(self):
    context = {'message': 'Hello, world!'}
    self.render_response('index.jade', **context)

app = webapp2.WSGIApplication([('/', MainPage)], debug=True)

Caveat

I’m by no means an expert in Google App Engine, and I’m a little worried that there’s something badly inefficient or similarly wrong about this approach. In particular, I’m hoping the returned templates are cached, but I’m not certain, and I’ve seen reports that Jinja2 Environment creation is an expensive operation on GAE. More to the point, pyjade seems to work as a preprocessor for Jinja2, so it would obviously be better not to need to run it for every request. In any event, this does at least find jade templates and render them properly as HTML.

Edit: as usual, the current code is available on GitHub.

The cake falls

(I’m running out of snappy cake-related puns here, but I’m saving “the cake is a lie” for a special occasion.)

So after thinking about my interface / implementation dilemma for a while (recap: I’m aiming to separate out the implementations of hexmap from their interface), I decided to approach it from a Java vantage point and see how I’d implement the same thing in Java.  The immediately obvious thing that jumped out at me was that the specific problem I was dealing with was sort of a classic application of plain old vanilla inheritance from an abstract class.  Having drunk deeply from the well of Spring over my last several years of Java experience, plain old inheritance had begun to seem in some ways like a quaint relic of a bygone era (having been replaced in many cases by service composition).

At any rate, a quick bit of refactoring to get rid of traits altogether got me back to a fairly sane codebase, where I’ve got this:

abstract class HexMap (h: Int, w: Int) {
  val height = h
  val width = w
  // This is where the magic happens
  def data(addr: Address): HexData
}
abstract class HexData(a: Address) {
  def toJson(): JObject = JObject(List()) // default: empty JSON object
}

case class ColorData(a: Address, color: String) extends HexData (a) {
  override def toJson(): JObject = {
    ("color" -> color)  // json {"color": "foo"} via JsonDSL
  }
}
class RandomColorMap(h: Int, w: Int) extends HexMap(h, w) {
  private val store: Map[Address, ColorData] = createRandomMap()
  def data(addr: Address): ColorData = {
    store(addr)
  }
  // Note, we're able to access width and height here
  private def createRandomMap(): Map[Address, ColorData] = {
    val pairs = for (x <- 0 until width; y <- 0 until height) yield {
      val addr = Address(x, y)
      addr -> ColorData(addr, randomColor())
    }
    pairs toMap
  }
}

I will come back and revisit this later; I’m certain there’s a way traits could be useful in this code, but for now I’m leaving them aside and pressing on with more actual program functionality.

As far as that is concerned, my next worry is persisting the hexmap the user sees in their Scalatra session, so that when the user asks for a random path from the server, he or she gets one within the actual bounds of the actual map he or she is seeing. The Scalatra session docs make this look fairly straightforward; we’ll see.

More delicious cake

I’ve made some progress on both the client and server sides of the hexmap project.  On the client side, there’s now some code to highlight tiles (though it’s still imperfect).  I’ve also updated the map representation some, so it looks like this:

[ {"x":1, "y":1, "data":{"color": "yellow"}},
  {"x":1, "y":2, "data":{"color":"white"}},
  {"x":2, "y":1, "data":{"color":"white"}},
  {"x":2, "y":2, "data":{"color":"lightgreen"}} ]

The idea is that the data object will expand to have more game or application-specific data.  I’m not 100% sure that moving to a sparse representation (versus a 2d matrix) is a good idea, but I don’t think it will hurt me all that much, and it shouldn’t be too difficult to change back if I need to.

On the Scala side of the fence, I’m using the lift-json library to perform serialization to JSON.  I’m still not sure I’ve structured this in a way that really makes sense in the long-term.  One thing I’m hoping to achieve with this project is to wind up with a fairly abstract, generic hexmap library which can be augmented by application-specific code to implement a particular game or the like.  So I’m trying to partition off some of the specific stuff, such as in this case tile colors, into its own conceptual space, but I’m not sure I’m doing it idiomatically.

In particular, I think I’m hitting a slight impedance mismatch between Java and Scala revolving around traits, self-type declarations, and the Cake Pattern versus interfaces and Spring-style dependency injection.  I’d like to separate out the interface of a hexmap from the implementation used to actually implement the hexmap, so what I have is this:

class HexMap (h: Int, w: Int) {
  // Every instance of a HexMap will be mixed in with a HexStore
  self: HexStore =>

  val height = h
  val width = w

  def hex(x: Int, y:Int): Hex // Return data for a specific cell
}

And then HexMapStore implementations derive from this:

abstract class HexData(a: Address) {
  def toJson(): JValue // return a json representation of this data
}
abstract trait HexStore {
  def data(addr: Address): HexData  // Get the specific data from addr
}

My idea for this was that there might be one implementation which uses a MongoDB store, one which uses an in-memory map, etc. The fundamental operation, though, is to pass in a grid address and retrieve the data for that cell. The actual implementation I’ve got now is this:

case class ColorData(a: Address, color: String) extends HexData (a) {
  def toJson(): JValue = {
    ("color" -> color)  // json {"color": "foo"} via JsonDSL
  }
}
trait RandomHexMapStore extends HexStore {
  private lazy val store: Map[Address, ColorData] = createRandomMap()
  def data(addr: Address): ColorData = {
    store(addr)
  }
}

I think where I’m hitting problems most is trying to figure out how exactly the actual instantiation of “a HexMap with a RandomHexMapStore mixed in” should take place, or in other words where the definition of the specific component assembly I’m creating should appear.  In Spring this would be an object graph defined either in a (probably big, crazy, and unwieldy) XML file or via annotations on various class declarations.

What I’ve been trying for starters is just to define a factory method which returns a new random map:

object RandomHexMapStore {
   def create(height: Int, width: Int): HexMap = {
     new HexMap(height, width) with RandomHexMapStore
  }
}

This is working OK for now, and it makes sense to me that different implementations of HexMapStore might have different constructor parameters, so I’m not too bothered by factory methods in general. I’m a little worried by how this approach might or might not scale to much larger class / object graphs, though.

Moreover, so far I’ve been glossing over the most serious immediate problem I have with this: the implementation of the RandomHexMapStore.createRandomMap() method. This method is meant to construct a map composed of randomly selected color elements. The problem is that, being a trait which is mixed in to HexMap, is has no access to HexMap’s height and width class parameters. What I have at the moment is this:

  private def createRandomMap(): Map[Address, ColorData] = {
    // Well, this isn't ideal
    val pairs = for (i <- 1 to 6; j <- 1 to 6) 
                yield Address(i,j) -> 
                      new ColorData(Address(i,j), randomColor())
    pairs toMap
  }

I’m a little baffled by how to get access to class parameters from a trait which is mixed in to that class. When I think about it, I can’t really see a good reason for Scala to allow it, since, after all, RandomHexMapStore could be mixed into any class, not just a descendant of HexMap. This suggests to me that I am probably not using the proper language mechanism for what I’m trying to do. I will have to do some thinking on this, and possibly call on Stack Overflow for help.