Into the weeds

So having come up with a decent if not outstanding scaffolding for random-album-cover, I’m now faced with actually making it do what it’s supposed to.  (Actually, there are several additional items on the immediate TODO list, including figuring out unit tests.)  This basically boils down to making three HTTP calls per new album cover, all of which are slightly different.

  • The call to get a quotation is a straightforward HTTP request; we’ll retrieve the response body, scrape it for the third-from-last quote, and store the quote (I’m thinking the entire thing, even though we’ll only be using the last several words for the album title).
  • The call to flickr should probably use their REST APIs; it looks as though flickr.interestingness.getList() is probably what we want here.  One added wrinkle is that this will introduce the first deployment artifact into the application (to wit, my flickr API key – at least I think it will).
  • The call to get a random Wikipedia page will result in a 302 redirect to a new page.  I imagine that just parsing the resulting target URL for the random page name will suffice for now, rather than loading the new page up and scraping it for the title.

None of this is terribly complex.  So how do we do it in Scala?  The Scala books I’ve got don’t really cover any network programming in detail; looking around the web, it seems like there are basically two well-supported approaches: using Java’s Apache HttpClient, or using the native, idiomatic Scala library Dispatch. (Caveat: a few other libraries exist, but as far as I can tell they are generally either small personal projects or else bits of larger frameworks.)

My first impulse was to use Dispatch, but I ran into a bit of a wall in figuring out exactly how to bend the API to my will.  Dispatch definitely seems like it’s pretty neat, and there is some documentation about how it works, but there seem to be woefully few examples of how to, say, send a request and check the HTTP response code.  It may be that my knowledge of the underlying implicit stuff that drives request / executor / handler generation is just hampered by my sub-par understanding of Scala implicts in general, but I’ve basically been unable to find documentation that is more comprehensive than this page plus this page, and as a newcomer the emoticon-esque operators give me some pause.

That’s not to say that I’m incapable of just messing around until I figure it out, of course, and one luxury I have in this project is that I don’t have a tight deadline for a deliverable.  Right now I’m sort of wavering on the edge of either spending some time tinkering with Dispatch in order to get it working, or else starting from a very imperative-style “java + HttpClient + syntax sugar” first draft, and then coming back to swap out Dispatch for HttpClient and taking the whole project in a more functional direction from there.  I’ve had similar thoughts about potentially moving the MVC stack from Scalatra to Unfiltered down the road some once it’s stable.

Note: on using HttpClient in Scala I’ve turned up a few links for future reference.  This page links to several online examples.  Here is another one.  This person recently published a very small client / Scala wrapper.  Here’s a REST client example in Dispatch, here’s another focused on testing with some explanatory apparatus, and another Dispatch-related example.

About these ads
Next Post
Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: