In my spare time I’ve recently been learning some Elixir, and I was excited to come across the Nerves project, with which you can create a Raspberry Pi firmware image that essentially boots directly into the Erlang BEAM VM. It’s an impressive system that works really nicely for embedded systems development, which is one of Erlang’s traditional strengths. The project I’m working on is controlling the Unicorn Hat HD, a little 16×16 LED grid, from a Raspberry Pi via the Elixir/ALE hardware-interfacing library. You can see my project’s source on GitHub, for what it’s worth, though it’s still not functional.
One potential roadblock when trying to develop on a remote Raspberry Pi is figuring out how to view console output from the device, especially if it’s running headless. After poking around some a bit I found a solution that seems to work well and was easy to set up: the Pi ships all of its logs to the Papertrail log-aggregation service, which has a free tier that should be adequate for anybody doing hobbyist development.
Below are the instructions for doing this. As a note, I’m running on the latest versions of Elixir, Phoenix and Nerves (1.5.3, 1.3.0, and 0.8.3 at the time of this writing), and I’m using the excellent nerves_init_gadget library to get mDNS set up on the Pi and to push firmware updates to it over ssh.
Also worth noting: my project is set up as a poncho project, as recommended in the nerves documentation. I’ve got a directory ui which contains the Phoenix web server code, and then a parallel directory fw containing the configuration and setup for the Nerves system itself.
Sign up for papertrail
This bit is straightforward: navigate to the Papertrail home page and sign up for a free account. After you sign up you’ll be at the “add systems” page, which gives you a hostname and port you will use to connect from Nerves. The page shows you an “install script” which you shouldn’t run – it’s for forwarding syslog events to Papertrail, which is not what we’re doing here.
Add the LoggerPapertrailBackend library to your dependencies
The LoggerPapertrailBackend library sends messages from Elixir’s standard Logger API to Papertrail. In my development cycle I typically run Phoenix on my local machine until things seem reasonably stable, and then build and push the firmware to the Pi when I want to test it out on its eventual destination.
During local development, all the console output is right there in the terminal so we don’t particularly need Papertrail. Therefore, we set up the Papertrail stuff in the fw project, so it will run on the Pi but not locally.
Per the instructions on github:
Add the dependency to fw/mix.exs. I added it under the deps(target) section, which seems to work fine.
def deps(target) do
[
# [...]
{:logger_papertrail_backend, "~> 1.0"}
] ++ system(target)
end
Configure the logger
You’ll want to set this up in your firmware project’s config/config.exs.
config :logger, :logger_papertrail_backend,
url: System.get_env("PAPERTRAIL_URL"),
level: :debug,
format: "$metadata $message"
config :logger,
backends: [ :console, LoggerPapertrailBackend.Logger ],
level: :debug
Two things to note here:
- We’re sending
:debuglogs to Papertrail. This is helpful to get started, but a lot of the debug output from Nerves is more chatty than useful – in particular, you’ll see a lot of chatter from thenerves_init_gadgetmDNS server and the WiFi modules that you probably won’t care about. - We’re storing the actual Papertrail URL in an environment variable and then plugging it into the configuration at compile time. This corresponds nicely with how Nerves controls what targets you build for via environment variables (
MIX_TARGET, etc). Submitting logs to Papertrail over UDP doesn’t seem to require an API key, so keeping the hostname and port out of your public source code will hopefully prevent Joe Internet from writing stuff to your logs.
Set up your environment
This is basically as simple as:
export PAPERTRAIL_URL='papertrail://logsX.papertrailapp.com:XYZZY/my-name'
Here we’re constructing a (fake) URL with the hostname and port that Papertrail gave us after we signed up. The my-name bit can be anything you want; Papertrail uses it to distinguish different applications connected to the same log sink. You’d probably want to vary this name to distinguish between production and dev builds, different servers of a multi-server system, etc.
In practice, I’ve got a file in my home directory ~/nerves-build, which I source in order to set up environment variables I need to build the firmware. Mine currently looks something like this:
export MIX_TARGET=rpi3 export NERVES_NETWORK_SSID="(the ssid to my wifi netowrk)" export NERVES_NETWORK_PSK="(the password for the same)" export PAPERTRAIL_API_TOKEN='(my papertrail API token, see below)' export PAPERTRAIL_URL='papertrail://logsX.papertrailapp.com:YYYYY/narwhal.rpi'
About the Papertrail API token, it’s not actually needed on the Pi itself, but it’s useful for viewing logs in the console via Papertrail’s CLI tools, see below. You can find yours on the Papertrail site under “Settings / Profile”.
Build and deploy the firmware
Given the above environment variable setup, building the firmware is as easy as running mix firmware from the fw directory. You would typically then get it onto the Pi by inserting an SD card, running mix firmware.burn, and popping the SD card into your Pi. If you have nerves_init_gadget set up, you can instead deploy over ssh by using mix firmware.push my-rpi-mdns-name.local, which is less of a hassle.
Fire it up and watch the logs
You’re ready to go, connect the power and wait for some logs to show up in the Papertrail event viewer. With the above configuration, you should at least see some mDNS :debug events in there if everything is working properly.
Papertrail also has a handy CLI interface you can use to see the logs it’s ingesting in realtime, if you prefer a console UI to a web UI. You’ll need to set up the API token as above, then you can just use papertrail -f to view your logs as they happen.
Tone it down
Once you’ve verified that the connection works, you’ll likely want to set the log level of your :logger_papertrail_backend config settings back to :info so you aren’t drowning in mDNS logs. (Actually, ideally you’d want :debug logs from your own application and :info logs from everything else. I’m sure there’s a way to do this, but I’m not sure how to do it quite yet.)
Tim Gilbert's Blog 