Visualizing RRD data using NVD3

Graph all the things

RRDtool is a great way of storing metric data, however the charts it produces look dated now, especially considering how good modern JS based visualization libraries such as D3 have become. I wanted to understand RRDtool better, so this gave me an opportunity to write a front end for RRDTool that uses NVD3 to display RRD data as charts.

The Live Example uses data from the Mosquitto MQTT Server as a source.


RRDViewer is hosted on Git Hub, you can download the zip file, or you can clone/fork your own copy of the repository.

First setup a Django project, Once you have the code, install using the supplied If you are using git, be sure to use the –recursive flag, or otherwise ensure all sub modules are downloaded before running setup.

$ sudo python install

Then add ‘rrdviewer’ to your INSTALLED_APPS list in


Then setup your GRAPH_DIR in to point to a directory containing a bunch of RRD files. RRDViewer will recursively search that directory for RRD files to include.


Once the application is installed, run collectstatic to copy the static files to their correct location:

$ sudo python collectstatic

Finally, setup your to include the urls for rrdviewer at your preferred location:

urlpatterns = patterns('',
    url(r'^rrdviewer/', include('rrdviewer.urls')),

You can now restart the web browser and visit the RRDViewer page.

MQTT to RRD Daemon: Graph all the things!!!

Graph all the things

MQTT2RRD is a daemon written in python that connects to an MQTT server, subscribes to selected topics, and stores values in RRD round-robin databases for later graphing.

The code is hosted on git hub, and is licensed under the GPL.



usage: [-h] {stop,restart,start} ...
optional arguments:
  -h, --help            show this help message and exit
  --config_file CONFIG_FILE
                        The location of the config file
  --no_daemon           Do not spawn a daemon, stay in the foreground


Configuration is via a configuration file, by default mqtt2rrd looks in the following locations:


You can specify the location of the configuration on the command line using the –config_file argument. For example: start --config_file=/home/bob/mqtt2rrdtest.conf

Configuration is stored using an ini file using the python ConfigParser format. The following configuration options are available:


The PID file location, used to store the PID of the daemon process.
Configures the root directory to store the RRD files in. RRDs are in sub directories corresponding to their topic name. For example, a message with topic “foo/bar/baz” would be stored as $DATA_DIR/foo/bar/baz.rrd. The default data directory is /var/lib/mqtt2rrd.
The user to run as, when a username and group is specified, and the script is executed as root, then the script will demote itself to the specified user and group
The group to run as, when a username and group is specified, and the script is executed as root, then the script will demote itself to the specified user and group


Logging is done using the python logging class, at this time you can configure the log file, and the log level.

The level to log at, 10:debug and above, 20:info and above, 30: warn and above, 40: error and above, 50: Critical log messages only. The default is to log at level info and above.
The file to log to, by default this is /var/log/mqtt2rrd.log

Example Log Output

2014-06-20 18:14:02,531: INFO: Connected:

2014-06-20 18:14:02,584: INFO: Connected to server.
2014-06-20 18:14:02,584: INFO: Subscribing to topic: #
2014-06-20 18:14:02,643: INFO: Message received on topic revspace/sensors/co2 with QoS 0 and payload 560 
2014-06-20 18:14:02,643: INFO: Updating: /tmp/revspace/sensors/co2.rrd with value: 560.0
2014-06-20 18:14:02,644: INFO: Message received on topic revspace/sensors/2-t1 with QoS 0 and payload 22 
2014-06-20 18:14:02,644: INFO: Updating: /tmp/revspace/sensors/2-t1.rrd with value: 22.5
2014-06-20 18:14:02,644: INFO: Message received on topic revspace/sensors/2-t3 with QoS 0 and payload 20 
2014-06-20 18:14:02,644: INFO: Updating: /tmp/revspace/sensors/2-t3.rrd with value: 20.5
2014-06-20 18:14:02,644: INFO: Message received on topic revspace/sensors/6-t0 with QoS 0 and payload 20 
2014-06-20 18:14:02,644: INFO: Updating: /tmp/revspace/sensors/6-t0.rrd with value: 20.0
2014-06-20 18:14:02,645: INFO: Message received on topic revspace/sensors/20-t0 with QoS 0 and payload 23 
2014-06-20 18:14:02,645: INFO: Updating: /tmp/revspace/sensors/20-t0.rrd with value: 23.0
2014-06-20 18:14:02,645: INFO: Message received on topic revspace/sensors/2-t2 with QoS 0 and payload 22 
2014-06-20 18:14:02,645: INFO: Updating: /tmp/revspace/sensors/2-t2.rrd with value: 22.75


The client identifier is a 23 byte string that identifies an MQ Telemetry Transport client. Each identifier must be unique to only one connected client at a time. The default is: MQTT2RRD Client.
The username to use when connecting to the server. If not configured, will attempt to connect anonymously. The default is anonymous.
The password to use when connecting to the server. If not configured, will attempt to connect anonymously. The default is anonymous.
The hostname or IP address of the MQTT server. The default is localhost.
The port to connect to on the MQTT server, the default is 1883.
The number of seconds in between sending a keepalive packet to the server. The default is 60 seconds.
A comma separated list of topics to subscribe to, can use # has a wildcard, the default is to subscribe to all topics.

Topic Specific

Topic specific configuration can be made by creating a section with the name of a topic. The following options are supported:

The step time in seconds to use when creating the RRD graph. The default is 60.
A friendly name to give the topic, this is stored in the info file for the RRD, and may be used for reference later.
A comma seperated list of RRA configuration strings to be used when creating the graph. The default is: RRA:AVERAGE:0.5:2:30,RRA:AVERAGE:0.5:5:288,RRA:AVERAGE:0.5:30:336,RRA:AVERAGE:0.5:60:1488,RRA:AVERAGE:0.5:720:744,RRA:AVERAGE:0.5:1440:265

Configration Example

data_dir = /tmp
pid_file = /tmp/
user = mqtt
group = mqtt

log_file = /tmp/mqtt2rrd.log
log_level = 10

client_id = "MQTT_to_RRD_TEST"
hostname =
port= 1883

friendly_name = "My Important Topic"

Measuring engine RPM with Arduino

You can measure engine RPM using the negative terminal of the coil, however you need to isolate the 12v circuit of the coil, from the 5v levels used in the Arduino. This can be done using an optical isolator circuit as detailed below.

The following schematic shows how to connect the circuit:

RPM Module Schematic

Assembly List

The following components, or their equivalent are required to build the RPM module.

Label Part Type Properties
4n25 DIP – 6 pins package DIP (Dual Inline) [THT]; hole size 1.0mm,0.508mm; true; chip label IC; pins 6; pin spacing 300mil
Arduino Arduino Uno (Rev3) type Arduino UNO (Rev3)
C1 Ceramic Capacitor package 100 mil [THT, multilayer]; capacitance 100nF; voltage 6.3V
C2 Ceramic Capacitor package 100 mil [THT, multilayer]; capacitance 100nF; voltage 6.3V
D1 Rectifier Diode package 300 mil [THT]; type Rectifier; part # 1N4001
D2 Zener Diode package Melf DO-213 AB [SMD]; breakdown voltage 5.1V; type Zener; power dissipation 0.5W; part # 1N4732A
R1 390 Ω Resistor package THT; tolerance ±5%; bands 4; resistance 390Ω; pin spacing 400 mil
R2 4.7k Ω Resistor package THT; tolerance ±5%; bands 4; resistance 4.7kΩ; pin spacing 400 mil

Example Code

volatile unsigned long timePointsOpen, timePointsClosed, lastChange;
volatile unsigned int numBangs;

void pointsOpening(){
    unsigned long t;
    t = millis();
    if (lastChange > 0){
        timePointsClosed += (t - lastChange);
    lastChange = t;

void setup(){
  pinMode(3, INPUT);
  digitalWrite(3, HIGH);

void loop(){
  // Number of coil pulses
  numBangs = 0;
  attachInterrupt(0, pointsOpening, RISING);
  numBangs = ((60000/50)*numBangs)/2;
  Serial.print("RPM: ");

Hiker Flap Jack

This is my recipe for flap jack, I use this when I’m hiking, its filling, calorie dense, and tastes good


  • 600g Rolled Oats
  • 300g Butter
  • 300g Sugar
  • 300g Honey
  • 200g Glace Cherries (chopped)
  • 200g Chopped Nuts (Walnuts/Brazil Nuts/Almonds/etc)


  1. Melt sugar, honey and butter in a sauce pan, and stir until the sugar is disolved.
  2. Add all other dry ingredients, mix well until the mixture is a nice sticky goo
  3. Empty into baking tray and bake for about 10-15minutes at 180C until golden on top
  4. Place in fridge until set

This contains about 7400 Calories, and makes enough for 10 GIANT flap jacks weighing about 200g each.

Fun times on the west highland way

The West Highland Way is a Long Distance Footpath running from Milngavie to Fort William. This trail has been on my list for a while, and as its close to home, and I have a spare week, now is the time. Of course, at 96 miles, its not really that long, and would normally take me around 4 to five days to complete. As I need to get back in shape for some peak bagging in the Himalayas, I decided to add some side trips.

The Plan

Day 1

Leave Milngavie following the west highland way for about 20 miles with seven days food. Aim to camp at the campground at Millarochy or in some woodland nearby.

Day 2

Leave Millarochy and take a side trip to climb Ben Lomond, camp nearby Inversnaid.

Day 3

Leave the West Highland way at Bein Glas farm, climb Beinn Chabhair, An Caisteal, Beinn a’ Chroin , Glas Tulaichean and Cruach Ardrain before sleeping in woods near Inverlochlarig

Day 4

Leaving Inverlochlarig and climbing Stob Binnein and Ben More, before descending into Crainlarich for lunch. Post lunch regain the West Highland way towards Tyndrum and turn left towards Cononish, then ascend regain WHW, Ascend Ben Dubhchraig, and camp somewhere nearby.

Day 5

Climb Ben Oss, return to Tyndrum for lunch and a resupply before following the West Highland Way as far as Gleann Achadh-innis Chailean, follow the Glean before climbing Beinn Mhanach before finding a suitable camp spot.

Day 6

Continue on to the summits of Beinn Achaladair, Beinn a Chreachain, Beinn an Dothaidh, Bein Dorain, and return to Bridge of Orchy, Camp on shore of Loch Tulla.

Day 7

Leave Loch Tulla, climbing Meall a’ Bhuiridh, Creise and camp in Kinlochmore, resupplying as required.

Day 8

Leave Kinlochmore, heading east and attain the ridge to the peaks of Sgurr Eilde Mor, Binnein Beag, Binnein Mor, Na Gruagaichean, Stob Coire a’ Chairn, An Gearanach, Am Bodach, Sgurr a Mhaim, Stob Ban, Mullach Nan Coirean camping a few miles out of Fort William

Day 9

Arrive in Fort William, find accomodation, food, shower and resupply.

This gives me the flexibility to skip sections due to bad weather, (which no doubt there will be) or time, but still push quite hard.

The Gear


ULA Catalyst 70L pack

Sleeping System

Hydration and Cooking System

Hygene/First Aid

Small first aid kit containing:

  • Medical tape
  • Bbandage
  • Sam splint
  • Pain killers
  • Suture strips
  • Iodine solution.

Hygene Kit containing:

  • Baby Wipes
  • Toothbrush/Tooth paste


  • GT-2000 Shoes
  • Darn Tough Socks (2 Pairs)
  • Icebreaker Boxers (2 Pairs)
  • Running Shorts
  • Merino T-Shirt
  • Sweater
  • Goretex Pants
  • Goretex Jacket
  • Gloves
  • Hat


Nutrition is an important part of any trip, if not the single most important part, especially when you are pushing for distance.  I wish to lose a few lbs on this trip, so I’ve set a daily calorie target of 3000calories.  Yes, that’s three thousand calories.  The recommended daily intake for an adult male is 2500 calories, that assumes a normal day sitting in the office pushing keys on a keyboard.  On this trip Ill likely burn between 4000 and 5000 calories a day, without at least 3000 calories a day, I’ll simply run out of steam after a few days as my body fails to get the energy it requires.

A typical day will consist of the following:


  • 1 Packet of Pop Tarts, these were my staple on the Pacific Crest Trail, and provide 400 calories per packet (4 packets per box)
  • Banana/Fresh fruit, when available, I’ll add fresh fruit to breakfast, this will likely be the first couple of days.


  • Pork Pie (350 Calories)
  • Sandwich/Burrito 500(Calories)


  • Home made Flap Jack (800 Calories)
  • Chocolate Bar (100 Calories)
  • Trail Mix (200 Calories)


  • Pasta/Rice meal 250 Calories
  • Noodle Soup 250 Calories
  • Pudding or other desert (200 Calories)

Arduino Megasquirt Library

This driver provides access to the MegaSquirt ECU with a level of abstraction above the MegaSquirt serial interface, enabling users with no knowledge of the MegaSquirt commands to access the register data easily without caring about the underlying communication required.

The MegaSquirt is a low cost engine control unit with support for fueling, ignition and idle control.


The MegaSquirt is connected to the Arduino using the RS232 output on the MegaSquirt, a MAX232 chip, and the Serial interface on the arduino board.


#include >MegaSquirt.h<
void setup(){
 byte status;
 String signature;
 switch (status){
   Serial.print("MegaSquirt Signature: ");
   Serial.println(" FAILURE: Communication Timeout");
   Serial.println(" FAILURE: Communication Overflow");
   Serial.println(" FAILURE: Unknown Error"); 

String revision;

 uint16_t uptime;

 MegaSquirtData registers;
 byte regTable[MS_TABLE_LENGTH];


 Serial.println(registers.lambda1()/10,DEC )


You can download the ZIP file to import into the IDE here, or you can fork your own.

Arduino LIS331 Library

The LIS331HH is an ultra low-power full-scale three axis MEMS linear accelerometer.  The device also features ultra low-power operational modes that allow advanced power saving and smart sleep to wake-up functions.  I wrote a library for Arduino that provides an object oriented interface to these devices using the I2C bus.

The project is hosted on GitHub here and is licensed under the GPL.


You can download the ZIP file to import into the IDE here, or you can fork your own.


Be sure to include the LIS331 and Wire libraries in your project first.

setPowerStatus(int status);

Set the power saving status of the device, options are: LR_POWER_OFF, LR_POWER_NORM, LR_POWER_LOW1, LR_POWER_LOW2,LR_POWER_LOW3

setXEnable(bool enable)

Enable or disable X axis

setYEnable(bool enable)

Enable or disable Y axis

setZEnable(bool enable)

Enable or disable Z axis

string lis.getXValue(int16_t &value);

Reads the X value and assigns it to value.

string lis.getYValue(int16_t &value);

Reads the Y value and assigns it to value.

string lis.getZValue(int16_t &value);

Reads the Z value and assigns it to value.


You can download an example project here.

DHT Support for Loguino

The DHT range of sensors provide a cheap way to grab temperature and humidity information. The sensors are not fast, (250ms per query) but have a good price and require only a single digital pin to use.  Kyle had a few sitting around, so I added loguino support for them.

Arduino ELM327 Library

The ELM327 IC is a multi-function OBD2 Interpreter that can be used to query information about most relatively new vehicles.  The ELM327 connects to the vehicle using the OBD2 port, and provides an RS232 Interface.   I recently purchased an OBD2 shield form SparkFun which uses an ELM clone, the STN1110.  I wanted to add OBD2 support to Loguino, but first I needed to be able to communicate using the ELM protocol.

Subsequently I created a the ELM327 class, which contains both high level and low level methods to get information from the controller.  The low level class is used to send arbitrary commands to the ELM chip, there are two main methods,

  • getBytes which requests a specific PID and parses the returned bytes into an array of values.
  • runCommand which sends an arbitrary command to the ELM device, and parses the output into a buffer which contains the response from the ELM.

The high level class provides methods to gather metrics from the controller, each method parses the response from the ELM device into the actual value.  The following code illustrates how to use the class:

#define ELM_TIMEOUT 9000
#define ELM_BAUD_RATE 9600
#define ELM_PORT Serial3

byte status;
Elm327 Elm;
if (status != ELM_SUCCESS){
    Serial.print("Error: ");
int temp;
Serial.print("OBD2 Coolant Temperature: ");
if (status  == ELM_SUCCESS)
    Serial.println ("Pass");
    Serial.print(" Value: ");
    Serial.print("Error: ");

byte values[2];
if (status == ELM_SUCCESS){
    Serial.print("Elm returned: ");
    Serial.print(values[0], HEX);
    Serial.print(" ");
    Serial.println(values[1], HEX);
    Serial.print("Which is: ");
    Serial.print(((values[0]*256)+values[1])/4, DEC);
    Serial.println(" RPM");
    Serial.print("Error: ");

char data[20];
Serial.print("Current version is: ");

Feature requests, bugs, suggestions and other feedback always appreciated.