MQTT to RRD Daemon: Graph all the things!!!

MQTT2RRD is a daemon written in python that connects to an MQTT server, subscribes to selected topics, and stores values in RRD round-robin databases for later graphing.

The code is hosted on git hub, and is licensed under the GPL.

Download

Usage

usage: mqtt2rrd.py [-h] {stop,restart,start} ...
optional arguments:
  -h, --help            show this help message and exit
  --config_file CONFIG_FILE
                        The location of the config file
  --no_daemon           Do not spawn a daemon, stay in the foreground

Configuration

Configuration is via a configuration file, by default mqtt2rrd looks in the following locations:

/etc/mqtt2rrd.conf
~/.mqtt2rrd.conf

You can specify the location of the configuration on the command line using the –config_file argument. For example:

mqtt2rrd.py start --config_file=/home/bob/mqtt2rrdtest.conf

Configuration is stored using an ini file using the python ConfigParser format. The following configuration options are available:

daemon

pid_file

The PID file location, used to store the PID of the daemon process.

data_dir

Configures the root directory to store the RRD files in. RRDs are in sub directories corresponding to their topic name. For example, a message with topic “foo/bar/baz” would be stored as $DATA_DIR/foo/bar/baz.rrd. The default data directory is /var/lib/mqtt2rrd.

user

The user to run as, when a username and group is specified, and the script is executed as root, then the script will demote itself to the specified user and group

group

The group to run as, when a username and group is specified, and the script is executed as root, then the script will demote itself to the specified user and group

logging

Logging is done using the python logging class, at this time you can configure the log file, and the log level.

log_level

The level to log at, 10:debug and above, 20:info and above, 30: warn and above, 40: error and above, 50: Critical log messages only. The default is to log at level info and above.

log_file

The file to log to, by default this is /var/log/mqtt2rrd.log

Example Log Output

2014-06-20 18:14:02,531: INFO: Connected: test.mosquitto.org:1883

2014-06-20 18:14:02,584: INFO: Connected to server.
2014-06-20 18:14:02,584: INFO: Subscribing to topic: #
2014-06-20 18:14:02,643: INFO: Message received on topic revspace/sensors/co2 with QoS 0 and payload 560 
2014-06-20 18:14:02,643: INFO: Updating: /tmp/revspace/sensors/co2.rrd with value: 560.0
2014-06-20 18:14:02,644: INFO: Message received on topic revspace/sensors/2-t1 with QoS 0 and payload 22 
2014-06-20 18:14:02,644: INFO: Updating: /tmp/revspace/sensors/2-t1.rrd with value: 22.5
2014-06-20 18:14:02,644: INFO: Message received on topic revspace/sensors/2-t3 with QoS 0 and payload 20 
2014-06-20 18:14:02,644: INFO: Updating: /tmp/revspace/sensors/2-t3.rrd with value: 20.5
2014-06-20 18:14:02,644: INFO: Message received on topic revspace/sensors/6-t0 with QoS 0 and payload 20 
2014-06-20 18:14:02,644: INFO: Updating: /tmp/revspace/sensors/6-t0.rrd with value: 20.0
2014-06-20 18:14:02,645: INFO: Message received on topic revspace/sensors/20-t0 with QoS 0 and payload 23 
2014-06-20 18:14:02,645: INFO: Updating: /tmp/revspace/sensors/20-t0.rrd with value: 23.0
2014-06-20 18:14:02,645: INFO: Message received on topic revspace/sensors/2-t2 with QoS 0 and payload 22 
2014-06-20 18:14:02,645: INFO: Updating: /tmp/revspace/sensors/2-t2.rrd with value: 22.75

mqtt

client_id

The client identifier is a 23 byte string that identifies an MQ Telemetry Transport client. Each identifier must be unique to only one connected client at a time. The default is: MQTT2RRD Client.

username

The username to use when connecting to the server. If not configured, will attempt to connect anonymously. The default is anonymous.

password

The password to use when connecting to the server. If not configured, will attempt to connect anonymously. The default is anonymous.

hostname

The hostname or IP address of the MQTT server. The default is localhost.

port

The port to connect to on the MQTT server, the default is 1883.

keepalive

The number of seconds in between sending a keepalive packet to the server. The default is 60 seconds.

subscriptions

A comma separated list of topics to subscribe to, can use # has a wildcard, the default is to subscribe to all topics.

Topic Specific

Topic specific configuration can be made by creating a section with the name of a topic. The following options are supported:

step

The step time in seconds to use when creating the RRD graph. The default is 60.

friendly_name

A friendly name to give the topic, this is stored in the info file for the RRD, and may be used for reference later.

archives

A comma seperated list of RRA configuration strings to be used when creating the graph. The default is: RRA:AVERAGE:0.5:2:30,RRA:AVERAGE:0.5:5:288,RRA:AVERAGE:0.5:30:336,RRA:AVERAGE:0.5:60:1488,RRA:AVERAGE:0.5:720:744,RRA:AVERAGE:0.5:1440:265

Configration Example

[daemon]
data_dir = /tmp
pid_file = /tmp/mqtt2rrd.pid
user = mqtt
group = mqtt

[logging]
log_file = /tmp/mqtt2rrd.log
log_level = 10


[mqtt]
client_id = "MQTT_to_RRD_TEST"
hostname = test.mosquitto.org
port= 1883
username="bob"
password="t0ps3cr3t!"
keepalive=60
subscriptions=/mytopics/#,/other/important/topic

[/other/important/topic]
friendly_name = "My Important Topic"
step=30
archives=RRA:AVERAGE:0.5:1:120,RRA:AVERAGE:0.5:5:288