Modernizing My Temperature and Humidity Sensor

My basement temperature and humidity sensor has been up and running for 2.5 years now with hardly a hiccup at all. But, it is time for an upgrade.

I’ve made some tweaks to the page over the years, but I ‘ve grown less and less happy with the matplotlib implementation. You can see on the X axes that they do not actually reflect the timestamp, but rather the record number. I’ve gone after this so many times, but no luck. Also, I’d like to be able to crunch some of those numbers.

In my research of how to modernize this, I discovered Grafana, which reads databases and makes pretty graphs out of the data. This does not natively read sqlite3 databases, but it can be made to. It isn’t easy or nice, however, due to how sqlite3 works with timestamps. Basically, it doesn’t. It deals with string-formatted timestamps, but Grafana, for all its power, can’t be arsed to figure out what I have seen in many, many json files:
YYYY-MM-DD hh:mm:ss
It relies on either straight UNIX time (I guess for speed, as only a machine can figure out what that means), or something called RF3339 format, which looks like this:
2008-04-02T20:00:00Z
Yeah, that is an actual thing. Somehow, you can get time zones involved with that mess, too, but, seriously, this time (no pun) it is me who couldn’t be arsed to figure it out.

What it boils down to is that I have 2.5 years of data trapped in a database that I will not be able to use in Grafana, unless I figure out one of a couple of not-so-easy routes forward. At first, I was thinking of migrating the data to MySQL (or MariaDB for the Mods), but then I figured that if I was going to start messing with actual SQL databases on small Linux devices, I might as well look at something useful: PostgreSQL. So, I set about installing PostgreSQL and Grafana on the target Pi, which is vintage 3B+. Grafana works, but it is not great. And, it has something of a learning curve to it. As for Postgres… Yeah, that was pretty much a non-starter. I’d still like to learn it, but not while I am working on this issue.

While learning how to use Grafana, I was introduced to another database, InfluxDB, which has great Grafana support, and is a time series database, which means something. So, I installed InfluxDB on the Pi and set out to learn how to make this work. Turns out, it is very different from SQL databases. It uses something called Series instead of Tables. This database write section is copied directly out of my test Python script:

#InfluxDB Connection Details
influxHost = 'localhost'
influxPort = '8086'
influxUser = 'grafana'
influxPasswd = 'none_of_your_business'
influxdbName = 'home'

influx_metric = [{
    'measurement': 'TemperatureSensor',
    'time': datetime.datetime.utcnow(),
    'fields': {
        'temperature': temperature,
        'humidity': humidity,
        'location': "basement"
    }
}]

#Saving data to InfluxDB
try:
    db = influxdb.InfluxDBClient(influxHost, influxPort, influxUser, influxPasswd, influxdbName)
    db.write_points(influx_metric)
finally:
    db.close()

If you are familiar with writing to SQL databases in Python, this is fairly different. But, it is pretty easy to figure out, too. In fact, I ran it a couple of time (that isn’t the whole script…the sensor set up stuff is all missing) for to see what was getting in the database. And hooking it up to Grafana was totes easy.

You can see that I am not at all unhappy with the layout of the older webpage. I’m not sure what is happening yet with the chart, however. There is no line connecting the points. It’ll get sorted eventually.

As I was working on this, it became clear to me that running the data collection via the sensor and Python, the database server, and the Grafana server all on the lower powered Pi 3B+ was not the best way forward. Grafana can reach out to databases on remote systems, so I installed Grafana on my damn-near idle Pi 4B (4GB), sitting on the desk in my office, and pointed it at the database running on the Pi 3B+, sitting in the network rack in the basement. To my amazement, it connected with no issues.

The more I played around with this, the more I started thinking about having multiple sensors around the house, all reporting in to the one Grafana dashboard. But then it occurred to me that that would be multiple database servers running and I might be able to get away with one…running on the same box as the Grafana server. I was looking at the fancy ways that InfluxDB might be able to get this accomplished, and then I remembered an often overlooked feature baked right into Raspberry Pi: Remote GPIO.

I enabled this via raspi-config on the 3B+ and installed the needed software. But before I dove in too deep, I grabbed the Pi4B (1GB) that has been sitting powered off behind me all summer. I mainly use this Pi for time-lapse photography. The last experiment I did on it was with an ultrasonic sensor, and it is still wired up to the breadboard. I really wanted to see about getting that to work from remote, but I followed the docs and made an LED blink…from across the LAN. I set such a massive cock-a-hoop that Jenny told Calin that I had obviously finally cracked.

#!/usr/bin/python3

from gpiozero import LED
from gpiozero.pins.pigpio import PiGPIOFactory
from time import sleep


remote_factory = PiGPIOFactory(host='10.25.68.154')
led = LED("BOARD13", pin_factory=remote_factory)

while True:
    led.value = 1
    led.on()
    sleep(1)
    led.off()
    sleep(2)

Just that easy. Tomorrow, I am going to see about getting readings from the HC-SR04 ultrasonic sensor. I don’t think this will be too difficult. The script for getting this sensor to work locally is not much more complex than that above. My worry is that first import statement, though. We’ll see.

That is where I am going to leave this for tonight. Once I get the pieces in place and working, I will then turn my attention to figuring out how to import the 2.5 years of data from the 3B+. There’s got to be a way to do this, but I simply haven’t found it yet.

I’d like to thank the authors of the main tutorials I have been following:
* Simon Hearne
* Mat at notenoughtech.com, who has obviously been down this same road many times previously
* And countless other bloggers and tinkerers.

Thank you all.