Week 2: IoT Infrastructure

Building an IoT Database as a Team

Diagram representing all three aspects of IoT

After the first week’s introduction to the data acquisition part of IoT product development, it was then time to explore the other half of IoT – the data storage and the data visualisation. Previously, both of these were ‘taken care of’ by the Blynk app, but this week’s task was to create our own IoT infrastructure as a team.

After forming our groups we first went away and did some individual initial research as we are all at different levels of knowledge about applications for databases and visualisations. We then came together to discuss what we found. During this initial research about IoT databases, I learnt quite a few new terminologies to do with the infrastructure:

Client = the device being used to collect data. In our case this is our temperature/humidity sensor device.

Broker = Receives messages from the client, then sends the data to clients subscribed to the server.

Publish = to send data to the server/broker.

Subscribe = to send data to the client/device.

To-do list in Teams

ooooo

As a group we used Teams to communicate tasks – as seen on the right – and send each other artifacts and resources to help each other understand the process of the infrastructure creation.

oooooo

The video below was also great help for understanding what is involved and how many applications may actually be required to create a functioning IoT infrastructure.

ooooooo

oooo

As a group we decided a similar approach to the one described in the video was the best approach, as seen below. For our infrastructure, rather than using a Raspberry Pi for our server, we used a remote server on Vultr as a “pi in the sky”.

We then allocated each person in the group to install one application onto the remote server.

sensor
Wiring for DHT22 with NodeMCU

We decided to begin with a simple temperature/humidity sensor to help us focus on the creation of the infrastructure.

We selected a DHT22 sensor with a NodeMCU board, as the majority of the group members are using this board for their personal projects. Last week, the ‘beginners’ including myself started with Arduino IDE to program the devices, however as a group we moved over to Platformio on Visual Studio. Malan and I were shown by Tim how to use Platformio and I now better understand the use of libraries and how to create multiple sections in the same project file. This will make it easier to start with some simple code then build upon it in later steps. Next week when I use Platformio on my own I will need to further learn how to use it and be more proficient with my programming learning.

After setting up our device we noticed that the data was in-congruent and continuously stopping. We asked our facilitator for guidance and learnt that pin D4 is not the best to use as GPIO2 is connected to one of the board’s built in LEDs. After looking up a pin layout diagram for the NodeMCU, we changed over D6 (GPIO12) instead, and this resolved our issue. I now better understand the pins of the NodeMCU and why I used D1 and D2 for my prototype as D1 and D2 are equivalent to SCL and SDA respectively. Later on I should still learn about all the other pin meanings and what they mean.

NodeMCU pinout
mosquitto broker

We decided to install Mosquitto as our broker as it is the most popular broker to use along side the MQTT protocol. We found a simple example from github to add to our sensor code. After having some connection issues, we reached out to our facilitator and were advised to switch over to his network rather than the UTS network as the uni has some firewall blocks on some ports. We also switched over to port 5269 rather than the default 1883, as seen in the code snippet below.

void setup() {
pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output
  Serial.begin(115200);
  dht.begin();
  setup_wifi();
  client.setServer(mqtt_server, 5269);
  client.setCallback(callback);
}

On the second day of this week we ran into more issues, however. Our Mosquitto broker was not connecting and after some research we learnt that we also needed to add a username and password due to authorisation issues, as seen in the highlighted sections below.

const char* ssid = secret_ssid;
const char* password = secret_password;
const char* mqtt_server = secret_mqtt_passwd;
const char* mqtt_username = secret_mqtt_username;
const char* mqtt_passwd = secret_mqtt_passwd;
const char* sensor_Name = "11.11.403 Temp/Hum";
void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str(), mqtt_username,mqtt_passwd)) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish("outTopic", "hello world");
      // ... and resubscribe
      client.subscribe("inTopic");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }

Another technical skill I learnt during this week was how to use the SSH using PuTTY to install the applications onto our sever. I have previously done a subject in which I used PuTTY to get data from a database using SQL queries but I have never made a database nor used PuTTY to install applications.

NODE-RED

We decided to use Node-RED as the link between the MQTT protocol and our database in Influxdb as it is very visual, allowing us to connect various nodes together to format our data. After installing Node-RED, we first created a flow that could inject some arbitrary values into the database to test our column names and that numbers were being entered correctly. Once we were happy with that we connected our ‘MQTT in’ node to the ‘Influxdb out’ node. This still wasn’t correct because the output was all three of our values in one line, when we needed one packet with three lines – one for each value. After some more research we discovered we should use a function node to correct the formatting. Using Node.js, we set up a ‘split’ that would separate the payload into an array based on the occurrence of a new line (\n). This was also where we could name our database columns – Sensor_Name, Humidity and temp. Node-RED was also my first introduction to the Node.js language and next week I should learn the basics of this language in order to successfully add all the team member’s device outputs into Node-RED.

influxdb

We installed Influxdb using these instructions. After a discussion with all summer studio students and our facilitator, I learnt that there are different types of databases – SQL and no SQL. Influxdb is a no SQL database, meaning there are no relationships between the data and it does not require a primary key. I also learnt that no SQL databases are best for large databases where there is less likely to be relationships and data repeats are not an issue.

grafana

We decided to use Grafana as our visualisation tool as it is recommended for Influxdb and offers a wide variety of visualisation options. After installing Grafana, we entered our data source by inputting our influxdb database name, username and password. We discovered that Grafana is not as intuitive as we first thought. We had to use PuTTY to check the column headings in the database, as after making many previous alterations it started to become confusing. After some more research we worked out how to enter queries into Grafana in the correct format, as seen below. Next week we will have to practice these queries more in order to create the correct graphs/visualisations for everyone’s individual data being collected.

SELECT Temperature FROM "TEMP".."TEMP" WHERE ("Sensor_Name" = '11.11.403 Temp/Hum')
SELECT Humidity FROM "TEMP".."TEMP" WHERE ("Sensor_Name" = '11.11.403 Temp/Hum')
Grafana dashboard displaying humidity and temperature graphs

Grafana was reasonably simple to use and has a good range of visualisation tools and customisation, however I’m not sure if I want to use Grafana for my individual project because I want to use some kind of of an app visualisation so it can be easier for an individual to quickly access on their person. Instead, Grafana could be the HCPs dashboard, while individuals have their own app. Because I would like to focus on the hardware aspect of IoT in the following weeks, I will continue with Grafana in the next sprint. From here I will need to learn how Grafana sends alerts as this will be a useful addition to my heart rate monitor.

S U M M A R Y

AccomplishedCreated an IoT infrastructure for a basic device as a team
Learnt– Structure of IoT back-end
– Platformio
– How to use the applications Node-RED, Grafana
– How to install applications using PuTTY
What next– Move over to Platformio for my project
– Set up everyone’s device onto our database
– Need to look into different visualisation options

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s