Final Thoughts

This summer studio has been extremely valuable for my development as a biomedical engineer, and has taught me a great amount from prototyping, to technical skills of programming and 3D printing design, to further developed my communication skills in team situations as well as presenting my project during the final summer studio showcase. I have also been able to cover all the SLOs of the subject and will be able to utilise the skills and knowledge gained from this studio in my future studies and career.



Engage with stakeholders to identify a problem or scope a defined problem
During week 1 I learnt a lot about the stakeholders of heart rate monitoring devices and the types of solutions currently on the market. Later, I began to understand the more technical aspects such as the use of Bluetooth Low Energy and the processing of biosignals. I was also able to identify a potential solution to make a device which is smaller, personal, more accessible and wireless – which are all common themes for the future of medical devices.

Apply design and systems thinking to respond to a defined or newly identified problem.
Also during week 1 I learnt about Proof of Concept prototyping and hacking. I learnt a lot about the design process of IoT devices and that the process is more cyclical than a straight line, as with most design processes. Unfortunately I was unable to test the validity of my device without access to either a pulse oximeter or ECG machine.

– Apply technical skills to develop, model and/or evaluate designs.
The most learning for me was surrounding technical skills. Before completing this studio I had extremely limited programming understanding and now I feel confident enough to work with either C++, node.js or Python in future projects. I now also have an understanding of microcontroller pins and how/when to use them after prototyping with both an Arduino Uno and NodeMCU board. Week 2 and 3 taught me about IoT infrastructure, from broker to protocol to database to visualisations. During week 4 I learnt more about reading component or IC datasheets, basic analogue concepts including the need of a DC reference, and using the DSO to probe and test different points of PCBs. During week 5 I learnt how to use Autodesk Fusion 360, how to design viable objects to 3D print – including filleting all edges to increase strength and using ‘draft’ to create 45 degree angles inside extruded text to make it clearer. Week 6 was back to programming again where I learnt how to use the Python library pandas to create dataframes and I further learnt about functions and other aspects more specific to Python such as global variables.

Demonstrate effective collaboration and communication skills.
Our collaboration in week 2 where we set up our IoT infrastructure as a team helped to reaffirm my communication while using Teams, when making sure to divide group work fairly and when coming together to explain to each other what we had achieved for that sprint. During the final summer studio showcase, I had the opportunity to practice my communication skills with various people of different levels and understanding of the work done in our IoT studio. It was very valuable for me to practice presenting my work with no preparation, however in the future I would be more organised in setting up a smoother live demonstration.

Conduct critical self, peer, and group review and performance evaluation.
In terms of self evaluation, this project portfolio was a good medium to use as it allowed me to make weekly review of what I had achieved and what challenges I was to face next. I was also able to receive performance evaluation from my peers during group work, and throughout the rest of studio – especially when I was completing new tasks in programming or in understanding IoT infrastructure. I was also able to give my peers feedback when they were completing the tasks which I was more familiar with, such as soldering our components to a perforated board or understanding circuit block diagrams.
From our facilitator and other studio facilitators, as well as friends from the Electronics Studio, I was also able to gain valuable feedback and insights into the best approaches to take, how to improve upon my own ideas and what might be next for me if I were to conduct another project similar to this in the future.

traversing technologies

During the summer studio project, we explored many different levels of technology. The highest level of these was global systems, which includes internet, and the lowest level being the quantum world.

Quite a few of these levels were explored over the past 6 weeks, with an emphasis on the higher levels as we were working with global internet for databases and sending our data to “the cloud” with Vultr. The timeline below shows a summarised experience with traversing these levels of technology.

The most evident example of getting ‘suck’ in searching for lower level solutions was when trying to find a solution for the battery problem. Going down to level 5 while reading the datasheet for the battery charger component did not give any solution as there was no information on how to “re-program” a resistor to change the charging rate. Instead of then looking to a higher level solution, we searched even lower at level 4 for a solution involving transistors to build a current limiter circuit. As I have extremely limited understanding of transistors this direction would have taken me a long time to understand and build a solution. At this point our facilitator recognised that I was beginning to be stuck in a “rabbit hole” – or in other words, becoming stuck on lower and lower level problems without solving the initial issue at hand. By then moving up levels, we were able to test and recognise two higher level solutions: one, that a simpler solution is to provide a DC charger which already incorporates a current limiter, and two, that the Lipo battery which I was using already had a current limiting circuit and there was no need for any changes at all. If this rabbit hole had gone unnoticed, I would have wasted even more time unnecessarily trying to learn how to build a circuit with transistors. This taught me that it is important to explore other possible solutions in different technology levels as an engineer, as a fitting solution may already be present.

Another example from this summer studio would be any programming challenge I faced. As someone with nearly zero programming experience prior to the summer studio, I didn’t know how to go about finding solutions to my problems. I would often get stuck in “rabbit holes” where I would not know what to search for and accidentally search too low level, rather than looking for higher level solutions better suited for beginners – such as libraries. For example, when solving the challenge of calculating live BPM, I was searching for solutions using C++ including things like how to create thresholds, or how to measure time in between two events. I also got caught up in reading about complex algorithms and s-transformations used by legitimate ECG machines to calculate various values including heart rate. From the start I should have looked for discrete analysis examples – especially in Python as I’ve learnt that the language is often used at a higher level than C++. Once I did search for this solution, I found what I was looking for almost instantaneously and finally started to make process on my project again.

Week 6: Jupyter Notebook – Calculating Live BPM

The next challenge for me was to find a way to calculate live BPM from my new ECG data, and then publish this to be visualised elsewhere. After feeling lost while researching for methods in C++, another summer studio facilitator Trang Nguyen recommended for me to instead look at using Python. Although I was unable to find her work, I was able to find a couple of examples using Python to calculate BPM from ECG data.

The example below used a csv to analyse a discrete amount of ECG data.

As I was still very unfamiliar with how to use Python, the process of understanding the example code and how to incorporate live data was quite challenging for me and I needed to ask for frequent additional help. Without understanding the fundamentals, it is hard to know where to even begin, but luckily I was able to access help and advice from both my studio team and our facilitator.

To still utilise this example for its peak detection and BPM calculation, I learnt that I should used a pandas data frame to store a set amount of my own ECG data to then be analysed. To do this, following some advice, I created an initial empty dataframe called “ECGDataSet”, then within the on_message function, created a new dataframe “newData” to be concatenated with the initial “ECGDataSet”. Also, I learnt that ECGDataSet needed to be a global variable as before it wasn’t concatenating and only recognising the initial empty data frame. I could tell this was occurring by adding the output “ECGDataSet.tail()” or “ECGDataSet.head()”. The data frame was also limited to 2500 inputs by adding “ECGDataSet.tail(2500)” to the concatenate line. To make the BPM calculation update automatically, our facilitator helped me set up a loop with a sleep time of 0.1s, meaning the plot will update every 0.1s and recalculate the average BPM.

# !pip install "notebook>=5.3" "ipywidgets>=7.2"
# !pip install paho-mqtt
# Get the pandas dataset ready for incoming data
import pandas as pd
ECGDataSet = pd.DataFrame([0], columns=['ECG'])
import paho.mqtt.client as mqtt

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))
    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe("ECG")
    
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    payloadStr = str(msg.payload).split("'")[1]
    payloadInt = float(payloadStr)
    #print(payloadInt)
    newData = pd.DataFrame([payloadInt], columns=['ECG'])
    global ECGDataSet
    ECGDataSet = pd.concat([ECGDataSet.tail(2500), newData], ignore_index=True)

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.username_pw_set("username", password="password")
client.connect("139.180.169.188", 5269, 60)
client.loop_start()
import time
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import math
from IPython import display

# Setup a loop with the ability to interrupt
try:
    while True:
        # begin analyzing the dataset for peaks
        dataset = pd.DataFrame()
        dataset['hart'] = ECGDataSet['ECG']
        #Calculate moving average with 0.75s in both directions, then append do dataset
        hrw = 0.75 #One-sided window size, as proportion of the sampling frequency
        fs = 100 #The example dataset was recorded at 100Hz
        mov_avg = dataset['hart'].rolling(int(hrw*fs)).mean() #Calculate moving average
        #Impute where moving average function returns NaN, which is the beginning of the signal where x hrw
        avg_hr = (np.mean(dataset.hart))
        mov_avg = [avg_hr if math.isnan(x) else x for x in mov_avg]
        mov_avg = [x*1.2 for x in mov_avg] #For now we raise the average by 20% to prevent the secondary heart contraction from interfering, in part 2 we will do this dynamically
        dataset['hart_rollingmean'] = mov_avg #Append the moving average to the dataframe
        #Mark regions of interest
        window = []
        peaklist = []
        listpos = 0 #We use a counter to move over the different data columns
        for datapoint in dataset.hart:
            rollingmean = dataset.hart_rollingmean[listpos] #Get local mean
            if (datapoint < rollingmean) and (len(window) < 1): #If no detectable R-complex activity -> do nothing
                listpos += 1
            elif (datapoint > rollingmean): #If signal comes above local mean, mark ROI
                window.append(datapoint)
                listpos += 1
            else: #If signal drops below local mean -> determine highest point
                maximum = max(window)
                beatposition = listpos - len(window) + (window.index(max(window))) #Notate the position of the point on the X-axis
                peaklist.append(beatposition) #Add detected peak to list
                window = [] #Clear marked ROI
                listpos += 1
        ybeat = [dataset.hart[x] for x in peaklist] #Get the y-value of all peaks for plotting purposes
        plt.title("Detected peaks in signal")
        plt.xlim(0,2500)
        plt.plot(dataset.hart, alpha=0.5, color='blue') #Plot semi-transparent HR
        plt.plot(mov_avg, color ='green') #Plot moving average
        plt.scatter(peaklist, ybeat, color='red') #Plot detected peaks
        display.clear_output(wait=True)
        plt.show()
        
        
        # Calculate the average bpm
        RR_list = []
        cnt = 0
        while (cnt < (len(peaklist)-1)):
            RR_interval = (peaklist[cnt+1] - peaklist[cnt]) #Calculate distance between beats in # of samples
            ms_dist = ((RR_interval / fs) * 1000.0) #Convert sample distances to ms distances
            RR_list.append(ms_dist) #Append to list
            cnt += 1
        bpm = 60000 / np.mean(RR_list) #60000 ms (1 minute) / average R-R interval of signal
        print ("Average Heart Beat is: %.01f" %bpm) #Round off to 1 decimal and print
        
        client.publish("BPM", (bpm))

        # sleep
        time.sleep(0.1)
except KeyboardInterrupt:
    pass
Output plot showing detected peaks and average BPM value
New BPM data stored in Influxdb

To then publish my BPM value being calculated every 0.1 seconds, it only required the simple addition of “client.publish(“BPM”, (bpm))”, with the topic being BPM and the message bpm. The Node-RED function node phrasing didn’t need any alterations and I could begin sending BPM data into our Influx database to then be visualised on Grafana once again.

Grafana dashboard with new BPM data
laser cutting the lid
Autodesk Fusion 630 sketch of lid

The final step to complete by project was to laser cut a clear acrylic lid for my enclosure. Although we went able to operate the laser cutter ourselves as it requires an induction, we were able to watch our facilitator set up each type of operation for the machine. Firstly, the outer 2mm perimeter was ‘vector rastered’ down by 1mm to create an indented lip to fit perfectly into the enclosure. The text was then rastered as well, however the text first needed to be flipped as the rastering was being executed onto the underneath side of the lid. The final stage was to ‘vector cut’ the outer perimeter and the hole for the switch. Upon putting the whole device and enclosure together, I slightly mis-measured where the hole needed to be as it was a bit off center and quite hard to turn the switch on and off without either long nails or a pen. If I were to make an enclosure again I would definitely do a test laser cut on plywood before jumping straight into the acrylic to correct any errors.

S U M M A R Y

Accomplished

– Calculated BPM using Python and visualised this data in Grafana
– Produced a clear acrylic lid for enclosure


Learnt

– Python: data frames, functions, global variables and loops
– Watched how to operate laser cutter machine


What next

– Continue to learn programming in C++ and Python
– Reflect on everything I have learnt in the past 6 weeks
– Begin to think about the next IoT project I can start to work on


Week 5: Mechanical Design

This week’s task was to design an enclosure for our device using Autodesk Fusion 360 to be 3D printed, as well as to ‘miniturise’ our breadboarded prototype onto perfboard to produce a tangible wireless device as a finalised prototype.

planning

At the start of this sprint, our facilitator ran through an introductory demonstration of Autodesk Fusion 360 while we followed along. Because I’ve previously used SolidWorks, I felt that I could pick up on Autodesk quite quickly, although I had never actually printed anything designed in SolidWorks so I had to learn and keep into consideration the limitations of 3D printing – such as tolerances and minimum thicknesses.

Next I began to plan out my perfboard layout ready to begin soldering. As seen in the sketches below, the easiest method to go about doing this was to draw each component and the potential wiring required. To help optimise the space on the board, I decided to include two rails – one for 3.3V and one for ground.

Initial Layout Sketch
Optimising space planning

Final Wiring Diagram
autodesk fusion 360

After planning out my perfboard layout, I began to draft my Autodesk design. To do this I began by measuring my components with digital callipers in order to accurately draw each component to then insert into the one drawing for the board its self. Following Tim’s advice, instead of drawing each component from scratch, I utilised GrabCAD to find common components already made. In my case, I could only find the mini ESP8266. Fortunately, most of the studio students were using the same components so we were able to share our drawings between us to lessen the amount of work.

soldering the perfboard

To begin soldering my components onto the perfboard, I began to cut various pieces of breadboard wire into appropriate lengths. Initially I found the process confusing as I had never done this type of soldering before. After asking our facilitator for guidance, I learnt that the best method is to solder the wires perpendicular to the header pins. Expanding on this, I decided to try to only use headers for the mini ESP8266, as it was the only component which requires them, and to only use the breadboard wires to make connections for the other three components. During soldering, I learnt very quickly that small sections of wire will move as soon as the solder is melted around them. To overcome this I found that applying solder first, then using tweezers to position the wires was a much easier method to use.

Soldered Perfboard Top View
Soldered Perfboard Bottom View
the battery charger problem

During this sprint the main issue raised was that the IP5306 lithium battery charger board a few of us were going to use was not suitable for smaller batteries such as the one I was using. We could test this as when supplying the board with around 1-2 amps using the lab power supply, it would not limit the current and charge the lithium battery too fast. During this week we took many approaches to try and solve this issue as many of us did not want to change to a larger battery or a different charging board.

As a side tangent related to the battery, there was also the issue of supplying the charging board a power source. Originally we discussed utilising the micro USB port on the mini ESP8266, however our facilitator recommended an easier approach that would not require any diodes. Instead we added a DC power supply jack to be positioned on the outside of our enclosures to supply 5V to the charger board.

The first direction our facilitator took to solve the charger issue was to look at the datasheet as I learnt that usually lithium battery chargers are able to be “re-programmed” to different charge rates by changing a set resistor. After attempting to change what we thought was the correct resistor multiple times, and seeing no change to the current during charging, our facilitator did some more research and found that that particular board cannot be reprogrammed and is only designed for larger batteries.

After making this discovery, another idea which our facilitator suggested was to design a current regulator using transistors, thereby digging deeper into the issue and tackling it at a lower level.

Block diagram of potential method to solve charger issue using transistors

While I was trying to understand transistor circuits, our facilitator brought up another suggestion. After a closer inspection of my battery, he saw that it might potentially have a limiter circuit already attached to it. Following this, we conducted some more testing of my battery alone to see if this was true.

By connecting a current probe at both the output of the charger board and the battery output, and viewing the current on the DSO, we were able to see how much current was passing through the circuit. We were able to force max discharge of the battery using large resistors, and by reading the LP553436 datasheet I found that the discharge limit was 1260mA, meaning that the board’s output was staying within the correct range.

As for charging, my battery’s charge limit was 630mA and by supplying the board with a few amps we could see on the DSO that it remained under 630mA. Therefore this meant that my battery had its own built in current limiting circuit and there was no need to make any additional changes.

In retrospect after this fiasco, it is sometimes better to not go to lower level solutions as it is easy to get stuck and not make any progress when trying to solve an issue. Especially for someone like me with limited experience, I found it very challenging to tackle lower level strategies such as designing circuits using transistors. This sprint has definitely taught me to first take higher level approaches and I will keep this in mind for the last step of my project – calculating the BPM value from my ECG data.

building the enclosure

After double checking my measurements of my perfboard, I began to draw up my enclosure on Autodesk. Following the last road-block, this process was very streamlined. I also learnt many more skills in Autodesk including using projection and adding extruded text to surfaces. After printing my enclosure, the final steps will be to produce a clear acrylic lid using the laser cutter and to added double sided foam tape to the underside of my board to hold everything in place.

S U M M A R Y

accomplished

– miniturised prototype on perfboard
– 3D printed enclosure


learnt

– Learnt basics on Autodesk Fusion 360
– New soldering technique
– Sometimes it is better to take a higher level approach to solve problems
– How to use the 3D printer


what next

– Calculate BPM value from ECG data and publish to MQTT
– Produce lid for enclosure using the laser cutter


Week 4: Deep Dive

Week 4 was everyone’s chance to chose their own topic to deep dive for this week’s sprint. Originally I wanted to deep dive into PCB design and potentially learn how to use PCB design software such as Altium. After some discussion with more experienced electrical/electronic students, I realised that it was not viable to pursue this direction within the 3 days of the sprint as the learning curve was too steep to jump into. Upon this reflection I decided to still pursue to make my device smaller than it currently is, but instead to use “prototyping board”, also called perfboard or stripboard, and smaller versions of the components I was currently using – such as the smaller version of the ESP8266.

bitalino

Our facilitator acquired some more sensors for me to trial and compare with my pulse oximeter sensor. These sensors were from BITalino. BITalino is a company based in Portugal which sells various biosignal components and modulaer kits, along with its own visualisation software OpenSignals. The kit which was lent to me was the BITalino (r)evolution Board Kit BT, with a retail price of €149 – approximately 240AUD.

BITalino (r)evolution board

To get started with the ECG module, I connected the module to the main board at A2, using the wires provided. The ECG leads were then connected to the other side of the module. The leads available to me were either a single-lead with reference (3 electrodes) or single-lead with virtual reference (2 electrodes).

BITalino (evolution main board
ECG set up
opensignals

To start visualising my ECG signal using OpenSignals, I first had to connect the device via bluetooth on the laptop with the pin 1234. I allowed OpenSignals to search for device then enabled it, then channel 2 was selected for ECG, with a sampling rate of 1000Hz. I found that OpenSignals was quite difficult to use and understand.

OpenSignals device configuration
“Off-the-person” placement for 2 electrodes from BITalino

For my first attempt, I tried the single lead attached to the wrists with pre-gelled electrode pads, following the image to the right as found on one of BITalino’s datasheets. The images being black and white was not very helpful as I had to guess which electrode was positive and negative, as none followed the colouration standards of ECG leads as seen below.

When I ran OpenSignals, this signal was very noisy and didn’t resemble an ECG signal and had no visible QRS complex. I then tried the 3 electrode lead, following the example picture below.

“On-the-person” placement for 3 electrodes from BITalino

However this resulted in zero signal on the software.

Single-lead placement

I then tried a different method with the 2 electrodes, with them placed across the heart, as seen in the illustration on the right. At this point I had worked out that the red lead was definitely positive and the black lead negative.

This looked a lot better, as seen in the video below, with a visible QRS complex, but would change dramatically if I moved, lifted the main device, walked around or had someone standing next to me.

I wanted to continue with the 2 electrode leads as following the Einthoven Triangle would make it harder to have a small wireless and wearable heart rate monitor, as the 3rd electrode would have to be placed quite far away, such as on the left leg.

Viewing the signal
Pin layout on ECG module

Electrical engineering expert Peter McLean proposed that this was a skin conductance issue. Out of curiosity we wanted to know what the signal looked like at the output of the ECG module at A2, without the Micro Controller Unit. If the MCU was required I could potentially snap off that module and incorporate it into the rest of my prototype. If not, I could simply go from the ECG module to my nodeMCU so I could utilise my previously set up database, broker and visualisations using WiFi rather than Bluetooth.

To view the signal, Peter used the DSO and probes to pinpoint the analogue signal coming out of A2, and the signal produced was a clear ECG signal with very clear R wave peak. This was very exciting for me as it meant that I could use the ECG module alone without the need of the rest of the BITalino board.

Output from ECG module pin A2

If each box represents 500ms, then within 5 seconds my heart beat 7 times. Multiplying this number by 12 I could calculate that my BPM at that moment was 84. This sounded very promising as a reliable way to measure my BPM using an ECG signal.

Viewing each of the other pins we found that the REF pin was approximately halfway between 0 and 3V, therefore half of the voltage supply.

the new plan
Block Diagram for Breadboarding, drawn by our facilitator

Our facilitator guided me on what approach to take next. First, I needed an analogue converter to convert the analogue ECG signal into a digital reading that can then be inputted to SCL (D1) and SDA (D2) on my NodeMCU. I also learnt that analogue signals need to have a reference value as well. Our facilitator explained to me that this was so that the ECG signal would be offset above 0V. To replicate this, I was advised to create a voltage divider with two 100Kohm resisters to supply the REF pin.

Breadboarding for new direction
BITalino ECG Module
ADS1115 analogue to digital converter
Problems and confusion

After breadboarding the NodeMCU with the ADS1115 and ECG module, following the diagram above, the first step was to test the ADS1115 with a simple example code after finding its library. With some assistance from our facilitator we found an appropriate library and example for ‘single’ rather than ‘differential’. This example printed the voltage output of each analogue pin in the serial monitor.

To start off with I just connected the ADS1115 and supplied the analogue pins with 3V3, one at a time. After noticing that only A3 would change and when it did change, it changed the readings on all the pins. I then connected the ECG module, again following the diagram, however on the serial monitor the output was a constant number around 1200mV, again only on A3.

After sitting in confusion for a while my electronics engineering friend gave me the advice to supply each pin with an analogue signal – sine wave – and slowly decrease the amplitude to see if the component has a minimum limit and therefore it was not correctly reading the small voltages from the ECG. My set up for this test can be seen in the slideshow below. I began with 3.3Vpp and decreased it slowly, however I started to see another issue. The outputs on the serial monitor were incorrect. For example a 3000mV supply was reading as -1200mV. After asking for more help, my friend noticed that the issue was that the value being recorded wasn’t the peak of the waveform but another random value from another part of the wave. He said this was a sampling rate issue. When we also lowered the supply to 1Vpp the serial monitor read 94mV, meaning small voltages are not the issue and it was most likely the sampling rate. He also explained to me that when I connected the analogue pins to the 3V3 supply of the NodeMCU, the outputs were correct as it was a constant DC source rather than a changing analogue signal.

resolving the issues

By checking the ADS1115 datasheet, we could see that the maximum sample rate for the component is 860. If the QRS complex has a duration of 80 to 120ms, this means that the ADS1115 could potentially sample the R wave 103 times, meaning the sample rate will not be an issue

Next was to solve the issue of the incorrect outputs of the ADS1115. As I still have very limited understanding of programming, reading the library for the component was not very much help to me. Our facilitator guided me through how to correct all the issues I was having and explained to me that because this library and example was designed for an Arduino board, it was slower than the NodeMCU and was therefore causing the problems with the pin outputs.

Firstly, I had missed a step in my wiring. We could tell there was a problem because when viewing the ALERT pin on the DSO, it wasn’t sampling correctly and the serial monitor was printing “Failed to wait for AlertReadyPin”. The code had a DataReady definition for pin 2 on an Arduino – which does not exist on the NodeMCU. We changed it to pin 14 and connected the ALERT pin to D5. Our facilitator explained that this was necessary to notify the board when the conversion was ready.

There were a few more changes that needed to be made. Single shot sampling was changed to continuous and I also learnt how to use an interrupt function and what it was. I also learnt that it is possible to have multiple of the same component and use different ‘channels’ to call them. For the ADS1115 it had 4 possible channels. I also learnt how to go to the header of the code, in this case ADS1115.h using ‘go to definition’. The corrected code can be seen below with important lines highlighted.

Although our facilitator explained every part of the new code to me, I will still need to learn a lot more about programming as I still don’t fully understand the meanings of each function and how everything operates together.

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <Wire.h>
#include "ADS1115.h"

ADS1115 PurplePCB(ADS1115_DEFAULT_ADDRESS);

#define DataReady_PIN 14
#define DataReady_CHANGE FALLING

volatile bool ProcessDataRequest = false;

void DataReady_Callback(void);
void GetProcessData(void);

//wifi and mqtt passwords
const char* ssid = "ssid";
const char* password = "password";
const char* mqtt_server = "server";
const char* mqtt_username = "username";
const char* mqtt_password = "password";
const char* sensor_Name = "ECG";

WiFiClient espClient;
PubSubClient client(espClient);
unsigned long lastMsg = 0;
char msg[50];
int value = 0;

void setup_wifi() {

  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();

  // Switch on the LED if an 1 was received as first character
  if ((char)payload[0] == '1') {
    digitalWrite(BUILTIN_LED, LOW);   // Turn the LED on (Note that LOW is the voltage level
    // but actually the LED is on; this is because
    // it is active low on the ESP-01)
  } else {
    digitalWrite(BUILTIN_LED, HIGH);  // Turn the LED off by making the voltage HIGH
  }

}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String("michaela");
    // Attempt to connect
    if (client.connect(clientId.c_str(), mqtt_username, mqtt_password)) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish("outTopic", "hello world");
      // ... and resubscribe
      client.subscribe("inTopic");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
    pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output
    setup_wifi();
    client.setServer(mqtt_server, 5269);
    client.setCallback(callback);

    // Setup serial
    Serial.begin(115200);  // initialize serial communication

    // Setup DataReady pin and Interrupt Service Routine
    pinMode(DataReady_PIN, INPUT); // Hardware pulldown on purple PCB
    attachInterrupt(digitalPinToInterrupt(DataReady_PIN), DataReady_Callback,
                    DataReady_CHANGE);

    // Setup I2C Device
    Wire.begin();

    Serial.println("Testing device connections...");
    Serial.println(PurplePCB.testConnection() ? "ADS1115 connection successful"
                                         : "ADS1115 connection failed");

    PurplePCB.initialize();  // initialize ADS1115 16 bit A/D chip

    // We're going to do continuous sampling
    PurplePCB.setMode(ADS1115_MODE_CONTINUOUS);

    // Slow things down so that we can see that the "poll for conversion" code
    // works
    PurplePCB.setRate(ADS1115_RATE_32);

    // Set the gain (PGA) +/- 6.144V
    // Note that any analog input must be higher than –0.3V and less than VDD
    // +0.3
    PurplePCB.setGain(ADS1115_PGA_6P144);
    // ALERT/RDY pin will indicate when conversion is ready

    // Set the device to let us know when data is ready to be read
    PurplePCB.setConversionReadyPinMode();

    // Tell the device to start conversions of analog data!
    PurplePCB.setMultiplexer(ADS1115_MUX_P0_NG);
    PurplePCB.triggerConversion();
}

// This is an ISR function. It should be short and sweet!
ICACHE_RAM_ATTR void DataReady_Callback(void) {
    // Alert the main program that it can now get the data and process
    ProcessDataRequest = true;
}

void GetProcessData(void) {
    ProcessDataRequest = false;

    // Get the data from the PurplePCB
    // false only get the data from the device and complete no other action.
    float analogData = PurplePCB.getMilliVolts(false);

    Serial.print("A0: ");
    Serial.print(analogData);
    Serial.println("mV");

    snprintf (msg, 50, "%f", analogData);

    // Serial.println(msg);
    client.publish("ECG", msg);
    // }
}

void loop() {

    if (ProcessDataRequest) {
        GetProcessData();
    }

    if (!client.connected()) {
    reconnect();
    }

  client.loop();

}

This time when viewing pin ALERT on the DSO it can be seen that the output was correct with even sampling at a rate of 128.

DSO output of ALERT pin showing sample rate of 128

It was then time to re try a sinusoidal input into A0 of the ADS1115 to test that it was now correctly working. The set up can be seen below.

After the test’s success I moved onto adding back the mosquitto connection code (seen in the above code). I made a few changes, including removing the 2 second delay I didn’t realise was in the original example the team used, and having client.publish within the GetProcessData function rather than the void loop function as the analogue data had been defined in GetProcessData. At this point I felt I was finally starting to understand the MQTT part of the code.

I then viewed the sinusoid on the IoT OnOff app to test that I had connected to the broker.

IoT OnOff app sinusoid output


I also checked on SSH by opening Mosquitto to see that my data was being published.

~$ mosquitto sub -u "username" -P "password" -p 5269 -t "ECG"
jupyter notebook – python

After successfully connecting to Mosquitto, our facilitator advised me to use Jupyter Notebook to plot the live data. This was the first time I had used python, so I first started with a demo graph our facilitator had made for us. By added my out topic “ECG” I could connect to Mosquitto. Lines 12 and 13 were necessary as the output from python had ” ‘ ” either side of the value, so we needed to remove them before graphing.

After successfully graphing my sinusoid input, I then connected the ECG module to the rest of my circuit and attached the electrodes as described earlier. To compare between the actual analogue signal and the data being published, I also viewed A0 of ADS1115 on the DSO. The signal was very noisy and only looked correct at times. It was also very difficult to spot the peaks on the graph and on the DSO the R wave peak was only slightly higher than the T wave peak.

2 electrode output
electrode placement with additional grounding

After some time of confusion, I happened to touch the NodeMCU and suddenly a perfect ECG waveform appeared on both the DSO and the live python graph. Assuming this was something to do with grounding, I switched over to the 3 electrode lead and by touching the reference electrode, the signal was once again stable with a clear R wave – seen in the graph below. I will have to do some further research about ECG reference electrodes as I believed this was used to read different parts of the ECG waveform, and the BITalino website claimed that the 2 electrode lead had a ‘virtual reference’.

3 electrode output

After going back to the BITalino online store, it appears that the 2 electrode leads are designed for the EDA (ElectroDermal Activity) module, whereas the 3 electrode leads are for ECG, EEG, and EMG.

For this week Malan decided to deep dive into learning python for Jupyter Notebook and she discovered that to change the graph title, you can add f.layout.title = ‘title’. Later this week and next week I can ask her further how to format graphs on Jupyter Notebook.

Although next week is designated for ‘mechanical week’ – involving designing a case for our device and 3D printing it – I still have a few more steps to complete for my deep dive. Firstly, I still need to successfully acquire BPM data from my analogue data. At first I thought the best plan was to count the number of peaks in 10 seconds and then multiply that number by 6 – only if the value reaches above a threshold of 1840mV, however I realised it would then take 10 seconds for the BPM to be recalculated. Nic added on this idea by suggesting a shifting range that moves the 10 second block along every second to recalculate the number of peaks. Our facilitator built upon this idea even further with a faster and potentially more accurate method suggestion of measuring the time between peaks, dividing that time over 1 to get the frequency (periods per second) and then multiplying this frequency by 60 to get BPM. The next step for me is to look into how to measure time in between events in C++ and how to detect either a rising or a falling edge. From my initial searching, it appears that using the function difftime is the best method to use, however I will have to ask for further help. This deep dive has made me realise that although I have learnt a lot about programming during this studio there is still so much more for me to learn and understand.

Once I have calculated BPM, I will create a new Out Topic to be pushed to Node-RED through our broker and into Influxdb, and I will once again be able to visualise BPM on Grafana.


S U M M A R Y

Accomplished

– Adopted a new sensor
– Publishing ECG sensor data to MQTT
– Live graph in Jupyter Notebook using python


learnt

– Basics about analogue signals and sample rates
– Testing using the function generator and DSO
– Better understanding of functions in C++ as well as interrupts
– Very basic understanding of Python


What next

– Calculate BPM value and publish to Influxdb
– CAD and SolidWorks for case
– Optimising device on perfboard


Week 3: Infrastructure with Personal Devices

After creating our IoT infrastructure as a team with a simple test device, the next step was to connect our own devices to our server and start collecting data in the shared database. At first I thought this would be a simple task as we had already created and set up our infrastructure, however challenges still arose as I was using a different sensor and had not yet had any practice with Platformio.

sensor readings in platformio

To begin, I used same code as week 1 when I was still using Arduino IDE for just getting the sensor readings alone. I quickly realised I also needed to add a library for that code to function. After some quick research and asking our facilitator I learnt that to add libraries in Platformio, they are added under lib_deps in the Project Configuration File, as shown below.

lib_deps =     
    SparkFun MAX3010x Pulse and Proximity Sensor Libra
mqtt

After successfully getting the sensor readings for the MAX30102, the next challenge was to add the MQTT connection part of the code. I first went back to the original MQTT example we used as a team to create our final code for the temp/humidity sensor.

I then added the following libraries, the same as from last week.

lib_deps =     
    Adafruit Unified Sensor
    ESP Async WebServer
    PubSubClient

After uploading the MQTT example, my terminal showed “????” symbols and I remembered last week Tim mentioned that this was a monitor speed issue. I originally changed it to default 9600 however after getting the same issue I changed the monitor speed in the Project Configuration File as well and this resolved the issue.

monitor_speed = 115200

After realising that Tim’s code from last week for the DHT22 was quite different from what I needed to incorporate for my sensor, I compared both Tim’s code and the original MQTT example to try to work out how to implement a working MQTT code into mine.

I realised that the snprintf function was the one responsible for sending the packet/message to node-RED. I also learnt how to printf a long value for the beatsperminute using “%ld”. I then changed the outTopic name to “heartrate”- this name will be used in node-RED.

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <Wire.h>
#include "MAX30105.h"
#include "heartRate.h"

MAX30105 particleSensor;

// Update these with values suitable for your network.

const char* ssid = "ssid";
const char* password = "password";
const char* mqtt_server = "secret_name";
const char* mqtt_username = "secret_username";
const char* mqtt_password = "secret_password";
const char* sensor_Name = "heartrate";

const byte RATE_SIZE = 4; //Increase this for more averaging. 4 is good.
byte rates[RATE_SIZE]; //Array of heart rates
byte rateSpot = 0;
long lastBeat = 0; //Time at which the last beat occurred

long beatsPerMinute;
int beatAvg;

WiFiClient espClient;
PubSubClient client(espClient);
unsigned long lastMsg = 0;
char msg[50];
int value = 0;

void setup_wifi() {

  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();

  // Switch on the LED if an 1 was received as first character
  if ((char)payload[0] == '1') {
    digitalWrite(BUILTIN_LED, LOW);   // Turn the LED on (Note that LOW is the voltage level
    // but actually the LED is on; this is because
    // it is active low on the ESP-01)
  } else {
    digitalWrite(BUILTIN_LED, HIGH);  // Turn the LED off by making the voltage HIGH
  }

}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str(), mqtt_username, mqtt_password)) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish("outTopic", "hello world");
      // ... and resubscribe
      client.subscribe("inTopic");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
  pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output
  Serial.begin(115200);
  setup_wifi();
  client.setServer(mqtt_server, 5269);
  client.setCallback(callback);

    // Initialize sensor
  if (!particleSensor.begin(Wire, I2C_SPEED_FAST)) //Use default I2C port, 400kHz speed
  {
    Serial.println("MAX30105 was not found. Please check wiring/power. ");
    while (1);
  }
  Serial.println("Place your index finger on the sensor with steady pressure.");

  particleSensor.setup(); //Configure sensor with default settings
  particleSensor.setPulseAmplitudeRed(0x0A); //Turn Red LED to low to indicate sensor is running
  particleSensor.setPulseAmplitudeGreen(0); //Turn off Green LED
}

void loop() {

  long irValue = particleSensor.getIR();

  if (checkForBeat(irValue) == true)
  {
    //We sensed a beat!
    long delta = millis() - lastBeat;
    lastBeat = millis();

    beatsPerMinute = 60 / (delta / 1000.0);

    if (beatsPerMinute < 255 && beatsPerMinute > 20)
    {
      rates[rateSpot++] = (byte)beatsPerMinute; //Store this reading in the array
      rateSpot %= RATE_SIZE; //Wrap variable

      //Take average of readings
      beatAvg = 0;
      for (byte x = 0 ; x < RATE_SIZE ; x++)
        beatAvg += rates[x];
      beatAvg /= RATE_SIZE;
    }
  }  

  if (!client.connected()) {
    reconnect();
  }
  client.loop();

  unsigned long now = millis();
  if (now - lastMsg > 2000) {
    lastMsg = now;
    ++value;
    snprintf (msg, 50, "%s\n%.ld", sensor_Name, beatsPerMinute);
    Serial.print("BPM: ");
    Serial.println(msg);
    client.publish("heartrate", msg);
  }
  Serial.print("IR=");
  Serial.print(irValue);
  Serial.print(", BPM=");
  Serial.print(beatsPerMinute);
  Serial.print(", Avg BPM=");
  Serial.print(beatAvg);

  if (irValue < 50000)
    Serial.print(" No finger?");

  Serial.println();
}
plaformio serial monitor
node-red

The next step was to configure Node-RED for my individual topic, format the message and direct it into our influxdb database.

I first added an ‘mqtt in’ node, and changed the topic to ‘heartrate’ to match the programming in Platformio.

‘mqtt in’ node configuration

Next I added a ‘function’ node and similar to last week I split the payload by the new line \n. This time I renamed the second split to ‘BPM’ which simultaneously renames the new column in my table within the shared database.

Function node configuration

By using the debug node, I could see that my rephrasing was working correctly.

I then added an ‘Influxdb out’ node and created a new table HEARTRATE in the database TEMP.

‘influxdb out’ node configuration
Node-RED flow
Checking Influxdb

I then opened Influxdb in the SSH to view what data had been put into my new table.

michaela@TiMiNiMa:~$ influx
> use TEMP
> SELECT Sensor_Name, BPM FROM HEARTRATE
BPM data in database TEMP
Grafana

Grafana was quite simple to configure following on from last week. I created a new dashboard and input the query below to display my data in the graph. Using the same query, I displayed a gauge with the current BPM value.

Personal Grafana dashboard
Live data recording
IoT OnOff app

After setting up Grafana I wanted to create some kind of app dashboard that could display live data. After finding an iOS app “IoT OnOff”, I set up our Mosquitto broker with host being our Vultr ip address, the port being 5269, and added our authentication.

To create the graph, I set the subscribe topic to “heartrate”. To turn on and off the built in LED on the NodeMCU board, I added a button and set subscribe to “inTopic” and for true to be 1 and false 0. For the value display I also set the subscribe topic to “heartrate”.

resolving node-red and grafana

By removing the Sensor_Name variable and changing the message being sent to node-RED, this stopped Grafana from displaying data. The new message was only the BPM value without the string Sensor_Name. To reformat the message in node-RED, I renamed the msg.payload to be “BPM: msg.payload” in order to keep the previous measurement label.

New function node configuration
Chronograf

After fixing Node-RED I still had no data showing in Grafana. I looked to PuTTY to try and view the database however when I opened it there was only old BPM values and nothing new was being entered. Our facilitator advised our group to look into an Influxdb database explorer instead of using PuTTY to view our databases and tables. We then found Chronograf (owned by Influxdb) and after setting it up and viewing our data it came to my surprise that my BPM data was actually being put into database “TIMMY2” rather than “TEMP”. After this discovery, I adjusted the query in Grafana and my current BPM recordings appeared in both the graph and the value display. Chronograf was much more visual and easy to use compared to viewing databases in PuTTY and I will definitely continue to use this when viewing my data in future weeks.

Chronograf query in Explore

S U M M A R Y

Accomplished

– Integrated my personal prototype into our team database infrastructure
– Set up a live data visualisation using MQTT
– Displayed historical and current data in Grafana


Leant

– Platformio: better understanding of C++
– Further understanding of parts of the code for MQTT connection
– Better understanding of Node-RED and how to re-format data
– The possibilities of publish and subscribe: controlling other peoples devices through MQTT broker
– Displaying real time values separate from a database


What next

– Look into potentially more accurate sensors
– Decide what I would like to do for the hardware design: look into simple PCB design software, or other prototyping options


Week 2: IoT Infrastructure

Building an IoT Database as a Team

Diagram representing all three aspects of IoT

After the first week’s introduction to the data acquisition part of IoT product development, it was then time to explore the other half of IoT – the data storage and the data visualisation. Previously, both of these were ‘taken care of’ by the Blynk app, but this week’s task was to create our own IoT infrastructure as a team.

After forming our groups we first went away and did some individual initial research as we are all at different levels of knowledge about applications for databases and visualisations. We then came together to discuss what we found. During this initial research about IoT databases, I learnt quite a few new terminologies to do with the infrastructure:

Client = the device being used to collect data. In our case this is our temperature/humidity sensor device.

Broker = Receives messages from the client, then sends the data to clients subscribed to the server.

Publish = to send data to the server/broker.

Subscribe = to send data to the client/device.

To-do list in Teams

ooooo

As a group we used Teams to communicate tasks – as seen on the right – and send each other artifacts and resources to help each other understand the process of the infrastructure creation.

oooooo

The video below was also great help for understanding what is involved and how many applications may actually be required to create a functioning IoT infrastructure.

ooooooo

oooo

As a group we decided a similar approach to the one described in the video was the best approach, as seen below. For our infrastructure, rather than using a Raspberry Pi for our server, we used a remote server on Vultr as a “pi in the sky”.

We then allocated each person in the group to install one application onto the remote server.

sensor
Wiring for DHT22 with NodeMCU

We decided to begin with a simple temperature/humidity sensor to help us focus on the creation of the infrastructure.

We selected a DHT22 sensor with a NodeMCU board, as the majority of the group members are using this board for their personal projects. Last week, the ‘beginners’ including myself started with Arduino IDE to program the devices, however as a group we moved over to Platformio on Visual Studio. Malan and I were shown by Tim how to use Platformio and I now better understand the use of libraries and how to create multiple sections in the same project file. This will make it easier to start with some simple code then build upon it in later steps. Next week when I use Platformio on my own I will need to further learn how to use it and be more proficient with my programming learning.

After setting up our device we noticed that the data was in-congruent and continuously stopping. We asked our facilitator for guidance and learnt that pin D4 is not the best to use as GPIO2 is connected to one of the board’s built in LEDs. After looking up a pin layout diagram for the NodeMCU, we changed over D6 (GPIO12) instead, and this resolved our issue. I now better understand the pins of the NodeMCU and why I used D1 and D2 for my prototype as D1 and D2 are equivalent to SCL and SDA respectively. Later on I should still learn about all the other pin meanings and what they mean.

NodeMCU pinout
mosquitto broker

We decided to install Mosquitto as our broker as it is the most popular broker to use along side the MQTT protocol. We found a simple example from github to add to our sensor code. After having some connection issues, we reached out to our facilitator and were advised to switch over to his network rather than the UTS network as the uni has some firewall blocks on some ports. We also switched over to port 5269 rather than the default 1883, as seen in the code snippet below.

void setup() {
pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output
  Serial.begin(115200);
  dht.begin();
  setup_wifi();
  client.setServer(mqtt_server, 5269);
  client.setCallback(callback);
}

On the second day of this week we ran into more issues, however. Our Mosquitto broker was not connecting and after some research we learnt that we also needed to add a username and password due to authorisation issues, as seen in the highlighted sections below.

const char* ssid = secret_ssid;
const char* password = secret_password;
const char* mqtt_server = secret_mqtt_passwd;
const char* mqtt_username = secret_mqtt_username;
const char* mqtt_passwd = secret_mqtt_passwd;
const char* sensor_Name = "11.11.403 Temp/Hum";
void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str(), mqtt_username,mqtt_passwd)) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish("outTopic", "hello world");
      // ... and resubscribe
      client.subscribe("inTopic");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }

Another technical skill I learnt during this week was how to use the SSH using PuTTY to install the applications onto our sever. I have previously done a subject in which I used PuTTY to get data from a database using SQL queries but I have never made a database nor used PuTTY to install applications.

NODE-RED

We decided to use Node-RED as the link between the MQTT protocol and our database in Influxdb as it is very visual, allowing us to connect various nodes together to format our data. After installing Node-RED, we first created a flow that could inject some arbitrary values into the database to test our column names and that numbers were being entered correctly. Once we were happy with that we connected our ‘MQTT in’ node to the ‘Influxdb out’ node. This still wasn’t correct because the output was all three of our values in one line, when we needed one packet with three lines – one for each value. After some more research we discovered we should use a function node to correct the formatting. Using Node.js, we set up a ‘split’ that would separate the payload into an array based on the occurrence of a new line (\n). This was also where we could name our database columns – Sensor_Name, Humidity and temp. Node-RED was also my first introduction to the Node.js language and next week I should learn the basics of this language in order to successfully add all the team member’s device outputs into Node-RED.

influxdb

We installed Influxdb using these instructions. After a discussion with all summer studio students and our facilitator, I learnt that there are different types of databases – SQL and no SQL. Influxdb is a no SQL database, meaning there are no relationships between the data and it does not require a primary key. I also learnt that no SQL databases are best for large databases where there is less likely to be relationships and data repeats are not an issue.

grafana

We decided to use Grafana as our visualisation tool as it is recommended for Influxdb and offers a wide variety of visualisation options. After installing Grafana, we entered our data source by inputting our influxdb database name, username and password. We discovered that Grafana is not as intuitive as we first thought. We had to use PuTTY to check the column headings in the database, as after making many previous alterations it started to become confusing. After some more research we worked out how to enter queries into Grafana in the correct format, as seen below. Next week we will have to practice these queries more in order to create the correct graphs/visualisations for everyone’s individual data being collected.

SELECT Temperature FROM "TEMP".."TEMP" WHERE ("Sensor_Name" = '11.11.403 Temp/Hum')
SELECT Humidity FROM "TEMP".."TEMP" WHERE ("Sensor_Name" = '11.11.403 Temp/Hum')
Grafana dashboard displaying humidity and temperature graphs

Grafana was reasonably simple to use and has a good range of visualisation tools and customisation, however I’m not sure if I want to use Grafana for my individual project because I want to use some kind of of an app visualisation so it can be easier for an individual to quickly access on their person. Instead, Grafana could be the HCPs dashboard, while individuals have their own app. Because I would like to focus on the hardware aspect of IoT in the following weeks, I will continue with Grafana in the next sprint. From here I will need to learn how Grafana sends alerts as this will be a useful addition to my heart rate monitor.

S U M M A R Y

AccomplishedCreated an IoT infrastructure for a basic device as a team
Learnt– Structure of IoT back-end
– Platformio
– How to use the applications Node-RED, Grafana
– How to install applications using PuTTY
What next– Move over to Platformio for my project
– Set up everyone’s device onto our database
– Need to look into different visualisation options

Week 1 Continued: Proof of Concept

MAX30102 Pulse Oximeter and
Heart-Rate Sensor

The next task was to produce a Proof of Concept (PoC) prototype for our previously identified problem. After the define and idea phases of the design process comes the prototyping phase. For my PoC I selected a pulse oximeter and heart rate sensor, although I will only be utilising its heart rate measuring functionality for my project.

Breadboarding and Arduino IDE

As someone with very limited programming experience and no experience in Arduino IDE, the PoC phase was quite challenging but offered an opportunity to learn this technical skill in an intense, hands-on approach. Following the advice of our facilitator, I first used an Arduino UNO board and found some example code from a tutorial for just the MAX30102 component and the UNO board – without any WiFi module. Doing this was very helpful for me to have an introduction to the technical skill of programming in Arduino IDE and I started to understand the use of libraries, the setup and loop sections, and choosing the correct board. The video below shows my initial PoC prototype successfully outputting data from the heart rate sensor in the Serial Plotter window.

Adding a WiFi Module

After I felt comfortable to move on, I then switched over to a component with a WiFi module – this being the NodeMCU board with an ESP8266. I chose to go with the ESP8266 after asking our facilitator’s advice that an older module will have more resources available to learn from. Another technical skill I built upon during the ‘hacking’ phase was breadboarding. I have previously done breadboarding in electrical engineering subjects however not in this context with components such as these. After looking up some tutorials and asking my more advanced peers to review my work I now understand the pins used in this prototype which i didn’t understand before – that being SCL (serial clock), and SDA (serial data).

NodeMCU with ESP8266
Breadboarding for final PoC
Blynk App

Although I had previously used the Blynk app for the IDL, I had not seen how to program the firmware to display the desired data on the widgets. At this point I started to utilise the Blynk Example pages as recommended by our facilitator. After noticing that my device was not connecting to the internet, I learnt that to get it working on the Blynk app I had already set up, my peers noticed I also needed to add the server and port provided by our facilitator, in order to connect to the UTS Blynk server rather than the Blynk cloud server. The code for the Blynk app is provided below. From this point I still need to learn more about what the server and port means – this will probably be explored later in the studio when we move onto developing the back-end framework for our devices.

#define BLYNK_PRINT Serial
#include <ESP8266WiFi.h>
#include <BlynkSimpleEsp8266.h>
// You should get Auth Token in the Blynk App.
// Go to the Project Settings (nut icon).
// Your WiFi credentials.
// Set password to "" for open networks.
char ssid[] = "iPhone";
char pass[] = "-------------";
char auth[] = "--------------------";
const char Blynk_Server[] = "iot.nortcele.win";
const uint16_t Blynk_Port = 8080;

void setup()
{
  Serial.begin(9600);
  Blynk.begin(auth, ssid, pass, Blynk_Server, Blynk_Port);
}
void loop()
{
  Blynk.run();
}
Correcting BPM Value

Although my device was now connecting to the Blynk app, at this point I recognisd two issues: my Blynk app was not displaying any data, and the BPM values measured were incorrect. I decided to first correct the BPM issue. This was quite a simple fix by looking up more example tutorials of heart rate monitor devices using the MAX30102 and NodeMCU, and finding the calculation required. The code below was added to the rest of my IDE. At this stage I also learnt that it is important to stay consistent with the source of the example code used and ensure that it is designed for the components used, otherwise it may not work.

#include "heartRate.h"
const byte RATE_SIZE = 4; //Increase this for more averaging. 4 is good.
byte rates[RATE_SIZE]; //Array of heart rates
byte rateSpot = 0;
long lastBeat = 0; //Time at which the last beat occurred
float beatsPerMinute;
int beatAvg;

void setup()
{
  particleSensor.setPulseAmplitudeRed(0x0A); //Turn Red LED to low to indicate sensor is running
  particleSensor.setPulseAmplitudeGreen(0); //Turn off Green LED
}
void loop()
{
 // Blynk.run();
  long irValue = particleSensor.getIR();
  if (checkForBeat(irValue) == true)
  {
    //We sensed a beat!
    long delta = millis() - lastBeat;
    lastBeat = millis();
    beatsPerMinute = 60 / (delta / 1000.0);
    if (beatsPerMinute < 255 && beatsPerMinute > 20)
    {
      rates[rateSpot++] = (byte)beatsPerMinute; //Store this reading in the array
      rateSpot %= RATE_SIZE; //Wrap variable
      //Take average of readings
      beatAvg = 0;
      for (byte x = 0 ; x < RATE_SIZE ; x++)
        beatAvg += rates[x];
      beatAvg /= RATE_SIZE;
    }
  }
   Serial.print("IR=");
   Serial.print(irValue);
   Serial.print(", BPM=");
   Serial.print(beatsPerMinute);
   Serial.print(", Avg BPM=");
   Serial.print(beatAvg);
   if (irValue < 50000)
     Serial.print(" No finger?");
   Serial.println();
}
Virtual Pins
Blynk app with outputs from V1, V5, V6

It was then time to fix the issue of no data being displayed on the Blynk app widgets. Our facilitator recommended for me to look into ‘Blynk timer’ when I asked for advice. I found Blynk timer on Blynk Examples, however didn’t demonstrate how to write data to the app, only to set up the up-time. After being unable to work out what to do I turned to some electrical/electronics engineering students for advice and guidance. It was also at this point it was getting confusing as to what I had added and when, and I was told it is better to save multiple versions at each step separately for version control. After adopting this method it became much clearer where my roadblocks were occurring. The more experienced students helped to further explain the code to me and I now better understand events in Arduino IDE and that these need to be ‘referenced’ in both the setup and loop sections for the function to work. We then tried moving the Blynk.virtualWrite() statements into the Timer event rather being a separate event and this resolved the issue. The added code below shows the working programming for the virtual pins V1 for the timer, V5 for the BPM and V6 for the average heart rate.

BlynkTimer timer;
void myTimerEvent()
{
  Blynk.virtualWrite(V1, millis() / 1000);
  Blynk.virtualWrite(V6, beatAvg);
  Blynk.virtualWrite(V5, beatsPerMinute);
}
void setup()
{
  timer.setInterval(1000L, myTimerEvent);
}
void loop()
{
  timer.run();
}

This week I achieved a great amount of progress, especially with my new understanding of Arduino IDE. I now feel that I have a grasp of the basics, however I still have a long way to go if I am going to do further embedded software in the future.

As a group, everyone achieved some level of PoC prototype by the end of the first week. In the following weeks we should all work on giving each other more regular peer feedback and evaluation as the studio progresses. Currently our peer reviewing consists of our facilitator asking us all what we are up to and some students ‘checking in’ with other’s progress every so often. Going forward we should organise a more systematic approach to review each other’s performance, especially as the studio becomes more challenging.

S U M M A R Y

AccomplishedPoC prototype
Learnt– Arduino IDE basics

– ‘Hacking’ phase of design process
What next– Better system of peer review each week

– Further Arduino IDE programming

– The IoT back-end framework to support everyone’s IoT products

– Need to ask for more peer and facilitator feedback

Week 1: Identify the Problem

The first step was to identify a problem that was personal and could be solved by an IoT product. For my first IoT project, I decided to pursue to design a heart rate monitor device as it seemed to be a relatively simple project, with many resources and tutorial examples.

The Stakeholder

There are currently several solutions to monitor heart rate. Fitness watches contain a green LED to measure heart rate, and continuously collect this data for the user, however prices range from $117-$550, making them not an affordable or viable solution for many. Free heart rate apps are also available. These apps use photoplethysmography through the use of the white light from a phone camera to sense and record heart rate. However, these apps cannot capture live data nor collect it over time, and are generally considered to be inaccurate.

During my initial research I have learnt that in emergency situations the most common technologies utilised to measure heart rate are a pulse oximeter or portable ECG machine. ECG machines are cumbersome and require multiple electrode leads to be adhered to the body. It is considered policy for nurses to manually measure patient vital signs including pulse rate [1], but automatic machine testing can be used for patients under more frequent observation. Inaccuracies of pulse oximeters are commonly caused by poor peripheral circulation, excessive movement or even nail polish [2]. In addition, ECG machines and pulse oximeters may display live data but are either not wireless or do not collect data over time.

Initial idea sketches

I wanted to think of an alternative solution to measure heart rate. My solution is for a device that can be placed anywhere on skin – suitable for an emergency situation where perhaps the wrist or fingers are not accessible, and also solving the problem of pulse oximeters’ inaccuracies by being closer to the torso. My initial idea was for a removable device consisting of a heart rate sensor and a WiFi module, positioned in a fabric pouch with a silicone backing, meaning the device can stick comfortably to the skin without movement causing loss of contact or inaccuracies. The data collected from the sensor can then be displayed with real time Beats Per Minute (BPM) display and graph to visualise the data over time. This concept can be seen in my initial idea sketch drawings above.

Upon further research, a similar product to my initial idea already exists on the market. VitalPatch by VitalConnect is a disposable single-use solution with an adhesive backing. It contains ECG electrodes to measure heart rate and a Bluetooth Low-Energy transceiver to capture data. Its efficacy was validated in a 2018 observational study [3] and was found to have great potential for the remote monitoring of high-risk patients.

VitalPatch

Although a similar solution has already been created, it does prove that this idea is viable with clinical evidence. From the VitalPatch I also identified an additional problem to address with my device. The VitalPatch is a single-use disposable option, whereas my solution can be reusable for a single patient by incorporating a washable reusable fabric and silicone pouch. This introduces a new stakeholder – the every day health-conscious user who does not want to or perhaps cannot wear a variation of a fitness watch.

IoT Data Logger
IoT Data Logger from previous experience

I have previously soldered an IoT Data Logger (IDL). This gave me technical soldering experience and an introduction to various sensors and what is possible for an IoT device. This device was also set up for use with the Blynk app – used to control the device and display the data readings. Because I had previous experience with soldering the IDL, I assisted my peers who were soldering for the first time and reviewed their performance. This also reaffirmed my communication skills when guiding students who were learning how to solder for the first time. At this point I had not had any experience in the programming for the firmware, which I will require the help of my peers and their evaluation of my progress.

As a group, everyone has their own individual problems they want to solve with an IoT device. After a quick briefing on the components available to us, everyone selected the sensors that interested them – from hygrometers and soil moisture sensors for a greenhouse device to facial recognition using an ESP32 with a built in camera, to weight sensors for stock tracking. Although everyone is at a different experience level with different degree backgrounds, everyone has started to identify what they would like to do for their projects.

S U M M A R Y

Accomplished– Researched potential stakeholder needs to identify the need of a reusable wireless heart rate monitor which utilises red and green light

– Began to establish the design of a solution

– Helped review/evaluate my peers soldering skills, while simultaneously practicing my communication skills
LearntA wireless heart rate monitoring using biosensors is a viable solution for Health Care Professionals, as well as every day people
What next– Create a Proof of Concept prototype

– Need to learn how to program

– Research more into heart rate sensors using red versus green LEDs