ESP32 Biometrics: PPG Wave on web page

Introduction

In this tutorial we are going to check how to draw a PPG wave in real time on a plot, rendered on a web page, served by the ESP32. The measurements that will be used on the plot will be taken from a Heart Rate sensor module connected to the ESP32. We will be using the Arduino core to program the ESP32.

This tutorial will be built upon some others that we have covered in the past. So, expect some references to those tutorials for more detailed explanations in some parts of the code.

The tests shown below were performed on a ESP32-E FireBeetle board from DFRobot. The Arduino core version used was 2.0.0 and the Arduino IDE version was 1.8.15, working on Windows 8.1. 

The application architecture

Like already mentioned, we will gather the measurements from a Heart Rate sensor module that will be connected to the ESP32. We have already covered how to use this module to get Heart Rate measurements here. This tutorial also explains how to wire the sensor to the ESP32.

After covering the basics on how to interact with the Heart Rate sensor, we have also checked how to get a PPG wave on the Arduino IDE serial plotter. If you haven’t done so yet, it is recommended that you follow this tutorial first because we will use most of the code here. It is also important to ensure that the sensor is working appropriately when connected to the ESP32 and trying to diagnose problems in that part in a complex end to end application like the one we are going to create here is much more difficult.

For this tutorial, we will assume that we will send the measurements in “real time” and the plot will get updated when new measurements are received by the browser that is rendering the web page. Note that the term “real time” is used here not totally correctly as “real time” has a different meaning in the world of embedded systems. Nonetheless, for the sake of simplicity, we are assuming that “real time” consists on sending the measurements as fast as possible to the client once we get them, and that the client is going to display them immediately.

Referencing again the tutorial where we checked how to draw a PPG wave, we can see that the delay between each measurement was very small (around 10 milliseconds). Naturally, the more points we can collect, the better our wave should look like. Note however that we won’t be able to keep such a small delay between samples (we will instead use a delay of 100 milliseconds) and the reason will be explained below.

But before that, we need to understand how we are going to make this measurements get to a web page. First, we can already expect to have to serve the web page that will contain some HTML and JavaScript to draw the plot and process the measurements. For that, we will setup an async HTTP web server on the ESP32 (check here how to get started and how to install all the libraries).

Once a web page is rendered, there are many ways to fetch data asynchronously to update elements on the page. The most basic one consists on doing async HTTP requests with JavaScript, to fetch the information we need. Nonetheless, we already mentioned that we will be sampling the sensor roughly every 100 milliseconds, which makes it impractical (and inefficient) to be doing a new HTTP request every 100 milliseconds.

Furthermore, it would be the client asking the server for new measurements instead of having the server pushing those values once they are ready. As such, a more viable alternative is using websockets, which allow to establish a permanent connection once (the client will send a request to the server to establish the connection). Then, the server can push the measurements periodically and the client will react accordingly, updating the plot.

Fortunately, the Async HTTP web server libraries that we will use to setup the HTTP server also have a module to expose websocket endpoints. You can read a very detailed explanation here.

Although this is a much more efficient alternative, it is important to keep in mind that it also has limitations. For example, the async web server uses a queue underneath to hold the messages to be sent to the websocket client. This queue has a limited size (we must not forget that, no matter how powerful the ESP32 is, it is still a microcontroller with resource constraints), meaning that if we produce measurements at a pace much higher than they can be sent to the client, the queue will become full and our system will misbehave.

This is the reason why we are keeping the delay between measurements in the 100 milliseconds. Naturally, you can try slightly smaller values but, during my tests, values such as 10 milliseconds caused issues with the internal queue. Other option that you can also explore in case you need to reduce the delay is to fine tune the size of this internal queue, which is defined here. At the time of writing I did not find any API to change this value, so you will need to edit the source code of your local copy of the library.

The file that contains the HTML and JS code will be served from the SPIFFS file system of the ESP32, which is supported by the async HTTP web server library. For a detailed explanation on how both interact to serve a file, you can consult this tutorial.

From the client side, we will be using the plotly library to render the chart. We will briefly analyze the functions called, but please take in consideration that the focus of the tutorial is not to enter in detail on how plotly works. For that, it is recommend that you consult the documentation, since the library allows for a tremendous amount of customization.

Also, the code we are going to use to draw and update the plot was adapted from here.

The Arduino code

We will start our code by doing all the library includes:

• WiFi.h: Allows to connect the ESP32 to a WiFi network.
• SPIFFS.h: Allows to mount and interact with a SPIFFS file system on the ESP32.
• ESPAsyncWebServer.h: Allows to setup an async HTTP web server to operate on the ESP32.

#include <WiFi.h>
#include <SPIFFS.h>
#include <ESPAsyncWebServer.h>

After this we will define two global variables that will hold the credentials of the WiFi network: the name (SSID) and the password. Note that, in your code, you should replace the placeholders I’ll be using below by the actual credentials of your network.

const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPassword";

We will also define a variable to hold the number of the analog ESP32 pin connected to the Heart Rate sensor module. Please take in consideration that, when we are using the WiFi interface of the ESP32, the ADC2 cannot be used [1]. This means that we should use ADC1, which is attached to GPIOs 32 to 39 [1]. In my case, I’ll be using GPIO 35.

const int sensorPin = 35;

To be able to setup the HTTP server on the ESP32, we will need an object of class AsyncWebServer. As input, the constructor of this class receives the port where the server will be listening for incoming HTTP requests. We’ll be using port 80, which is the default HTTP port.

AsyncWebServer server(80);

To setup a websocket endpoint we need to create an object of class AsyncWebSocket. As input, the constructor receives a string with the websocket endpoint. We will call it “/wave“.

AsyncWebSocket ws("/wave");

Since we are going to need to access the websocket client object to periodically send it the sensor data, we will declare a pointer to an object of class AsyncWebSocketClient. As we have covered in detail on this previous tutorial, the websocket events handling function will receive a pointer to an object of this class when an event occurs.

In order to send data back to the client, one of the possible options is to store this object pointer and then use it where we need (in our case, it will be in our main loop, where we will periodically poll the sensor and send the data).

For now, after the program starts, we know that no client is connected. Taking this in consideration, we will initialize the global pointer explicitly as NULL. As long as it has the value NULL, we know that no client is connected and that we should not send any data.

Just for reference, this is not the only way to send data to a websocket client. As we can see here, the AsyncWebSocketClient class provides a function to broadcast data to all the clients connected and also to send a message to a single client, given his ID.

AsyncWebSocketClient * globalClient = NULL;

After finishing the definition of all global variables, we will move to the Arduino setup function. As we usually do, we will open a serial connection so we can output some results from the execution of our program.

Serial.begin(115200);

After that we will take care of mounting the SPIFFS file system. We will need this to be able to access and serve the HTML code.

if(!SPIFFS.begin()){
   Serial.println("An Error has occurred while mounting SPIFFS");
   return;
}

The next step will be to connect the ESP32 to the WiFi network, using the previously defined credentials. Once the connection is established, we will print the local IP address assigned to the ESP32, so we can use it to reach the HTTP server from a web browser.

WiFi.begin(ssid, password);
 
while (WiFi.status() != WL_CONNECTED) {
  delay(1000);
  Serial.println("Connecting to WiFi..");
}
 
Serial.println(WiFi.localIP());

Then we will take care of binding the websocket endpoint to its event handling function. We will call it onWsEvent and analyze its implementation later.

ws.onEvent(onWsEvent);

Then we will register the AsyncWebSocket object on our server.

server.addHandler(&ws);

We will also register a route on our server. It will answer to HTTP GET requests on the “/wave” route and it will return the HTML and JavaScript code. Note that we are using the C++ lambda syntax to define the route handling function.

The implementation of this route handling function consists on calling the send method on the AsyncWebServerRequest object to which we receive a pointer as input of the of the function. This method allows to send the response back to the client and it has many overloads.

In our particular case, we will need to pass as first input the SPIFFS external object, which allows to interact with the file system, as second input the name of the file we want to serve and as third and last input the content-type, so the browser knows how to interpret it.

Note that we will analyze the code of the HTML file in the next section.

server.on("/wave", HTTP_GET, [](AsyncWebServerRequest *request){
  request->send(SPIFFS, "/wave.html", "text/html");
});

To finalize the Arduino setup we will call the begin method on our server object, so it starts listening to incoming requests.

server.begin();

The complete setup code is available on the snippet below.

void setup(){
  Serial.begin(115200);
 
  if(!SPIFFS.begin()){
     Serial.println("An Error has occurred while mounting SPIFFS");
     return;
  }
 
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
 
  Serial.println(WiFi.localIP());
 
  ws.onEvent(onWsEvent);
  server.addHandler(&ws);
 
  server.on("/wave", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SPIFFS, "/wave.html", "text/html");
  });
 
  server.begin();
}

We will now check the implementation of the callback function that will handle the websocket events. Note that we are not going to receive any data, only sending it, meaning we are not going to need to worry about received messages. Consequently, the only two events that we will handle are when a connection is received and when a client disconnects.

Before we analyze the implementation, let’s first consider the signature of this callback function. It should return void and receive 5 inputs. In our case we will only need to worry about the second and the third:

• A pointer to an AsyncWebSocketClient object.
• The type of event.

void onWsEvent(AsyncWebSocket * server, AsyncWebSocketClient * client, AwsEventType type, void * arg, uint8_t *data, size_t len){
   // implementation
}

So, in case we detect a client connection event (WS_EVT_CONNECT), we will store the pointer in our previously defined global variable. In case we detect a client disconnection event (WS_EVT_DISCONNECT), we will set our global AsyncWebSocketClient pointer to NULL.

void onWsEvent(AsyncWebSocket * server, AsyncWebSocketClient * client, AwsEventType type, void * arg, uint8_t *data, size_t len){
 
  if(type == WS_EVT_CONNECT){
 
    Serial.println("Websocket client connection received");
    globalClient = client;
 
  } else if(type == WS_EVT_DISCONNECT){
 
    Serial.println("Websocket client connection finished");
    globalClient = NULL;
 
  }
}

To finalize our code, we will now take a look at the implementation of our Arduino main loop. The first thing we will do is checking if our client pointer is different from NULL and, in case it is, also check that its status is equal to WS_CONNECTED. In case both conditions are true, we can send data to it.

if(globalClient != NULL && globalClient->status() == WS_CONNECTED){

  // Get Measurement and send to client
}

In case a client is connected, we will get a sensor measurement with a simple analogRead function call. As input, we will pass the variable that holds the number of the GPIO connected to the sensor.

int value = analogRead(sensorPin);

Then we will send this measurement to the client. Note however that only binary or string messages are supported, meaning that we will convert the measurement to a string before sending it.

String randomNumber = String(value);
globalClient->text(randomNumber);

We will also print the value (as integer) to the serial port, which can be helpful for debugging (we can still open the serial plotter to check if the measurements are being correctly obtained from the sensor and the PPG wave can be drawn).

Serial.println(value);

Finally we will introduce a 100 milliseconds delay between each iteration of the loop. The whole Arduino loop is shown below.

void loop(){
   if(globalClient != NULL && globalClient->status() == WS_CONNECTED){
      
      int value = analogRead(sensorPin);
      String randomNumber = String(value);
      globalClient->text(randomNumber);
      Serial.println(value);
   }
   
   delay(100);
}

The full code is available below.

#include <WiFi.h>
#include <SPIFFS.h>
#include <ESPAsyncWebServer.h>
 
const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPassword";

const int sensorPin = 35;
 
AsyncWebServer server(80);
AsyncWebSocket ws("/wave");
 
AsyncWebSocketClient * globalClient = NULL;
 
void onWsEvent(AsyncWebSocket * server, AsyncWebSocketClient * client, AwsEventType type, void * arg, uint8_t *data, size_t len){
 
  if(type == WS_EVT_CONNECT){
 
    Serial.println("Websocket client connection received");
    globalClient = client;
 
  } else if(type == WS_EVT_DISCONNECT){
 
    Serial.println("Websocket client connection finished");
    globalClient = NULL;
 
  }
}
 
void setup(){
  Serial.begin(115200);
 
  if(!SPIFFS.begin()){
     Serial.println("An Error has occurred while mounting SPIFFS");
     return;
  }
 
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
 
  Serial.println(WiFi.localIP());
 
  ws.onEvent(onWsEvent);
  server.addHandler(&ws);
 
  server.on("/wave", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SPIFFS, "/wave.html", "text/html");
  });
 
  server.begin();
}
 
void loop(){
   if(globalClient != NULL && globalClient->status() == WS_CONNECTED){
      
      int value = analogRead(sensorPin);
      String randomNumber = String(value);
      globalClient->text(randomNumber);
      Serial.println(value);
   }
   
   delay(100);
}

The HTML and JavaScript code

We will start by including plotly in the head section of the HTML document. We will point the src atribute of the script tag to the CDN link that provides the minified version of the library:

<head>
	<script src="https://cdn.plot.ly/plotly-2.4.2.min.js"></script>	
</head>

Note that, by using this link, we are assuming that the machine that will be connecting to the ESP32 has Internet access, so it can fetch this external resource. I’ve opted for this approach since the minified version of plotly is actually quite big (around 3.4 MB), meaning it would occupy a lot of space in boards that have only 4MB of flash memory.

Nonetheless, if you have a use case that requires to serve all the resources from the ESP32 (example, the device is supposed to work only in soft AP mode), then there are multiple ways to solve the issue:

• Use a module with more memory (there are modules that have more than 4MB of flash).
• Use a SD card to store the file.
• Use a smaller chart library (most of the complexity of the code doesn’t depend on the chart library, which can be easily changed).

We will need a div in the document body, where plotly will then draw the chart. We will assign an id to this div and set its width and height with an inline CSS style.

<div id = "plot" style="width:1000px;height:300px;"></div>

Then we will write the JavaScript code that will take care of opening a websocket connection, initializing the plot and updating it every time a message is received.

We will start by creating an object of class WebSocket. As input of the constructor we will pass a string containing the websocket endpoint in our ESP32 server.

Note that we will use the hostname property of the Location interface to get the IP address of our ESP32 without having to hardcod it or write it on the server before sending the response to the client. This will ensure that the HTML page below will still work even if your ESP32 gets a different local IP address than mine when it gets connected to the WiFi network (this is the most likely scenario).

var ws = new WebSocket("ws://" + document.location.hostname + "/wave");

We will also initialize a JavaScript Array with 500 positions set to zero. This Array will hold the latest 500 measurements done at any given moment. Naturally, when the application starts, none will be available and it will start getting filled as the server starts sending them.

var measurements = Array(500);
measurements.fill(0);

Next we are going to define a handling function for the WebSocket connection event. It will simply open an alert to indicate that we are connected to the ESP32.

ws.onopen = function() {
		window.alert("Connected to ESP32");
};

Then we will create a new plot with a call to the newPlot function. As first input we pass a string with the ID of our div. As second input we pass an array of objects called “data“, which contains the data to be plotted. You can check the full reference here, but in our case we will have a single object in the array, with the following properties:

• y: The array containing the y coordinates (in our case, the measurements).
• mode: Defines the drawing mode. We will pass the value ‘lines’.
• line: An object that contains some definitions for the lines of the plot. In our case, we will simply set the color.

Note that there is a huge amount of parameters to be configured and adapt the plot for your use case. Those are very well described in the documentation.

Plotly.newPlot('plot', [{
	y: measurements,
	mode: 'lines',
	line: {color: '#80CAF6'}
}]);

Finally we will define the handling function for the websocket message received event.

ws.onmessage = function(evt) {

    // Handling function implementation
};

We will start by adding the new measurement received on the message to the end of our Array. We can do so by calling the push method on our Array of measurements.

var newVal = evt.data;
measurements.push(newVal);

Since we want to keep the same Array size (number of measurements), we will now remove the oldest measurement, which is the one at the beginning of the Array (basically it will work as a FIFO – First In First Out). We do so with a call to the shift method.

measurements.shift();

Then we will create a new object to represent the updated measurements. It will contain only a property called y, where we will wrap our array of measurements inside another array.

var updatedData = {
	y: [measurements]
};

Finally, we will call the update function to update the plot with the new values. This function receives as first input the ID of the plot and as second the object with the new data.

Plotly.update('plot', updatedData);

The complete callback definition is shown below.

ws.onmessage = function(evt) {

ws.onmessage = function(evt) {
				
	var newVal = evt.data;
	measurements.push(newVal);
	measurements.shift();

	var updatedData = {
		y: [measurements]
	};

	Plotly.update('plot', updatedData);
};

The full code is available below.

<!DOCTYPE html>
<html>
	<head>
		<script src="https://cdn.plot.ly/plotly-2.4.2.min.js"></script>	
	</head>

	<body>
	  <div id = "plot" style="width:1000px;height:300px;"></div>
	  <script>
			var ws = new WebSocket("ws://" + document.location.hostname + "/wave");

			var measurements = Array(500);
			measurements.fill(0);

			ws.onopen = function() {
				window.alert("Connected to ESP32");
			};

			Plotly.newPlot('plot', [{
			  y: measurements,
			  mode: 'lines',
			  line: {color: '#80CAF6'}
			}]);
			
			ws.onmessage = function(evt) {
				
				var newVal = evt.data;
				measurements.push(newVal);
				measurements.shift();

				var updatedData = {
					y: [measurements]
				};

				Plotly.update('plot', updatedData);
			};

		</script>
	</body>
</html>

Now that we have the HTML file ready, the easiest way to upload it to the file system is using the Arduino IDE SPIFFS plugin. The procedure on how to do it is covered in detail on this previous tutorial. Note that the file should be called wave.html, in order to match the name of the file we are serving on the Arduino code.

Testing the code

The first thing that needs to be done is ensuring that the sensor module is correctly wired to the ESP32, accordingly to the diagram from here. Note however that, on the mentioned diagram, I’ve used pin 25 but, for this test, we cannot use that pin because it uses ADC2 of the ESP32, which becomes unavailable when we use WiFi. Instead, we should use a GPIO from ADC1 (as seen on the code, I’m assuming the usage of GPIO 35).

We also need to guarantee that the switch on the Heart Rate sensor module is on the Analog position (signaled with an “A”). This is needed because it will operate in Analog mode.

Once this is done, we can compile and upload the code to the ESP32. Once the procedure finishes, we need to open the Arduino IDE serial monitor, wait for the WiFi connection to be established and then copy the IP address that gets printed.

Them, we must open a web browser and write the following in the address bar, replacing #ESP_IP# by the IP address we have just copied:

http://#ESP_IP#/wave

Once we do that, and assuming that we haven’t yet attached the sensor to a finger or a wrist, we should see a plot with all values set to zero. We should also see the alert indicating the connection to the ESP32.

After that, if we attach the sensor to a finger or wrist, we should start seeing the wave getting printed, as illustrated below in figure 1.

Note that we did not specify any x axis values in the HTML page. Consequently, the chart will represent 1 unit in the x axis per each measurement in the array (recall that it has a length of 500). It is also important to mention that the more stable the sensor is, the cleaner the wave will be.

Regarding the update of the plot, in my experiences I’ve noticed some small freezes every once in a while. I did not investigate in detail what caused them, but it may be related with the rate of updates done.

Debugging tips

The application we have covered on this tutorial already has a reasonable amount of complexity. It has the interaction between the ESP32 and the sensor and then the interaction between the ESP32 and the client. It’s very normal that you will find some issues while trying to reproduce the results from the tutorial, but it is very important to know how to overcome them.

The most important message is that you shouldn’t try to find the issue by looking to the whole end to end system. Like already briefly mentioned, you should first make sure that each individual part is working as expected, and only then connect everything. This is even more important if you are not yet experienced with any of the libraries / concepts we are using, making it even harder to guess where the issue may be.

Some suggestions that may be helpful:

• Make sure the ESP32 and the sensor are working correctly. Like already mentioned, you can start by making sure that you can obtain the PPG wave in the Arduino IDE serial plotter, without adding any of the WiFi / HTTP server code (tutorial here). Even in the end to end code, notice that we are printing the measurements to the serial console . This was done on purpose because even after you connect everything together, you can still start the serial plotter and confirm that the measurements are being correctly obtained.
• Test the HTML / JS from a computer. You don’t need to start immediately by serving the page from the ESP32. You can change the document.location.hostname piece of code by a string with the IP address of the ESP32 and render the HTML page in a computer (just double click it and a browser should open it). Note that the computer must be on the same WiFi network as the ESP32.
• Make sure the correct HTML / JS is being served. In most browsers, you can simply right-click the page and select the option to see the source code. There you can confirm that the HTML and JS is exactly the same we have covered above.

Final Notes

It is important to look at the code and the end to end system we have analyzed here with a critical perspective. Naturally, this is a very simplistic implementation that is not optimized and neither considers some error situations.

For example, instead of sending an individual message every 100 milliseconds, we could explore a mechanism to buffer a couple of measurements and send them with less frequency.

We could have also used Timer Interrupts or the Ticker library to setup the periodicity of the measurements in a cleaner way, instead of having this defined with delays in the Arduino main loop.

Other thing that could have been done was minifying the HTML and JS code before uploading it to the Arduino SPIFFS file system.

Still, many of the optimizations that could be done depend on the actual use case that we want to cover. Naturally, there’s no single solution for such an application and it always needs to take in consideration the requirements. As such, the main objective was just to explain how we can put together some of the concepts we have been covering in the blog, and hopefully serve as a starting point for anyone who wants to implement a similar application but has more concrete requirements.

References

[1] lhttps://docs.espressif.com/projects/esp-idf/en/v4.3-beta3/esp32/api-reference/peripherals/adc.html