ESP32: Dynamic sensor network

Contents hide

1 Introduction

2 The ESP32 Sensor Network architecture

3 Necessary material

4 The gateway code

5 Testing the Gateway

6 The client nodes code

7 Testing the Sensor Network

8 Possible Improvements

9 Related tutorials

10 References

Introduction

In this tutorial we are going to learn how to build a dynamic sensor network that can be monitored through a web-based real-time dashboard. We will be using the ESP32 to implement the nodes of the sensor network and the Arduino core to program them. For exemplification purposes we will be using temperature sensors, but the architecture of the network fits other sensor types with minimal changes to the code.

One very common Internet of Things application is to build some sort of sensor network that can be monitored by a user. For beginners, it is a very interesting exercise since it allows to cover topics from the interaction between the microcontroller and the sensor to the communication between devices. Furthermore, it has a practical application that is easy to show and explain.

Nonetheless, there are also plenty of applications for these sensor networks outside the hobbyist field. One can use such solution to monitor room conditions in buildings or soil conditions in a farm, just to mention a few examples.

Naturally, there are plenty of architectures for a sensor network. From self organizing mesh networks to networks with central nodes, each one comes with its advantages and disadvantages and should be choosen accordingly to the requirements of the problem we want to solve.

For this tutorial we are not implementing any particular set of requirements for our sensor network and the main purpose is to have an example that allows to learn different concepts while building something interesting that you can later adapt to a real use case you may want to solve. As such, the architecture we will going to analyze below is just one of the many we could have chosen, but that is relatively simple to implement without sacrificing much the flexibility.

Please take in consideration that the code below is not designed to be production ready, but rather a simplified proof of concept. As such, there are many possible error situations that we won’t tackle that would need to be accounted for in a real scenario.

It’s also important to take in consideration that most of what we are going to cover on this tutorial builds on top of some other tutorials from this blog. As such, we won’t enter in detail in every part of the code, but I’ll try to include references to other tutorials. These should help in case you are not familiar with that particular concept or need a refresher. Also, the “Related Tutorials” section at the end sums up the main tutorials that serve as base for this post.

Since Arduino is basically C++ code [1], we will also use some features from this programming language that we may not see so commonly in Arduino sketches. Nonetheless, understanding that it is possible to use C++ in our code opens up a new set of possibilities and also allows us to better understand what goes under the hood in Arduino libraries.

Although we are using some of these C++ features, you don’t need to be an expert in the language to understand them, meaning that the tutorial should be easy to follow anyway.

The code was tested using both the Arduino IDE and Platformio. In both cases, the Arduino core version used was 2.0.0.

The ESP32 Sensor Network architecture

Before we jump right into the coding, it is important to understand what we want to build. The more complex is an application, the more we benefit from analyzing it from a higher level perspective. In other words, we can break down the application in different parts without having to immediately worry about the actual code we are going to write.

Since we are not solving any particular real scenario with a previous list of requirements, this section will both serve to describe the ESP32 sensor network architecture and what to expect from our application.

As such, the objective is to build an end-to-end dynamic sensor network that can be monitored through a web-based dashboard. This network will operate in a WiFi network, meaning all the devices will need to be connected to the same network and we will only be able to access the dashboard using a device (computer, smartphone, etc..) connected to the same network.

Since we want to provide a dashboard to allow the user to check the sensor measurements, some device in the network will need to be responsible to provide the corresponding application code (HTML, CSS, JavaScript..) and maintaining the values up to date. Since the ESP32 is more than capable to do so, and also to avoid a dependency in a device such as a computer, we will have a node in the network responsible to serve this dashboard (thus acting as a HTTP server).

We will call this node gateway (although the term is usually used with other meaning, we will use it in this post to refer to this special node). Figure 1 illustrates the basic interaction between the Gateway (an ESP32 acting as HTTP server) and the devices that will display the dashboard to a user on a web browser.

Naturally, when we access the dashboard on a web browser, under the hood it will do a HTTP request to the gateway to obtain the files needed to render that dashboard.

**Figure 1** – Gateway node serving the dashboard.

Additionally to serving the dashboard, the gateway node will also be responsible to receive the measurements from all the other nodes in the sensor network and push those values to the dashboard. Although this node could also have the responsibility of generating measurements itself (it has more than enough processing power to do all these tasks), we aim to have a code that is not too complex and thus this node won’t be connected to any sensor.

We will also assume that the gateway node will be running its specific code, which will be different from the code of the sensor nodes. Naturally, we could design our code to allow any node to assume the role of gateway (it would be much more resilient to failures), but again we are trying to keep things simple.

Figure 2 sumarizes the basic elements that compose the sensor network, with the sensor nodes sending the measurements to the gateway (we are not yet detailing what will be the communication protocol, but it should be easy to guess by now).

**Figure 2** – Sensor nodes connecting to the Gateway node.

We also mentioned the fact that the dashboard will be real-time. Here, we are using the term “real-time” to imply that the measurements should be pushed to the dashboard as fast as possible (real-time has a different meaning in the world of embedded systems, as described here, but many people use this term to indicate something that should be done “as fast a possible”).

For that, we are going to rely on the ESP-DASH library we have already been using in previous tutorials. You can read a detailed introduction about this library here. Under the hood, this library uses websockets to support a persistent connection between the frontend application (the dashboard) and the backend (the ESP32).

Using websockets, the ESP32 can push new measurements as soon as they are available, without the necessity of having the frontend using a polling approach, which is much more inefficient.

Figure 3 illustrates the addition of this detail of having a persistent websocket connection between the browser and the gateway. Note that this diagram doesn’t capture the time sequence of the events, but naturally the websocket connection is established only after the browser does the initial request to get the dashboard code.

**Figure 3** – Websocket connection for real-time update of the dashboard.

The sensor nodes will be other ESP32 devices, each one connected to a temperature sensor. Each node will be responsible for fetching measurements periodically and sending them to the gateway, which in turn will push each received measurement to the dashboard.

Since the gateway will already need to have a HTTP server to serve the dashboard, we can also add an endpoint to receive sensor measurements. This way, the sensor nodes can simply do a HTTP POST request every time they have a new measurement. Since it is actually the producer of the measurement that will push it to the gateway, there is no necessity for any sort of polling.

Naturally, we pay the cost of establishing a new connection every time a sensor node wants to push a notification. We could instead rely also on websockets to potentially reduce this latency, but since the number of permanent connections to a device (the gateway) is limited, it would limit the growth of the sensor nodes with little benefit. As such, the term “real-time” here is again under some restrictions within reasonable for the type of measurements we want to monitor.

Important: We are going to use HTTP (not HTTPS), which means that the messages will be exchanged over the network in plain text. Due to the nature of the application, this should not be a problem, but it is important to keep that in mind in case you plan to use the code below in an application with sensitive data.

Figure 4 summarizes the whole sensor network architecture and how the different elements interact.

**Figure 4** – Final architecture of the ESP32 sensor network.

It’s also important to mentioned that we don’t want to have a pre-defined number of sensor nodes. Instead, we want our gateway to be able to receive measurements from new nodes just added to the network. As such, whenever a the gateway receives a measurement from a node it doesn’t know yet, it will add a new card to the dashboard and keep an entry in memory to identify that node and push future measurements for the correct card. Naturally, this is done under the assumption that each sensor node has a unique identifier that is sent alongside each measurement.

Finally, we want the sensor nodes to be able to find the gateway without us having to hardcode its IP address on the source code. This is very important because, when we connect a device to a WiFi network, we don’t know beforehand which IP address will be assigned to it. Also, it can change in the future.

As such, we will use the mDNS protocol to define a domain name for the gateway that the sensor nodes can dynamically convert into an IP address. This way, we won’t the need to hardcode this IP in the source code of the sensor nodes and we won’t need additional domain name resolution infrastructure.

A couple of restrictions that we will apply to our sensor network application to keep the code from the sections below simple (however, our base architecture could be reused to solve these limitations with some improvements in the code):

We assume that nodes are not removed once registered in the gateway. As such, if a node leaves the network and we want it removed from the dashboard, we need to reset the gateway and wait for the remaining nodes to register back.
We won’t be covering most exception scenarios in the actual code (ex: JSON objects missing fields, incorrectly formatted, etc..). This is left as an exercise.
The network credentials will be hardcoded in the source code.
We support only a single sensor per client node.
If the gateway changes IP (ex: the device is shutdown and comes back with a different IP), the sensor nodes won’t do a new IP address resolution. As such, they need to be reset to be able to re-connect to the gateway.

Necessary material

In order to be able to implement the sensor network described in the previous section, we are going to need at least two ESP32 devices (one for the gateway and another to act as sensor node). Naturally, it is more interesting to test the application with multiple sensor nodes. In my case, I’ve used multiple ESP32-E FireBeetle boards and ESP32 Firebeetle boards to build the sensor network.

Additionally, we will need a temperature sensor to connect to the ESP32 client nodes. In my case, I’ve used multiple DHT22 sensor modules. If you don’t have enough sensors for all your boards but still want to do a test with many nodes, you can easily generate some fake measurements with the Arduino random function.

The gateway code

The central node of our sensor network is the gateway. Like we described in the architecture section, it will be responsible for serving the real-time dashboard and to receive the temperature measurements from all the client nodes. As such, we will start by analyzing its implementation.

As usual, we will start our code by the library includes:

WiFi.h: Allows to connect the ESP32 to a WiFi network. Both the gateway and the client nodes will need to be connected to the same network for the end-to-end system to work.
ESPAsyncWebServer.h: Allows to setup a HTTP server to run on the ESP32. This will allow the gateway to serve the dashboard frontend code and also to expose an endpoint to receive measurements from the client nodes. The library GitHub page is available here and a getting started tutorial is available here.
ESPDash.h: Exposes the classes needed to setup a real-time dashboard served by the ESP32. Intro to the library here.
json.hpp: Exposes the JSON related functionality. In the gateway node in particular, we will use it to parse incoming messages from the sensor nodes. You can check the installation instructions for the library here. Note that, although we are going to be use this library to do the JSON handling, it can be easily replaced by the ArduinoJson if you feel more comfortable with it.
map: Allows to create a map, which is an associative container (kind of a dictionary) that will allow us to store the sensor node identifiers and their corresponding Card objects.
ESPmDNS.h: Allows to set a hostname for the gateway node that the client nodes can dynamically resolve into an IP address, meaning that they don’t need to have the IP address of the gateway hardcoded in the code.

#include <WiFi.h>
#include <ESPAsyncWebServer.h>
#include <ESPDash.h>
#include <json.hpp>
#include <map>   
#include <ESPmDNS.h>

Now that we have taken care of the library includes, we will define two global variables to hold the credentials of the network: network name (SSID) and password. Below I’m going to use placeholders, which you should substitute by the actual credentials of your network.

const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPassword";

We will also set the domain name of our gateway in a global variable, so it is easy to change if we want. Note that the client sensor nodes will need to know this name, to resolve it into an IP address.

const char * host =  "gateway";

Then we will create an object of class AsyncWebServer. This will be used under the hood by the ESP-DASH library to serve the real-time dashboard and to receive websocket client connections from the frontend to update the dashboard. We will also use this instance to configure a route that will receive and process sensor measurements.

As input of the constructor, we need to pass the number of the port where the server will be listening to incoming requests. We will use port 80, which is the default HTTP port.

AsyncWebServer server(80);

After having our server instance, we will create an ESPDash object, passing as input the address of our server.

ESPDash dashboard(&server);

Finally we will define a global map that will have std::string objects as keys and Card objects as values. With this data structure, we will be able to keep track of the sensor nodes already registered (their IDs will be used as keys) and the corresponding Cards. So, whenever we receive a measurement, which will also have the associated node ID, we can check if it is already in the map. If not, we add it, along with a new Card object. Otherwise, we just update the existing card.

Naturally, we will start with an empty map, as when the program starts running no node should have been registered yet.

std::map<std::string, Card> sensorCards;

Moving on to the Arduino setup, we will open a serial connection. This is helpful for us to be able to print some content while we are debugging our program.

Serial.begin(115200);

Then, we will call a function called connectWiFi, which we will analyze later. This function takes no arguments and returns void, and it will be responsible for connecting the ESP32 to the WiFi network. Naturally, we could have had the WiFi connection code here in the Setup, like we have done in many other tutorials. Nonetheless, since this program will be a bit more complex than usual, we will encapsulate some of the initializaion precedures in functions.

connectWiFi();

The same goes for the next function we will invoke: startMdns. Like the previous one, this function takes no arguments and returns void. It will be responsible for starting mDNS and setting the gateway hostname. We will check its implementation below.

startMdns();

At this point we still need to add a route to our server to receive measurements from the sensor nodes. Although the ESP-DASH will register a route for the dashboard and set a websocket endpoint under the hood, we are still free to add new routes to our server, as long as they don’t clash with these.

So, we will create a route called “/measurement” that will answer to POST requests (note that we won’t be concerned with REST best practices for this proof of concept). To do so, we call the on method on our server object, passing as input the following parameters:

The name of the route, as a string. Like mentioned, we will use the “/measurement” route.
The HTTP method that the route accepts. We will pass the HTTP_POST constant.
The route handling function. Note however that, for POST requests, this function won’t have access to the body of the request, which we actually need. Consequently, we can implement it as a C++ lambda with an empty body. You can read more about handling body data here.
An upload handling function. We won’t need it, and in this particular case we can set it to NULL.
The body received handling function. This is where we will write the code to handle the received measurement and we will also use the C++ lambda syntax to define this function.

Note that this body handling function needs to respect the signature defined by the ArBodyHandlerFunction type (can be seen in this header file). This function should return void and receive some parameters:

A pointer to an object of class AsyncWebServerRequest. This is the same object that we have used in route handling functions covered in past tutorials to return back a response to the client. We can also use it in the body handling function to do that.
A pointer to an array of bytes, containing the body of the request.
The length of the array of bytes that contains the body of the request.
An index. Although not clearly documented, it seems to be necessary only when the payload is too big and the callback is invoked multiple times. Won’t be necessary for us.
A total size. Same observation as in the previous point.

server.on(
    "/measurement",
    HTTP_POST,
    [](AsyncWebServerRequest * request){},
    NULL,
    [](AsyncWebServerRequest * request, uint8_t *data, size_t len, size_t index, size_t total) {

      // Handling function implementation    
});

The first thing we will do in our handling function will be deserializing the JSON payload we have received. We can do that with a call to the static parse method from the JSON library we have imported at the beginning of our code. The deserialization operation was covered in detail here.

nlohmann::json obj = nlohmann::json::parse(data, data+len);

After that, we will get two fields from this json object:

id: the identifier of the sensor node, which should be a std::string.
val: the value of the measurement, which should be a float.

To access a given JSON parameter by its name, we can simply use the [] operator, passing as input a string with the name of that parameter. Then, we can simply call the get method to obtain the value for that key, converted to a compatible type. This compatible type should be specified as a template parameter.

Since we are already specifying the type of the value when calling the get method, we will leverage the auto keyword on the left side of the assignment. With the auto keyword, the compiler will infer the type of the variable for us.

auto id = obj["id"].get<std::string>();
auto val = obj["val"].get<float>();

Note that we are assuming that the string we received is correctly formatted as a JSON and both the “id” and “val” properties exist. Naturally, we are doing this for simplicity and, in a final application, you should do some error checking before accepting the values.

After this we will call a function that will check if the sensor node is already registered in our map and has a Card, or if it needs to be added. This function, which we will analyze in detail later, will be called addCardIfNotExists. This function receives as input the id of the node and check if it is already registered. If not, it will instantiate an object of class Card and add it to our map.

addCardIfNotExists(id);

Past this point, we are sure our card will exist. Consequently, we will call a function to update its value with the measurement we have received on the HTTP request. We encapsulated this logic in a function called updateCard. It receives as first input the ID of the sensor and as second the new value. We will also check its implementation later.

updateCard(id, val);

To finalize the body handling function, we will return back to the client a 200 HTTP response, indicating success.

request->send(200);

The full registration of the route is shown below.

server.on(
    "/measurement",
    HTTP_POST,
    [](AsyncWebServerRequest * request){},
    NULL,
    [](AsyncWebServerRequest * request, uint8_t *data, size_t len, size_t index, size_t total) {

      nlohmann::json obj = nlohmann::json::parse(data, data+len);

      auto id = obj["id"].get<std::string>();
      auto val = obj["val"].get<float>();

      addCardIfNotExists(id);
      updateCard(id, val);
      
      request->send(200);          
});

Focusing back in the Arduino setup, the only think left to do is calling the begin method on our server object. This will make the server start listening to incoming requests.

server.begin();

The complete setup can be seen below.

void setup() {
  
  Serial.begin(115200);
  
  connectWiFi();
  startMdns(); 
 
  server.on(
    "/measurement",
    HTTP_POST,
    [](AsyncWebServerRequest * request){},
    NULL,
    [](AsyncWebServerRequest * request, uint8_t *data, size_t len, size_t index, size_t total) {

      nlohmann::json obj = nlohmann::json::parse(data, data+len);

      auto id = obj["id"].get<std::string>();
      auto val = obj["val"].get<float>();

      addCardIfNotExists(id);
      updateCard(id, val);
      
      request->send(200);          
  });

  server.begin();
}

Since we are using the Async HTTP web server lib, we don’t need to periodically poll any object to process incoming requests. Consequently, we can leave the Arduino main loop empty.

void loop() {}

Now we will take a look at the implementation of the connectWiFi function. Like we have been doing in many previous posts, we simply need to call the begin method on the WiFi extern variable, passing as input the network name (SSID) and the password.

Following that, we will poll the connection state, with a small delay, until it is equal to WL_CONNECTED. To get this connection state, we only need to call the status method on the same WiFi extern variable.

The complete connectWiFi function is shown below. I’ve added some prints to make debugging easier. Note that we are basically entering a potentially infinite loop, waiting for the connection to be established, in case something goes wrong. We are doing this to keep the code simple and because there’s no point in trying to proceed in case we can’t connect to the network. Naturally, if you intend to use this code in a real use case, you should implement a more robust error validation and eventually have some action in case the connection attempts keep failing.

void connectWiFi(){

  WiFi.begin(ssid, password); 
 
  while (WiFi.status() != WL_CONNECTED) { 
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println("Connected to WiFi network");
}

The startMdns function will also be very simple. We will call the begin method on the MDNS extern variable, passing as input a string containing the host name of our gateway (recall that we have this value defined in a global var).

We will do this in a loop until this method call returns true, thus indicating success. Once again, this is a simplification too keep the code short and because we cannot advance in case we cannot setup mDNS properly (it will be used by the client nodes to get the address of the gateway). On a real application, make sure to do proper error checking and implement a proper strategy to deal with errors accordingly to your requirements.

void startMdns(){
  
  while(!MDNS.begin(host)) {
     Serial.println("Starting mDNS...");
     delay(1000);
  }

  Serial.println("MDNS started");
}

Then, we are going to analyze the implementation of the addCardIfNotExists method. This method returns void and, as input, it will receive a string with the identifier of the sensor node.

void addCardIfNotExists(std::string id){
    // Implementation
}

Like already briefly mentioned, this method will be responsible for checking if we have this sensor registered in our map and, if not, create a new Card object and add it. Consequently, the first thing we will do is searching in the map for the ID we receive as input.

To do so, we can use the find method. As input, this method receives the key we want to search for and, as output, it returns an iterator to the element, if an element with specified key is found, or map::end otherwise. Since we want to add the new element to the map in case it is not found, we will add a condition to check for this.

Note that it might be tempting to use the [] operator, but it actually inserts a new element with that key when it is not found [2], which is not the behaviour we are looking for.

if (sensorCards.find(id) == sensorCards.end()) {
     // Register sensor
}

To add a new Card object with the key equal to the sensor node ID, we will use the emplace method. The reason to use the emplace method requires a bit of understanding of C++.

Basically, when we use insert to add an element to a map, it will either be copied or moved [3]. Furthermore, it is when we call the constructor of the Card class that the ESP-DASH lib will send the message to the frontend, via websocket, to update the dashboard. It is also when the destructor of the Card class is called that the dashboard is updated to remove that card. Consequently, this could lead to unnecessary operations if not carefully implemented.

As an alternative, we could use the new operator to create an object that will live in the heap and doesn’t get destroyed when the object goes out of scope. Nonetheless, this would force us to work with a map of pointers and to be responsible for freeing each individual object.

Consequently, we will take advantage of the emplace method, which inserts a new element into the container constructed in-place with the given arguments [4]. If correctly used, the new element to be constructed while avoiding unnecessary copy or move operations [4].

We will pass the following parameters to the emplace method:

The piecewise_construct constant. Since the reason why this constant was introduced is a bit complex, it is outside the scope of this post to enter in detail. If you are interested in knowing more about it, you can read this document and this StackOverflow question.
A tuple of arguments to build the key of the map (which is an std::string), built with the forward_as_tuple function.
A tuple of arguments to build the value of the map for the given key (which is a Card object). This should also be built with the forward_as_tuple function.

If you are curious about the different ways of building a tuple, please check this article.

Analyzing in the concrete values of both the key and the object to be stored, we will use the sensor ID passed as argument of our addCardIfNotExists function as key, and the following values to create a new object of class Card:

The address of our dashboard object.
The enum value TEMPERATURE_CARD, which indicates that our card will have temperature measurements (and in the frontend, the corresponding card will have an icon matching it, which can be seen here).
The name of the card that will be displayed in the frontend dashboard. We will use the identifier of the sensor.
The unit of the measurements. We are going to work with degrees Celsius.

sensorCards.emplace(
      std::piecewise_construct,
      std::forward_as_tuple(id), 
      std::forward_as_tuple(&dashboard, TEMPERATURE_CARD, id.c_str(), "°C"));

The complete function is shown below.

void addCardIfNotExists(std::string id){
  
  if (sensorCards.find(id) == sensorCards.end()) {

    sensorCards.emplace(
      std::piecewise_construct,
      std::forward_as_tuple(id), 
      std::forward_as_tuple(&dashboard, TEMPERATURE_CARD, id.c_str(), "°C"));
  }
}

To finalize, we will analyse the implementation of the updateCard function, which receives as input the identifier of the sensor node and the new measurement.

void updateCard(std::string id, float val){
  // Implementation
}

The first thing we will do is accessing the Card for the given ID from the map structure. Note that, at this stage, and assuming that this function is called after the addCardIfNotExists one, we should always have an entry for the card.

We will use the at method, which receives as input the key of the map we want to look for and returns a reference to the mapped value of the element with that key [5]. Note that if the passed key doesn’t match the key of any element in the container, the function throws an out_of_range exception [5] (shouldn’t happen in our use case, but in a real application we should be able to handle these exceptions).

Once again, it could be tempting to use the [] operator, given the fact that we should always find an element at this stage, and no new one should end up being inserted. Nonetheless, this operator uses the default parameterless constructor to insert an entry when the key is not found. In our case, the Card class doesn’t have such constructor, meaning that we would run into compilation errors if we tried to use this operator.

After calling the at method we should have access to the Card object for the key we used. After that, we simply need to call the update method on the Card object, passing as input the measurement.

sensorCards.at(id).update(val);

Note however that this update method call won’t immediately send the new value to the frontend. Consequently, we also need to call the sendUpdates method on our dashboard object. This final call will be responsible for actually sending the values to the client, through a websocket connection.

dashboard.sendUpdates();

The complete function is available in the snippet below.

void updateCard(std::string id, float val){
  sensorCards.at(id).update(val);
  dashboard.sendUpdates();
}

The whole code is available below.

#include <WiFi.h>
#include <ESPAsyncWebServer.h>
#include <ESPDash.h>
#include <json.hpp>
#include <map>   
#include <ESPmDNS.h>

const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPassword";

const char * host =  "gateway";

AsyncWebServer server(80);
ESPDash dashboard(&server); 

std::map<std::string, Card> sensorCards;

void connectWiFi(){

  WiFi.begin(ssid, password); 
 
  while (WiFi.status() != WL_CONNECTED) { 
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println("Connected to WiFi network");
}

void startMdns(){
  
  while(!MDNS.begin(host)) {
     Serial.println("Starting mDNS...");
     delay(1000);
  }

  Serial.println("MDNS started");
}

void addCardIfNotExists(std::string id){
  
  if (sensorCards.find(id) == sensorCards.end()) {

    sensorCards.emplace(
      std::piecewise_construct,
      std::forward_as_tuple(id), 
      std::forward_as_tuple(&dashboard, TEMPERATURE_CARD, id.c_str(), "°C"));
  }
}

void updateCard(std::string id, float val){
  sensorCards.at(id).update(val);
  dashboard.sendUpdates();
}

void setup() {
  
  Serial.begin(115200);
  
  connectWiFi();
  startMdns(); 
 
  server.on(
    "/measurement",
    HTTP_POST,
    [](AsyncWebServerRequest * request){},
    NULL,
    [](AsyncWebServerRequest * request, uint8_t *data, size_t len, size_t index, size_t total) {

      nlohmann::json obj = nlohmann::json::parse(data, data+len);

      auto id = obj["id"].get<std::string>();
      auto val = obj["val"].get<float>();

      addCardIfNotExists(id);
      updateCard(id, val);
      
      request->send(200);          
  });

  server.begin();
}
 
void loop() {}

Testing the Gateway

We can easily test the gateway node isolated, without having to code the sensor client nodes. It is important to test things in isolation because it is much harder to track the origin of a bug in the end-to-end sensor network.

The first thing to do is the usual: compiling and uploading the code to the ESP32 device that will act as gateway node. Once the procedure is finished, we can open the Arduino IDE serial monitor (or any other serial tool) and confirm that all the initialization steps happen as expected: the device can connect to the WiFi network and initialize the mDNS interface.

Then, we can simply open a web browser of our choice and try to access the dashboard. To do so, simply put the following in your browser’s address bar and click enter:

http://gateway.local/

If your operating system supports mDNS resolution, you should see a web page with the dashboard (which at this stage should not show any card, as no node is connected yet). If your operating system doesn’t support mDNS, you have one of two options:

Add support for mDNS. Last time I’ve encountered this issue was while using Windows 8 and I had to install Apple’s Bounjour.
Print the local IP address of the gateway to the serial port and use it directly. The ESP32 clients should still be able to do the resolution without this hardcoded value because, as we will see in the next section, we will use the Arduino core mDNS library to perform the resolution.

In case you need to use the second approach, keep in mind that the IP address assigned to the gateway may change over time, thus making this a not so convenient solution. Anyway, you should be able to get the IP address by adding the following line to the connectWiFi function, at the end:

Serial.println(WiFi.localIP());

So, you would have something like:

void connectWiFi(){

  WiFi.begin(ssid, password); 
 
  while (WiFi.status() != WL_CONNECTED) { 
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println("Connected to WiFi network");
  Serial.println(WiFi.localIP());
}

In that case, when accessing the dashboard from the web browser, you should access the following URL instead, changing #yourDeviceIP# by the IP that will get printed to the serial tool, after the ESP32 connects to the network:

http://#yourDeviceIP#/

Either ways, you should be able to access the dashboard, like illustrated below in figure 5. Note that the loading icon will remain while there is no card to show to the user.

**Figure 5** – Dashboard without any card.

Now that we have our dashboard up and running, we can easily simulate a request in the format that is expected to be done by an ESP32 client node. To do so, one of the easiest ways is to use a tool such as Postman, which allows us to perform HTTP requests without the need to code. Naturally, you can use other tools, such as cURL or even write a simply script to test.

Assuming that we go with Postman, we simply need to choose a POST request from the method dropdown. For the URL, we should use the endpoint we defined in our server (“/measurement“):

http://gateway.local/measurement

Note that above I’m assuming that the machine where Postman is running is able to perform the mDNS address resolution. If not, you need to use the IP address instead, similarly to what was explained for reaching the dashboard.

On the body tab we should choose the “raw” option from the radio buttons and on the right side dropdown we should choose “JSON“. Then, we simply need to put a valid JSON in the text editor that appears, which will contain the body to be sent.

If you want a more detailed guide on how to do a POST request with Postman, please check this video (the Postman version used is a bit old, but the steps are the same in the new versions of the tool).

A valid JSON to test can be the following:

{
    "id": "kitchen",
    "val": 10.7
}

Note that I’m using “kitchen” as ID in the example below to illustrate a possible use case where the client sensor nodes map to divisions of a house. Our gateway makes no assumptions about the format of the ID, meaning that this value should be correctly processed. Nonetheless, in the client nodes code, we will assume the usage of the device MAC address as unique identifier.

If you have the request correctly configured in Postman, it should look like figure 6.

**Figure 6** – Testing the gateway node with Postman.

After sending the request, you should receive a 200 HTTP status code response and you should see a card called “kitchen” added to the dashboard, with the value defined in the “val” property.

If you send more requests with the same ID but with different values, you should also see the dashboard card value getting updated. If you change to a new ID, the first request you do for that ID should add a new card to the dashboard.

Figure 7 illustrates the dashboard with some cards created from Postman requests.

**Figure 7** – Sensor network dashboard, with measurements sent from Postman.

After performing these tests with success, we should be more confident that the end-to-end sensor network will work after we connect the nodes. Naturally, if you experience other issues in the gateway not described here, the best approach is to debug the code also in isolation.

The client nodes code

Now that we have analyzed the implementation of the gateway node and also covered how to perform some basic tests in isolation, we will check the code for the sensor nodes, which are going to compose our sensor network.

Like before, the first part of our code will cover the library includes.

WiFi.h: Allows to connect the ESP32 to a WiFi network. Both the clients and the gateway will be connected to the same network.
HTTPClient.h: Exposes the class needed to perform HTTP requests from the ESP32. We will use it to send the measurements to the gateway.
json.hpp: Exposes the JSON related functionality needed to build the JSON messages containing the sensor data.
ESPmDNS.h: Exposes the method necessary to perform the resolution of an address using mDNS, which we will need to obtain the IP address of the gateway.
Ticker.h: Allows to setup a callback function to run periodically. We will use this functionality to setup the periodic execution of a function that will produce the measurements and send them to the gateway. We could implement something similar by adding some delays to the Arduino main loop, but using the Ticker.h classes tends to lead to cleaner code.
DHTesp.h: Exposes the class we need to get temperature measurements from the DHT22 sensor without having to worry about the low level details of the communication.

#include <WiFi.h>
#include <HTTPClient.h>
#include <json.hpp>
#include <ESPmDNS.h>
#include <Ticker.h>
#include <DHTesp.h>

After that we will define two global variables that will hold the credentials of the WiFi network: SSID and password. Please take in consideration that I’m using placeholders below that you should replace by the actual credentials of your network.

const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPass";

Then we will define a string containing the host name for the gateway. This should contain the exact same value that we used in the gateway code. We will also create an object of class IPAddress which will later hold the IP address of the gateway, after we perform the name resolution with mDNS.

const char * host =  "gateway";
IPAddress serverIp;

We will also create an object of class Ticker, which we will use to setup the periodic execution of the function that will obtain a sensor measurement and send it to the gateway. To keep configurations easy to change, we will also define the interval between executions of this function in a global variable. We will set it to run every 20000 milliseconds (20 seconds).

Ticker periodicTicker;
const int measurementIntervalMs = 20000;

To finalize our global variable definitions, we will create an object of class DHTesp, which will later expose to us the method we need to obtain a temperature measurement. We will also set the number of the ESP32 pin that is connected to the sensor in a global var. In my case, I’ll be using pin 17, but you can use other pins. Just don’t forget to make sure that all the client ESP32 nodes have the sensor connected to the same pin or to change the code to the adequate pin number before uploading the program.

const int sensorPin = 17;
DHTesp dht;

The setup function will be very short, as we will encapsulate most of the initializations in functions. Naturally, if you prefer, you can perform these initializations in the Arduino setup like we have done in many other posts.

As usual, we will start by opening a serial connection. For debugging purposes, it will help to have the possibility of getting some outputs in a serial tool in case we experience some issue.

Serial.begin(115200);

To initialize the DHT22 sensor interface, we will call the setup method on our previously created DHTesp object. As first input we pass the number of the ESP32 pin connected to the sensor and as second we pass an enum indicating what is the sensor model. For our use case, the enum value should be DHT22. Note below the usage of the C++ scope resolution operator to use the value of the enumeration we want.

dht.setup(sensorPin, DHTesp::DHT22);

After that we will take care of connecting the ESP32 to the WiFi network, using the previously defined credentials (SSID and password). We will do so by calling a function named connectWiFi. It will be basically the same function we have used in the code of the gateway node.

connectWiFi();

Then, to perform the resolution of the gateway host name into an IP address, we will also use a function that we will define later. It will be called resolveHostAddress and, like the previous, it will take no arguments and return void.

resolveHostAddress();

Finally, we will use our Ticker object to setup the function that will get the measurements to execute periodically. We do so with a call to the attach_ms method, passing as first input the interval between executions of the function, in milliseconds, and as second input the actual callback function to be executed.

Our callback will be called postMeasurement and we will define it later. The interval between executions of the function was already defined in the global variable called measurementIntervalMs. As such, we will use this variable.

periodicTicker.attach_ms(measurementIntervalMs, postMeasurement);

The full setup is available below.

void setup() {
 
  Serial.begin(115200);
 
  dht.setup(sensorPin, DHTesp::DHT22);

  connectWiFi();
  resolveHostAddress();

  periodicTicker.attach_ms(measurementIntervalMs, postMeasurement);

}

Since we are using the Ticker.h lib to define the periodic execution of the function to obtain the measurements, we can leave the Arduino main loop empty.

void loop() {}

To end our code, we need to check the implementation of the 3 functions we have mentioned when analyzing the Arduino setup code. The first one will be the simplest, and it will basically consist on connecting the ESP32 to the WiFi network. Since it is the same we have already seen in the gateway code, we won’t cover it in detail again.

void connectWiFi(){

  WiFi.begin(ssid, password); 
 
  while (WiFi.status() != WL_CONNECTED) { 
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println("Connected to WiFi network");
}

The resolveHostAddress function shares some similarities with the previous function, as it will also try indefinitely to resolve the gateway address. Once again, this is done for simplicity and should be implemented in a more robust way in a real application.

The first thing we will do is calling the mdns_init function from IDF, which takes care of initializing mDNS. Note that the Arduino core exposes the begin method on its MDNS extern variable (the one we have used on the gateway), but it forces us to pass a hostname, as can be seen here. In our particular case we don’t need the client to have a host name but rather to resolve the host name of the gateway. As such, we use the mdns_init function to avoid having to provide one.

We will do this function call in a loop until the value is equal to ESP_OK, thus indicating success.

while(mdns_init()!= ESP_OK){
    delay(1000);
    Serial.println("Starting MDNS...");
}

Once mDNS is initialized, we are able to perform the gateway host name resolution. This time, we will take advantage of the MDNS extern variable and its queryHost method (the process was covered in detail here). This method receives as input a string with the host name we want to resolve and returns as output the corresponding IP address, as an IPAddress object.

Note however that, if the resolution fails, it will return an IP address with the value “0.0.0.0“. As such, we will do this resolution in a loop, until we get a value different from that. We can leverage the toString method of the IPAddress class to obtain its string representation and perform the mentioned comparison.

while (serverIp.toString() == "0.0.0.0") {
    Serial.println("Resolving host...");
    delay(250);
    serverIp = MDNS.queryHost(host);
 }

Once again, doing an infinite loop is not the perfect solution. Nonetheless, for a very simplistic implementation, it offers us the guarantee that, when a client starts and there is no gateway available, it will keep trying the connection until we have an answer from a gateway node. Naturally, the approach we have done above is not very well optimized, as we are basically doing a poll every 250 ms until we have success.

Take in consideration that the resolved address will be assigned to our IPAddress global variable, so we can also use it on the function that will send the measurements to the gateway.

The complete code for the function is shown below.

void resolveHostAddress(){
  
  while(mdns_init()!= ESP_OK){
    delay(1000);
    Serial.println("Starting MDNS...");
  }

  Serial.println("MDNS started");

  while (serverIp.toString() == "0.0.0.0") {
    Serial.println("Resolving host...");
    delay(250);
    serverIp = MDNS.queryHost(host);
  }

  Serial.println("Gateway address resolved");
}

The last function we need to check is the postMeasurement, which will run periodically to send the measurements to the gateway. Once we reach this point, we are assuming that the ESP32 is already connected to the WiFi network and the IP address of the gateway was already obtained and that node is up and running.

Naturally, this is another simplification and, in a real case scenario, these assumptions are too strong, and one would need to check if the client node is still connected to the WiFi network and have some recovery routine if the gateway node is no longer reachable.

void postMeasurement() {
     // Get Measurement
     // Build JSON message
     // Do POST request
}

The first thing we will do is obtaining a temperature measurement from our DHT22 sensor. Once again, we will use the DHTesp object. We simply need to call the getTemperature method, which takes no arguments and returns a float with the temperature.

float temp = dht.getTemperature();

Then we will create a json object, which will contain all the fields of data that we want to send to the gateway.

nlohmann::json measurement;

Now that we have our object, we will set a property called “id“, which corresponds to the ID of our node. To avoid having to change the code per each client node we want to test, we will be assigning to this ID the mac address of the ESP32, which should be unique. We can obtain the MAC address with a call to the macAddress method on our WiFi extern variable (a detailed tutorial can be followed here).

Note that this method will return as output an Arduino String object, which the nlohmann/json lib doesn’t know how to convert to a string in its internal representation (recall that it is a C++ library, which was not designed to work with the Arduino specific classes). Fortunately we can call the c_str method on an Arduino String to obtain its C-style version, which the nlohmann/json lib knows how to handle.

It’s important to mention that we are using the MAC address for simplicity when testing, but for a final user it wouldn’t be very nice to see a MAC in the cards of the dashboard. Nonetheless, if you want to give custom names to each node (ex: “kitchen”, for a node deployed in this division of the house), it is trivial to create a global variable at the top of the code and assign it to the ID property of the JSON. This makes it much simpler to change the values if we are uploading the code to different devices.

Alternatively, we can have a startup section that dynamically reads this value from a configuration file that we upload to the file system (to keep the code exactly the same and have this configuration based) or, in a more sophisticated approach, have the client nodes themselves hosting a simple configuration portal where the user can input the node name he wants.

We are not going to cover these approaches here, but they could be interesting follow up exercises for this tutorial.

measurement["id"] = WiFi.macAddress().c_str();

We are going to assign another property, called “val“, which basically corresponds to the measurement we previously obtained.

measurement["val"] = temp;

Now that we have our json object, we will serialize it to a string, to be able to send the data as body of a HTTP POST request. To do so, we simply need to call the dump method on our json object.

Since our objective is to send this to another device, we will use the parameterless version of the dump method, which will serialize the object to the most compact string representation. If by some reason we wanted to get the JSON in a pretty printed format (ex: to debug what is being sent to the gateway), we could simply pass a number as input specifying the indentation level. If you want a more detailed explanation on the serialization procedure, please consult this post.

std::string serializedObject = measurement.dump();

To send the actual request, we will create an object of class HTTPClient.

HTTPClient http;

Then, to initialize the request, we will call the begin method on the HTTPClient object, passing as input the URL of the endpoint we want to reach. Recall that we have the actual IP address of the gateway in a global IPAddress object, meaning that we can easily build the final URL with some strings concatenation.

Since we are going to send a HTTP request, the URL starts with the string “http”, the colon and two forward slashes. Followed by that, we need to add the IP address of the gateway, which we can obtain, as a string, by calling the toString method on the global IPAddress object. At the end, we need to append the route we want to reach, which is “/measurement“.

Note that in the snippet below we will have the route hardcoded as a string, but we could have also defined a global variable for it instead.

http.begin("http://" + serverIp.toString() + "/measurement");

In order for the gateway to know that we are sending it a JSON payload, we will call the addHeader method on our HTTPClient object, to set the “content-type” header to “application/json“.

http.addHeader("Content-Type", "application/json");

To perform the actual request, we will call the POST method on our object, passing as input the serialized string we haver obtained before. In this case, it is the Arduino lib that doesn’t know how to handle a std::string, which is why we will call the c_str method before passing our string to the POST method.

As output this method will return the HTTP status code, which we will store in a variable.

int httpResponseCode = http.POST(serializedObject.c_str());

If the status code obtained is lesser that zero, then an error occurred on the connection. If it is greater than zero, then it’s a standard HTTP code. For simplicity, we will only print the code to the serial port, but in a real use case we could use this to take different actions. Naturally, in a success scenario, we expect our gateway to return the 200 HTTP status code.

Serial.println(httpResponseCode);

To finish our function, we will call the end mehtod on our HTTPClient object, to free the resources.

http.end();

The complete function is available below.

void postMeasurement() {
  
  float temp = dht.getTemperature();

  nlohmann::json measurement;  
  measurement["id"] = WiFi.macAddress().c_str();
  measurement["val"] = temp;

  std::string serializedObject = measurement.dump();

  HTTPClient http;   
 
  http.begin("http://" + serverIp.toString() + "/measurement");
  http.addHeader("Content-Type", "application/json");            
 
  int httpResponseCode = http.POST(serializedObject.c_str());
 
  Serial.println(httpResponseCode);

  http.end();
}

The full code for the client can be seen below.

#include <WiFi.h>
#include <HTTPClient.h>
#include <json.hpp>
#include <ESPmDNS.h>
#include <Ticker.h>
#include <DHTesp.h>
 
const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPass";
 
const char * host =  "gateway";
IPAddress serverIp;

Ticker periodicTicker;
const int measurementIntervalMs = 20000;

const int sensorPin = 17;
DHTesp dht;

void connectWiFi(){

  WiFi.begin(ssid, password); 
 
  while (WiFi.status() != WL_CONNECTED) { 
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }

  Serial.println("Connected to WiFi network");
}

void resolveHostAddress(){
  
  while(mdns_init()!= ESP_OK){
    delay(1000);
    Serial.println("Starting MDNS...");
  }

  Serial.println("MDNS started");

  while (serverIp.toString() == "0.0.0.0") {
    Serial.println("Resolving host...");
    delay(250);
    serverIp = MDNS.queryHost(host);
  }

  Serial.println("Gateway address resolved");
}

void postMeasurement() {
  
  float temp = dht.getTemperature();

  nlohmann::json measurement;  
  measurement["id"] = WiFi.macAddress().c_str();
  measurement["val"] = temp;

  std::string serializedObject = measurement.dump();

  HTTPClient http;   
 
  http.begin("http://" + serverIp.toString() + "/measurement");
  http.addHeader("Content-Type", "application/json");            
 
  int httpResponseCode = http.POST(serializedObject.c_str());
 
  Serial.println(httpResponseCode);

  http.end();
}

void setup() {
 
  Serial.begin(115200);
 
  dht.setup(sensorPin, DHTesp::DHT22);

  connectWiFi();
  resolveHostAddress();

  periodicTicker.attach_ms(measurementIntervalMs, postMeasurement);

}
 
void loop() {}

Testing the Sensor Network

Moving on to the end-to-end sensor network, first make sure to have an ESP32 acting as gateway already up and running in your WiFi network. Then, simply compile and upload the sensor node code to an ESP32 and open a serial tool to check the outputs.

If everything goes well, you should see all the success messages from the initialization part of the sensor node (WiFi connection, initialization of mDNS and address resolution). After that, you should see the HTTP code 200 getting printed periodically in the console, indicating that the client was able to deliver measurements to the gateway.

If you consult the dashboard, you should see a card for this ESP32, with a title equal to the MAC address of the device. You should also see the periodic updates of the value in the card.

Note that if you connect a client node without having a gateway, it should stay in the loop trying to resolve the address without success. If then you connect the gateway, the node should be able to do the resolution and keep functional.

If after the device is publishing messages the gateway goes down, you should no longer see the 200 status code but rather an error trying to connect to the device. If you get the same gateway back online and your router assigns the same IP address to it, the client node should be able to start sending measurements with success again.

Naturally, if you use other device or the IP address of the same gateway changes, the already running nodes won’t be able to find that new IP address since our client code only does the address resolution once, at the beginning of the program. Fixing this could be an interesting exercise to extend the functionality of the system and make it more robust to failures. For our implementation, you can simply reset that device.

Assuming that everything goes well with a single client node, you should then be able to flash more ESP32 devices and see the corresponding cards appearing in the dashboard. They should then send the measurements periodically.

Figure 8 shows the dashboard for my sensor network.

**Figure 8** – Dashboard of the dynamic sensor network, with real nodes.

Possible Improvements

Now that we have a pretty good overview of the basic code to have our sensor network up and running, we dedicate this section to comment on some interesting improvements that could be done to extend the functionality:

Supporting cleanup of sensor nodes in the gateway. Like we have seen, we keep nodes indefinitely once they register on the gateway. Nonetheless, we could easily define a maximum time interval to cleanup a given node in case we don’t receive a new measurement from it. We could simply store a timestamp from the last received measurement and run a routine periodically to cleanup all the nodes that pass the defined threshold.
Supporting sensors other than temperature. We could eventually introduce an enum if the types of sensors are known beforehand or even come up with a more dynamic scheme where the type of sensor can be communicated by the sensor node.
Supporting multiple sensors per node. Instead of having a 1 to 1 mapping between a card and a sensor ID, we could instead have an array of cards.
Allowing each node to assume the role of a gateway. Instead of having a code for the gateway and another for the sensor node, all nodes could behave in such a way that, if they detect that there is no gateway in the WiFi network, they assume that role. If there is already one, then they act as sensor nodes.
Forwarding sensor measurements to a remote server, so they can be consulted by someone outside the WiFi network. Although this wouldn’t probably work with the same “real-time” approach, the gateway could easily forward measurements to be processed by a remote server.
Supporting AP mode. If a WiFi network is not available, the gateway could also host its own WiFi network, to which the other nodes could connect to. Also, a computer or smartphone could then connect to that network and access the dashboard.
Adding a page to gather the WiFi network credentials. Any node could host its own network and serve a simple portal to gather WiFi network credentials, store them in persistent memory and use them for future connections.

The list could go on as there are many interesting features that could be added to this sensor network application. The main point is to highlight that it can be used as base to other applications that you may be looking to build.

Also, and very important, in a real case scenario you should handle all the possible error situations that we didn’t in our code.