ESP32: JSON

Introduction

In this tutorial we are going to learn how to use the Nlohmann/json library on the ESP32, using the Arduino core.

Although this is a C++ library, we can use it easily as an Arduino library because, in fact, Arduino code is basically C++ with some abstractions [1][2], which means that we are able to use many C++ libraries out there (not all, depending on standard libraries that may not be available or some other details).

Nonetheless, the syntax for this library is very intuitive, as we will see in the code sections below, so you should be able to use it easily without having a deep knowledge of C++.

In this tutorial we are going to cover the basics of the library (building an object, serializing, deserializing) and some additional functions it offers, such as flattening a JSON. Nonetheless, the library offers many more features, so I encourage you to read the documentation to better understand its capabilities. Some examples of features it supports are converting a JSON object to MessagePack or BSON.

The version of the Nlohmann/json library used on this tutorial was 3.10.2. It’s also worth mentioning that the single include version was used (more on this on the section “Installing the library“). The Arduino core version used was 2.0.0 and the Arduino IDE version was 1.8.15, running on Windows 8.1.

The tests shown below were performed on a ESP32-E FireBeetle board from DFRobot.

A quick intro on JSON

For those who are new to the world of programming or simply never heard about JSON, this sections presents a quick intro about it.

JSON (JavaScript Object Notation) is a lightweight data-interchange format that is very easy for humans to write and read and is also simple for machines to parse and generate [3].

It is programming language independent [3] (don’t get mislead by the JavaScript in the acronym) meaning that it can be used in a huge variety of languages. You can check here some examples of libraries and languages supported.

Being language agnostic, widely supported and easy to use, the applications of JSON range from storing data to being used in communication protocols, amongst many others.

In the scope of the ESP32 programming and, more broadly, the Internet of Things, JSON is a very good choice for lightweight data exchange format for protocols such as HTTP or WebSockets (ex: to send sensor information, configurations or operation logs). This is why learning how to work with it can be very useful for a wide range of applications.

You can also read here a very interesting between JSON and XML, also a very well known data exchange format.

About the Library

Nowadays there are plenty of libraries to work in JSON, in different programming languages. C++ is no exception, and we can also find a big list of possibilities.

One of the catch phrases that we can see in the Nlohmann/json library GitHub page is that the library offers “JSON for Modern C++” [4], which I believe is a very accurate sentence. If you ever worked with JSON libs in higher level languages (ex: json.net for C#) or with objects in JavaScript, then you will most likely feel that the syntax is very intuitive, at least for the more basic operations.

Other interesting aspect about this library is that it is available as a single header file [4], like we briefly mentioned in the introductory section. This means that “installing” it as an Arduino library is quite trivial, as we will see below.

It is also mentioned in the GitHub page that the library is heavily unit tested with a coverage of 100% [4]. This should make it a very reliable choice for real scenario applications, which is very important for something as core as handling the JSON related functionality of an application.

Since there are many C++ JSON libraries available, it is interesting to understand how Nlohmann/json compares against others. You can check a generic benchmark of JSON libs here, being Nlohmann/json among the ones tested.

Installing the Library

Since Nlohmann/json is not an Arduino library, we won’t be able to install it using the Arduino IDE libraries manager. At the time of writing there is also a clash between a define in the library of the Arduino core and the Nlohmann/json library. Nonetheless, the installation procedure will still be quite simple and detailed on this section.

The first thing we need to do is locating the Arduino IDE libraries folder. This may vary depending on your installation, but mine are located at C:\Users\MyUserName\Documents\Arduino\libraries.

There, create a folder called nlohmann. Once you have the folder, simply download the single header version of the library and place it in the folder you just created. You can access the single file header here. To learn more about single header libraries, please check here.

Please take in consideration that the extension of the header file is .hpp and you should keep it. It will also be the extension that we will use later in our Arduino code to include the library. You can read more about the difference between a .hpp and a .h file on this Stack Overflow thread.

After finishing the previous steps, you should already be able to include the library from an Arduino program by adding the following include:

#include <json.hpp>

Nonetheless, if you add this include and try to compile the Arduino program without any additional changes, you will receive compilation errors such as the ones shown in figure 1 below.

Basically, there is a a constant called “B1” in a header file called binary.h from the Arduino core that conflicts with a declaration done in the JSON library.

The block of code for that declaration, in the Nlohmann/json single header file, is highlighted below in figure 2. As can be seen, there are also references to “B1“, which cause the previously mentioned compilation errors.

As a quick fix, we will simply apply some renames in the previous 5 lines of code. So, you should replace those lines by the ones shown in the snippet below. Please take in consideration that, depending on the version of the Nlohmann/json library that you are using, these declarations may be located in different lines. If so, simply search by “B1”.

template<class...> struct conjunction : std::true_type { };
template<class C1> struct conjunction<C1> : C1 { };
template<class C1, class... Cn>
struct conjunction<C1, Cn...>
: std::conditional<bool(C1::value), conjunction<Cn...>, C1>::type {};

After applying this change, you should end up with something similar to figure 3.

Once you apply the fix and save the file, you should be able to compile Arduino programs with the library include.

If instead of the error mentioned above you run into something related with “to_string is not a member of std“, it means that you are using an older version of the Arduino core. I’ve faced that error in the past trying to use the library and was only able to solve it by updating the Arduino core version to 2.0.0, which is the one I’ve used while doing the tutorial.

A simple example

In this section we will start by a very simple example where we will introduce the json specialization, which is a specialization of the basic_json class template. To put things simple, we can think of it as the structure that we will use to store the fields of our JSON object, amongst many other functionalities it supports.

Naturally, the first thing we will do is including the library.

#include <json.hpp>

Followed by that, and as a convenience to make our code shorter, we will declare the usage of the json specialization from the nlohmann namespace.

using json = nlohmann::json;

We will write the rest of our code in the Arduino setup. The first thing we will do is opening a serial connection, so we can output the results of our program.

Serial.begin(115200);

After that we will create a json object. You can check the constructors of the class here but, in our case, we are going to create a null JSON value that we will populate below.

json obj;

After that we can start using the [] operator to add fields to our JSON. The syntax is quite simple:

obj["fieldName"] = value;

Note that for types such as strings, integers and Booleans, for example, we don’t need to explicit tell the compiler what is the JSON value type to be used, making it even simpler to use. We will add some fields to our JSON:

obj["int"] = 10;
obj["bool"] = true;
obj["float"] = 3.14;
obj["str"] = "Hello World";

In the case of the string, it will be stored as a std::string.

Now that our json object has the new fields, we will simply access them again using the [] operator, putting inside the square brackets the name of the key we want to obtain. This will return a reference to the element at the given key [5].

After that we can simply call the get method to obtain the value for that key, converted to a compatible type [6]. This compatible type should be specified as a template parameter. In other words, if for example we want to obtain the integer we have assigned before, we should use int as template parameter.

Below we can se how to get the values we stored for each data type.

Serial.println(obj["int"].get<int>());
Serial.println(obj["bool"].get<bool>());
Serial.println(obj["float"].get<float>());
Serial.println(obj["str"].get<std::string>().c_str());

Note that in the particular case of the std::string, before printing the result, we had to call the c_str method to obtain a pointer to a char array representing the current value of the string. This is necessary because none of the versions of the println method accepts a std::string as parameter.

The complete code is shown below.

#include <json.hpp>
using json = nlohmann::json;

void setup()
{
  Serial.begin(115200);

  json obj;

  obj["int"] = 10;
  obj["bool"] = true;
  obj["float"] = 3.14;
  obj["str"] = "Hello World";

  Serial.println(obj["int"].get<int>());
  Serial.println(obj["bool"].get<bool>());
  Serial.println(obj["float"].get<float>());
  Serial.println(obj["str"].get<std::string>().c_str());
}

void loop()
{
}

To test the code, simply compile it and upload it to your ESP32 using the Arduino IDE. When the process finishes, open your IDE serial monitor. As we can see in figure 4 below, we obtain the 4 fields we have assigned to our json object.

Nested JSON Objects and Arrays

Besides primitive types, like we have seen above, a JSON object can also hold nested objects and arrays. Naturally, the Nlohmann/json library also supports these and the syntax is equally simple. We are going to cover it on this section.

We will move right to the Arduino setup, since the include and the using is the same as the previous example. In the setup, we will start by opening a serial connection and creating a json object.

Serial.begin(115200);

json obj;

The first thing we will do is creating a property, called “address”, that has two string properties: a street name and a street code. To do it, we can use the exact same [] operator as before. Basically, we just need to add another pair of square brackets per each depth level. The code below should illustrate this syntax better:

obj["address"]["street"] = "John's Street";
obj["address"]["code"] = "2700-12";

Adding an array is equally simple. We still use the [] operator to define the property and we can assign an array directly. In our case, we are going to assign a list of languages. Internally, the array will be stored as a std::vector [4].

obj["languages"] = {"pt-pt", "en-us"};

To access these properties, we use exactly the same sequence of square brackets we used before, plus a call to the get method we already learned, to specify the data type.

I’ll be using the auto keyword on the left side of the assignment, so the compiler infers the data type of the variables. I’m doing this just for illustration purposes and to have a shorter code but, if you prefer, you can still explicitly declare the type of each variable.

auto street = obj["address"]["street"].get<std::string>();
auto code = obj["address"]["code"].get<std::string>();

auto languages = obj["languages"].get<std::vector<std::string>>();

We will then print each of the obtained values. In the case of the languages std::vector we will iterate through each element and print it.

Serial.println(street.c_str());
Serial.println(code.c_str());

for(int i=0; i<languages.size(); ++i){
  Serial.println(languages[i].c_str());
}  

The full code can be seen below.

#include <json.hpp>
using json = nlohmann::json;

void setup()
{
  Serial.begin(115200);

  json obj;

  obj["address"]["street"] = "John's Street";
  obj["address"]["code"] = "2700-12";

  obj["languages"] = {"pt-pt", "en-us"};

  auto street = obj["address"]["street"].get<std::string>();
  auto code = obj["address"]["code"].get<std::string>();

  auto languages = obj["languages"].get<std::vector<std::string>>();

  Serial.println(street.c_str());
  Serial.println(code.c_str());

  Serial.println("\nList:");
  for(int i=0; i<languages.size(); ++i){
    Serial.println(languages[i].c_str());
  }  
}

void loop()
{
}

Upon running the previous code on your device, you should obtain a result similar to figure 5. We obtained the fields of our nested object and the elements of our array, as expected.

JSON Serialization

Now that we have learned the basics of the json class and how to create a JSON object, we will learn how to perform its serialization.

As before, we start by the library include and the using.

#include <json.hpp>
using json = nlohmann::json;

Moving on to the Arduino setup, we will open a serial connection to later output the serialized object. After this we will create a json object and add some fields to it. We will create a structure representing a person with an age, a name and an address, for illustration purposes.

Serial.begin(115200);

json obj;

obj["age"]=10;
obj["name"] = "John";
obj["address"]["street"] = "St. Street";
obj["address"]["code"] = "2790-124";

Now that we have our JSON structure, we simply need to call the dump method on our json object. If we pass no arguments, we will obtain a compact serialized string without newlines. This is the optimal format to be used if we are going to send this string to another system (ex: in the body of a HTTP request).

Note that this method returns a std::string. So, like we already mentioned before, we cannot directly print it to the serial port. As such, we need to call the c_str method before.

std::string serializedObject = obj.dump();
Serial.println(serializedObject.c_str());

If instead of a compact string we prefer a more human readable format (for example, for debugging purposes), we can pass an integer as input of the dump method. The object members will then be printed with that indent level [7].

Note that an indent level of 0 will only insert newlines and -1 (the default parameter level) selects the most compact representation [7]. For our use case we will pass a value of 3.

std::string serializedObjectPretty = obj.dump(3);
Serial.println(serializedObjectPretty.c_str());

If by some reason we want to use an indent character other than an empty space (the default), we can pass a char as second argument of the dump method and, in case the indent value is greater than zero, that char will be used [7]. We will use a “|” character.

std::string customIndentChar = obj.dump(3, '|');
Serial.println(customIndentChar.c_str());

The method supports two additional optional parameters but we are not going to cover their usage here. You can learn more about them here.

The full code is available on the snippet below.

#include <json.hpp>
using json = nlohmann::json;

void setup()
{
  Serial.begin(115200);

  json obj;

  obj["age"]=10;
  obj["name"] = "John";
  obj["address"]["street"] = "St. Street";
  obj["address"]["code"] = "2790-124";

  std::string serializedObject = obj.dump();
  Serial.println("Serialized object:");
  Serial.println(serializedObject.c_str());

  std::string serializedObjectPretty = obj.dump(3);
  Serial.println("\n\nSerialized object pretty:");
  Serial.println(serializedObjectPretty.c_str());

  std::string customIndentChar = obj.dump(3, '|');
  Serial.println("\n\nCustom Indent Char:");
  Serial.println(customIndentChar.c_str());
}

void loop()
{
}

When testing the previous code, you should get a result similar to figure 6. As can be seen, the first print has the serialized object in its most compact format. The second one shows a pretty printed JSON string, with the levels of indentation we have defined in our code. Finally, the third print shows the indented JSON using a custom indentation character (in our case, a “|“).

JSON Deserialization

Naturally, besides serializing JSON, we should also be able to deserialize a JSON string to a json object using the Nlohmann/json lib. In this section, we are going to cover the process to deserialize a JSON string.

As before, we start by the library include and the using.

#include <json.hpp>
using json = nlohmann::json;

Then we will move on to the Arduino setup, starting by opening a serial connection.

Serial.begin(115200);

After this we will define a JSON string, which we will then parse to a json object. We will be defining a raw string literal, to avoid having to escape characters.

For testing purposes, our string will contain a JSON with the same person structure we have used in the previous sections.

char str[] = R"(
   {
     "name": "John",
     "age": 10,
     "address": {
       "street": "St. Street",
       "code": "1234-12"
     }
   }
)";

To get a parsed json object from this string we simply need to call the parse static method, passing as input our previously defined string. As output, the method will return a json object.

json obj = json::parse(str);

To confirm that the deserialization was correctly performed, we will access each of the fields and print their value to the serial port.

std::string name = obj["name"].get<std::string>();
int age = obj["age"].get<int>();
std::string street = obj["address"]["street"].get<std::string>();
std::string code = obj["address"]["code"].get<std::string>();

Serial.println(name.c_str());
Serial.println(age);
Serial.println(street.c_str());
Serial.println(code.c_str());

To complement, we will also serialize our json object back to a string and print the result to the serial port. We will print it without indentation, just to confirm everything is working as expected.

std::string serializedObject = obj.dump();
Serial.println();
Serial.println(serializedObject.c_str());

The whole code is summarized below.

#include <json.hpp>
using json = nlohmann::json;

void setup()
{
  Serial.begin(115200);

    char str[] = R"(
    {
      "name": "John",
      "age": 10,
      "address": {
        "street": "St. Street",
        "code": "1234-12"
      }
    }
    )";

    json obj = json::parse(str);

    std::string name = obj["name"].get<std::string>();
    int age = obj["age"].get<int>();
    std::string street = obj["address"]["street"].get<std::string>();
    std::string code = obj["address"]["code"].get<std::string>();

    Serial.println(name.c_str());
    Serial.println(age);
    Serial.println(street.c_str());
    Serial.println(code.c_str());

    
    std::string serializedObject = obj.dump();
    Serial.println();
    Serial.println(serializedObject.c_str());
}

void loop()
{
}

When testing the code, you should obtain something similar to figure 7, which shows the values of all the fields from the original JSON string, and also the serialized string.

Getting the Keys of a JSON object

In this section we are going to learn how to obtain the list of the keys of a JSON object. Note however that this method won’t return keys of nested objects, meaning we will only obtain the keys at the root. We will illustrate this in our code.

So, on the setup function, we will define a json object with the properties we have been using before. Note that “address” will hold a nested object that contains the properties “street” and “code“. “address” will be obtained because it is a key at the root of our object, but “street” and “code” won’t.

json obj;
  
obj["age"]=10;
obj["name"] = "John";
obj["address"]["street"] = "St. Street";
obj["address"]["code"] = "2790-124";

Now we are going to call the items method to get an iterator to the values of the json object. We can then use this iterator in a ranged-based for loop to get each value [8].

For each element, we then simply call the key method, which will return the key as a string. Then, to print it to the serial port, we need to call the c_str method, for the reasons already explained before.

for (auto item : obj.items())
{
        Serial.println(item.key().c_str());
}

The whole code can be copied from the snippet below.

#include <json.hpp>
using json = nlohmann::json;

void setup()
{
  Serial.begin(115200);

  json obj;
  
  obj["age"]=10;
  obj["name"] = "John";
  obj["address"]["street"] = "St. Street";
  obj["address"]["code"] = "2790-124";

  for (auto item : obj.items())
  {
        Serial.println(item.key().c_str());
  }
}

void loop()
{
}

Upon running the code we should obtain the same result as in figure 8. Like we mentioned, only the keys on the root of the object are obtained.

JSON Flatten

In this section we are going to cover an operation called flatten. After flattening a JSON object, we will obtain a new JSON object only with one level of depth, independently of how nested the original object was. In the resulting object, it very easy to iterate through all the leaves.

The property names of the resulting flattened object will correspond to the full path of each property in the original object. In the implementation of this library, each part of the path is separated by a “/” character.

Focusing on the setup function, we will again create a json object and set some fields in a possible representation of a person.

json obj;

obj["age"]=10;
obj["name"] = "John";
obj["address"]["street"] = "St. Street";
obj["address"]["code"] = "2790-124";

Then, to obtain the flattened version of this object, we simply need to call the flatten method. This method takes no arguments and it will return a new json object.

json flat = obj.flatten();

After that, we can simply serialize it like we have seen above and print the result to the serial port. We will be adding an indentation of 3, to make the result easier to read.

std::string serializedFlat = flat.dump(3);
Serial.println("\n\nSerialized flat object:");
Serial.println(serializedFlat.c_str());

The whole code is shown below.

#include <json.hpp>
using json = nlohmann::json;

void setup()
{
  Serial.begin(115200);

  json obj;

  obj["age"]=10;
  obj["name"] = "John";
  obj["address"]["street"] = "St. Street";
  obj["address"]["code"] = "2790-124";

  json flat = obj.flatten();

  std::string serializedFlat = flat.dump(3);
  Serial.println("\n\nSerialized flat object:");
  Serial.println(serializedFlat.c_str());

}

void loop()
{
}

Upon testing the code you should obtain a result similar to figure 9.

As a note regarding the format of the flattened JSON, I’ve seen different implementations for the final format of the JSON, in different libraries (check here an example for C# and here for JavaScript). In the examples mentioned, the separator character is different (it is a “.”) and it also doesn’t appear in the beginning of the path, like for our case.

Nonetheless, with some string manipulation, you should be able to obtain different results that better adapt to your application, if necessary.

It’s also important to take in consideration that the flatten operation is reversible, and the library also offers a way of unflatten an object, like described here. We are not going to cover that procedure here but it should be very straightforward, like the documentation illustrates.

Parsing HTTP JSON response

We will finish this tutorial with an example where we will parse the body of a HTTP GET request. For a detailed tutorial on how to do HTTP GET requests using the ESP32, please check here.

We will be doing the request against a testing API, more precisely the following endpoint:

http://jsonplaceholder.typicode.com/comments/1

If we use the browser to access this endpoint, we can see that it will return the following JSON body (the result will always be the same):

{
  "postId": 1,
  "id": 1,
  "name": "id labore ex et quam laborum",
  "email": "Eliseo@gardner.biz",
  "body": "laudantium enim quasi est quidem magnam voluptate ipsam eos\ntempora quo necessitatibus\ndolor quam autem quasi\nreiciendis et nam sapiente accusantium"
}

So, in our code, we will parse the body of the response and access some of its fields.

We will start our code with the library includes. We will need the following ones:

• WiFi.h: To connect the ESP32 to a WiFi network.
• HTTPClient.h: To be able to do a HTTP GET request.
• json.hpp: To parse the response of the request.

We will also add the using we have been seen in the previous sections.

#include <WiFi.h>
#include <HTTPClient.h>
#include <json.hpp>

using json = nlohmann::json;

After that we will define two global variables to hold the credentials of the WiFi network: name (SSID) and password.

const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPass";

Moving on to the Arduino setup, we will connect the ESP32 to the WiFi network, using the previously defined variables.

void setup() {
 
  Serial.begin(115200);
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
 
  Serial.println("Connected to the WiFi network");
 
}

We will write the rest of our code in the Arduino loop. We will do a request periodically (every 10 seconds), parse the result JSON body and print some of the fields from the parsed json object.

First we will create an object of class HTTPClient. We will use this object to perform our HTTP GET request.

HTTPClient http;

Then we will call the begin method on our object. As input we pass a string with the URL we want to reach.

http.begin("http://jsonplaceholder.typicode.com/comments/1");

To send the actual request, we now need to call the GET method on our HTTPClient object. This method receives no parameters and returns the HTTP response code, in case of success doing the request, and a code lesser than 0 if any issue has occurred.

int httpCode = http.GET();      

In case of success doing the request, we will get the response payload by calling the getString method. This method receives no arguments and returns an Arduino String with the response.

String payload = http.getString();

Once we get the payload, we will use the c_str method (the Arduino String class also has a method with this name), which will return a pointer to the C-style version of the string (in other words, a char *).

Then we will call the parse method to obtain the corresponding parsed json object, passing as input the payload string.

json obj = json::parse(payload.c_str());

Then we will access the following fields and print their values:

• name
• email
• id

std::string name = obj["name"].get<std::string>();
std::string email = obj["email"].get<std::string>();
int id = obj["id"].get<int>();

Serial.println(name.c_str());      
Serial.println(email.c_str());
Serial.println(id);

After this, to free the resources, we need to call the end method on the HTTPClient.

http.end();

The complete code is shown below. As can be seen, it includes an error check for the result of the HTTP request and also a 10 seconds delay.

#include <WiFi.h>
#include <HTTPClient.h>
#include <json.hpp>

using json = nlohmann::json;
 
const char* ssid = "yourNetworkName";
const char* password =  "yourNetworkPass";
 
void setup() {
 
  Serial.begin(115200);
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
 
  Serial.println("Connected to the WiFi network");
 
}
 
void loop() {
 
  HTTPClient http;

  http.begin("http://jsonplaceholder.typicode.com/comments/1"); 
  int httpCode = http.GET();                                        

  if (httpCode > 0) { 

      String payload = http.getString();

      json obj = json::parse(payload.c_str());

      std::string name = obj["name"].get<std::string>();
      std::string email = obj["email"].get<std::string>();
      int id = obj["id"].get<int>();

      Serial.println("------Result--------");
      Serial.println(name.c_str());      
      Serial.println(email.c_str());
      Serial.println(id);
  }

  else {
    Serial.println("Error on HTTP request");
  }

  http.end();
 
  delay(10000);
}

To test the code, first replace the credentials of the network by the ones that apply to yours. Then, compile and upload the code and open the Arduino IDE serial monitor.

You should get a result similar to figure 10. As can been, we have obtained the fields of the JSON defined in the code, meaning the parsing has occurred successfully.

Final Notes

As we can see from the previous code, this library is very flexible and offers an elegant syntax that feels very natural, specially for those who are used to higher level languages.

Note however that I did not do any testing to the library performance on the ESP32, but rather focused on how to use it for simple use cases. As such, I cannot guarantee that it will be adequate for more complex use cases with a lot of objects or operations.

As for the library name, I’ve been referring to it as Nlohmann/json as I did not find any formal name for it.

Recommended Resources

• Library website
• Doxygen documentation
• GitHub page

Suggested Readings

• ESP32 ArduinoJson: parsing JSON
• ESP32 ArduinoJson: serializing JSON

References

[1] https://www.circuito.io/blog/arduino-code/

[2] https://arduino.stackexchange.com/questions/816/c-vs-the-arduino-language

[3] https://www.json.org/json-en.html

[4] https://github.com/nlohmann/json

[5] https://json.nlohmann.me/doxygen/classnlohmann_1_1basic__json_a3f45f3820c456ad2e3f3df2926564151.html#a3f45f3820c456ad2e3f3df2926564151

[6] https://json.nlohmann.me/api/basic_json/get/

[7] https://json.nlohmann.me/api/basic_json/dump/

[8] https://json.nlohmann.me/api/basic_json/items/