Introduction
In this tutorial we are going to learn how to preserve the order of object keys when handling JSON with the Nlohmann/json library on the ESP32. For this tutorial we will be using the Arduino core.
It is important to take in consideration that the JSON standard specifies that an object is an unordered collection of zero or more name/value
pairs [1]. You can read the RFC here.
The statement above is coherent with the common use case where we access a JSON object by its keys, meaning that the order how they are stored and serialized / deserialized should not matter.
Nonetheless, there might by specific use cases where we may want to preserve the order of the keys. Since the library offers a very simple way of doing it, we are going to learn it on this tutorial.
Important: The default behavior of the json class is traversing the keys in alphabetical order [1]. Alphabetical order is different from the order of insertion of properties in the object. In our case, our objective is to preserve the insertion order of the keys, regardless of how they are sorted alphabetically. Thus, when we refer to the json class, we will refer as unordered keys from the insertion point of view.
If it is your first time with the Nlohmann/json library and if you haven’t yet installed it as an Arduino lib, please read the introductory tutorial. There you can learn how to install the library and the basics on how to use it.
The version of the Nlohmann/json library used was 3.10.2. The Arduino core version used was 2.0.0. The tests shown below were performed on a ESP32-E FireBeetle board from DFRobot.
Keeping key insertion order on JSON serialization
In this section we will create two objects with the same keys, leaving one with the default behaviour (unordered) and ensuring that the keys will keep the insertion order for the other.
To confirm order is kept in one of them, we will perform the serialization of both objects to a string. The procedure on how to serialize a json object to a string was already covered in detail here.
To start our code, the first thing we will do is including the library.
#include <json.hpp>
Then we will declare the usage of the json specialization from the nlohmann namespace. Like covered in the introductory tutorial, this is a specialization of the basic_json class template.
Like we already mentioned, the default behavior for this class is to provide the keys unordered (they will be sorted alphabetically, which is not what we are looking for).
using json = nlohmann::json;
We will also declare the usage of the ordered_json specialization. This is another specialization of basic_json that, as the name implies, preserves the order of the keys.
using ordered_json = nlohmann::ordered_json;
Moving on to the Arduino setup, we will start by opening a serial connection, so we can output the results of our program.
Serial.begin(115200);
Then we will create a json object and fill it with 3 properties, by this order (their values are irrelevant for what we are testing):
- c_prop
- z_prop
- b_prop
json unordered;
unordered["c_prop"] = 10;
unordered["z_prop"] = "John";
unordered["b_prop"] = "Test";
We called our json object “unordered” because, like we already mentioned, the insertion order of the keys won’t be preserved.
Now we will do the exact same properties assignment but to an ordered_json object.
ordered_json ordered;
ordered["c_prop"] = 10;
ordered["z_prop"] = "John";
ordered["b_prop"] = "Test";
Finally, we will convert both objects to serialized strings, using the dump method, and print them to the serial port. Recall from the previous tutorial that this method returns a std:string, meaning we cannot directly print it using the Serial println method. As such, we will call the c_str method on the result of the dump call, before printing.
Serial.println(unordered.dump().c_str());
Serial.println(ordered.dump().c_str());
The complete code is shown below.
#include <json.hpp>
using json = nlohmann::json;
using ordered_json = nlohmann::ordered_json;
void setup() {
Serial.begin(115200);
json unordered;
unordered["c_prop"] = 10;
unordered["z_prop"] = "John";
unordered["b_prop"] = "Test";
ordered_json ordered;
ordered["c_prop"] = 10;
ordered["z_prop"] = "John";
ordered["b_prop"] = "Test";
Serial.println("Unordered: ");
Serial.println(unordered.dump().c_str());
Serial.println("\nOrdered: ");
Serial.println(ordered.dump().c_str());
}
void loop() {
}
To test the code, as usual, simply compile it and upload it using the Arduino IDE. After the process is finished, open the IDE serial monitor.
You should see a result like figure 1. We can confirm that, for the first object, the order of the inserted keys is not preserved and, instead, we get the keys by alphabetical order in the serialized string. On the other hand, in the second JSON object, the order of insertion is maintained, as expected.
Listing all keys
In this section we are going to confirm that the ordered_json object will return the keys in the insertion order in case we iterate through all the keys, like we have learned here.
We will start by the library includes and the usings, like in the previous section.
#include <json.hpp>
using json = nlohmann::json;
using ordered_json = nlohmann::ordered_json;
After opening a serial connection, we will create a json obejct and an ordered_json object and populate them with the exact 3 properties we have used above.
json unordered;
unordered["c_prop"] = 10;
unordered["z_prop"] = "John";
unordered["b_prop"] = "Test";
ordered_json ordered;
ordered["c_prop"] = 10;
ordered["z_prop"] = "John";
ordered["b_prop"] = "Test";
Finally we will iterate through the keys of each object and print them to the serial port.
for (auto item : unordered.items())
{
Serial.println(item.key().c_str());
}
for (auto item : ordered.items())
{
Serial.println(item.key().c_str());
}
The complete code is shown in the snippet below.
#include <json.hpp>
using json = nlohmann::json;
using ordered_json = nlohmann::ordered_json;
void setup() {
Serial.begin(115200);
json unordered;
unordered["c_prop"] = 10;
unordered["z_prop"] = "John";
unordered["b_prop"] = "Test";
ordered_json ordered;
ordered["c_prop"] = 10;
ordered["z_prop"] = "John";
ordered["b_prop"] = "Test";
Serial.println("\nUnordered: ");
for (auto item : unordered.items())
{
Serial.println(item.key().c_str());
}
Serial.println("\nOrdered: ");
for (auto item : ordered.items())
{
Serial.println(item.key().c_str());
}
}
void loop() {
}
Upon running the code, you should obtain a result similar to the one shown in figure 2. As can be seen, like in the previous section, the json object provides the properties unordered (following the alphabetical order) and the ordered_json object respects the insertion order.
Deserializing JSON strings
To finalize, we are going to test that our approach works for another common operation: deserializing a JSON string. For a detailed guide on this process, please check here.
Like before, we start by the library include plus the two usings.
#include <json.hpp>
using json = nlohmann::json;
using ordered_json = nlohmann::ordered_json;
Moving on to the Arduino setup and after opening a serial connection, we will define a raw string literal with a JSON object that contains the same 3 properties we have been using. We will follow a different order this time:
- b_prop
- z_prop
- c_prop
char str[] = R"(
{
"b_prop": 10,
"z_prop": "John",
"c_prop": "test"
}
)";
We will then parse this string to a json object and to an ordered_json object. This is done with a call to the parse static method, passing as input the JSON string.
json unordered = json::parse(str);
ordered_json ordered = ordered_json::parse(str);
Finally, we will serialize both objects back to a string using the dump method. This time, we will pass an indentation level of 3, so we obtain a pretty printed string.
Serial.println(unordered.dump(3).c_str());
Serial.println(ordered.dump(3).c_str());
The complete code can be obtained in the snippet below.
#include <json.hpp>
using json = nlohmann::json;
using ordered_json = nlohmann::ordered_json;
void setup() {
Serial.begin(115200);
char str[] = R"(
{
"b_prop": 10,
"z_prop": "John",
"c_prop": "test"
}
)";
json unordered = json::parse(str);
ordered_json ordered = ordered_json::parse(str);
Serial.println("Unordered: ");
Serial.println(unordered.dump(3).c_str());
Serial.println("\nOrdered: ");
Serial.println(ordered.dump(3).c_str());
}
void loop() {
}
After testing the code, you should obtain the same result as figure 3. Once again, the order of the keys for each of the objects is accordingly to the expectation: the json object didn’t preserve the original order in the serialized string and the ordered_json object did.
References
[1] https://datatracker.ietf.org/doc/html/rfc8259.html