ESP32: BSON

Contents hide

Introduction

In this tutorial we will learn how to perform BSON serialization and deserialization, using the ESP32, the Arduino core and the Nlohmann/json library.

BSON (Binary JSON) is a binary-encoded serialization of JSON-like documents [1] that originated in 2009 at MongoDB [2]. BSON was designed to be efficient, easy to traverse and efficient [1].

Note however that BSON is not necessarily more compact than JSON. In some cases, BSON may use more space than a JSON. This results from the traversability design goal: BSON adds some “extra” information to documents, like length of strings and sub-objects, making traversal faster [3]. You can read here a very interesting comparison between JSON and BSON.

The tests shown in the sections below were performed on a ESP32-E FireBeetle board from DFRobot. The Arduino core version used was 2.0.0 and the Arduino IDE version was 1.8.15, working on Windows 8.1. The version of the Nlohmann/json library used was 3.10.2.

Serializing to BSON

As usual, we will start our code by including the json.hpp library. This library will expose to us all the functionality we need to work with JSON and to perform the conversion to BSON.

#include <json.hpp>

Then we will write the rest of our code in the Arduino setup. We will begin by opening a serial connection, to later output the results of our code.

Serial.begin(115200);

After this we will define a string that will contain the JSON object we will serialize to the BSON format. We will create a very simple structure that could represent a person.

char str[] = R"(
  {
    "name": "Jake",
    "age": 23,
    "address": {
      "street": "St. Street",
      "code": "1234-127"
    }
  }
)";

Now that we have our JSON string, it is time to deserialize it to a json object. As already covered in detail here, we can parse a JSON string simply by calling the parse static method, passing as input the string. As output, this method will return a json object with the parsed data structure.

nlohmann::json obj = nlohmann::json::parse(str);

Serializing our json object to BSON is equally trivial: we simply need to call the to_bson static method. As input we need to pass the json object and as output we will get a vector of bytes.

std::vector<std::uint8_t> bsonArray = nlohmann::json::to_bson(obj);

We will first iterate through all the bytes of our vector and print them to the serial port in hexadecimal format. In our particular case, we will be using a range-based for loop to do the iteration.

for(uint8_t i : bsonArray){ 
  Serial.printf("%02X ", i);
}

We will also print the same bytes in decimal format, so we can later reuse them to perform the deserialization operation in the next section.

for(uint8_t i : bsonArray){ 
  Serial.printf("%d ", i);
}

The full code is available below.

#include <json.hpp>

void setup() {

  Serial.begin(115200);

  char str[] = R"(
    {
      "name": "Jake",
      "age": 23,
      "address": {
        "street": "St. Street",
        "code": "1234-127"
      }
    }
  )";

  nlohmann::json obj = nlohmann::json::parse(str);
  std::vector<std::uint8_t> bsonArray = nlohmann::json::to_bson(obj);

  for(uint8_t i : bsonArray){ 
    Serial.printf("%02X ", i);
  }

  Serial.println("\n--------");
  for(uint8_t i : bsonArray){ 
    Serial.printf("%d ", i);
  }
}

void loop() {}

To test the code, simply compile and upload it to your ESP32. As output, you should get a result similar to figure 1. As can be seen, we have obtained the serialized bytes both in hexadecimal and decimal formats, as expected.

**Figure 1** – Result of the BSON serialization, performed on the ESP32.

Deserializing from BSON

In this section we will learn how to perform the deserialization of a BSON payload to a json object.

Like before, we start by including the json.hpp lib.

#include <json.hpp>

Moving on to the Arduino setup and after opening a serial connection, we will define a vector of bytes that will correspond to our BSON payload. For simplicity, we will use the bytes that we obtained by running the code from the previous section. Consequently, we expect to obtain the same JSON object from before.

std::vector<std::uint8_t> bsonArray = {
    85, 0, 0, 0, 3, 97, 100, 100, 114, 101, 115, 115, 0, 
    47, 0, 0, 0, 2, 99, 111, 100, 101, 0, 9, 0, 0, 0, 49,
    50, 51, 52, 45, 49, 50, 55, 0, 2, 115, 116, 114, 101,
    101, 116, 0, 11, 0, 0, 0, 83, 116, 46, 32, 83, 116, 
    114, 101, 101, 116, 0, 0, 16, 97, 103, 101, 0, 23, 0, 
    0, 0, 2, 110, 97, 109, 101, 0, 5, 0, 0, 0, 74, 97, 107,
    101, 0, 0, 
};

To perform the deserialization of the previous vector to a json object we need to call the from_bson static method, passing as input the vector. As output we will obtain a json object.

nlohmann::json obj = nlohmann::json::from_bson(bsonArray);

Finally, to confirm that the content was properly deserialized, we will serialize our json object to a string and print it to the serial port. The result should match the same JSON string object from the previous section (note that the order of the keys and the indentation will be different, but that is irrelevant for a JSON object).

std::string serializedObject = obj.dump(3);
Serial.println(serializedObject.c_str());

The full code is shown below.

#include <json.hpp>

void setup() {

  Serial.begin(115200);

  std::vector<std::uint8_t> bsonArray = {
    85, 0, 0, 0, 3, 97, 100, 100, 114, 101, 115, 115, 0, 
    47, 0, 0, 0, 2, 99, 111, 100, 101, 0, 9, 0, 0, 0, 49,
    50, 51, 52, 45, 49, 50, 55, 0, 2, 115, 116, 114, 101,
    101, 116, 0, 11, 0, 0, 0, 83, 116, 46, 32, 83, 116, 
    114, 101, 101, 116, 0, 0, 16, 97, 103, 101, 0, 23, 0, 
    0, 0, 2, 110, 97, 109, 101, 0, 5, 0, 0, 0, 74, 97, 107,
    101, 0, 0, 
  };

  nlohmann::json obj = nlohmann::json::from_bson(bsonArray);

  std::string serializedObject = obj.dump(3);
  Serial.println(serializedObject.c_str());
}

void loop() {}

Upon running the code, the expected result is shown on figure 2 below. As can be seen, we have obtained the same JSON object that we originally used in the previous code section.