ESP8266: Posting JSON data to a Flask server on the cloud

The objective of this post is to explain how to make a POST request containing JSON data to a cloud server, with the ESP8266. The cloud server will be implemented using Flask and will be hosted in Pythonanywhere.

Introduction

The objective of this post is to explain how to make a POST request containing JSON data to a cloud server, with the ESP8266. The cloud server will be implemented using Flask and will be hosted in Pythonanywhere.

So, I’m assuming you already have the ESP8266 libraries for the Arduino IDE installed and also created and configured an account on Pythonanywhere.

This tutorial puts together a couple of different functionalities introduced in previous posts, both for the ESP8266 and Flask. So, it will not contain a detailed description of every piece of code. You can check the previous post that support this tutorial on the related posts section.

The Python code

The Python code to get the parsed JSON content is very simple and was explained in more detail in this previous post. Access your account on Python anywhere and get to the code editor to edit your Flask app. You can check here how to set a simple Flask app on Pythonanywhere.

First of all, we need to import the Flask class from the flask module, so all the functionality we need becomes available. We will also import the request object from the flask module, which we will use later.

Then, we create an instance of the Flask class. In the constructor, we pass the name of the module of our application, using the __name__ global variable.

from flask import Flask
from flask import request

app = Flask(__name__)

We will assume that the client will be posting JSON data, so we will specify a route that only answers to HTTP POST requests. We do this by passing the POST keyword to the methods argument of the route decorator. Our route will be listening on the /postjson URL. You can read more on how to control the HTTP allowed methods of a Flask app here.

To get the posted JSON data, we just need to call the get_json method on the request object we previously imported. This mehod parses the incoming JSON request data and returns it as a Python dictionary. Then, we print this object, so we can later confirm the content was correctly received.

Finally, we return a response to the client. The full code is shown bellow. Note that, in Pythonanywhere, we don’t need to call the run method on our app object.

from flask import Flask
from flask import request

app = Flask(__name__)

@app.route('/postjson', methods = ['POST'])
def postJsonHandler():
    content = request.get_json()
    print (content)
    return 'JSON posted'

Testing the Python code

We will test the Python code first, before going to the ESP8266. This way, it’s easier to debug and identify the root of the problems, since we will test things separately.

First, after completing the Python code, don’t forget to save it and reload it, using the buttons on the top right side of the Pythonanywhere editor.

Then, open Postman to test the request. On the HTTP method dropdown choose POST. On the URL bar, insert the URL where your application is hosted, and the path (/postjson) we defined for the route.

Then, select the body separator and the raw radio button. In the dropdown on the right of all the radio buttons, select JSON (application/json). Then, on the text editor, put some valid JSON and hit the send button.

You should get the “JSON posted” message we defined as response of requests on the /postjson route. The main areas indicated before are highlighted in figure 1, which includes the correct response from the server.

Figure 1 – Posting JSON data to the cloud with Postman.

We can also check the result of this request on the server side. On the main page of Pythonanywhere (the one we land after logging to our account) select the Web tab. Scroll to the middle, to the Log files section and select the link on the Server log entry, as indicated in figure 2.

Figure 2 – Accessing the log files of the server.

You should now get to the log file of your server. In my case, as indicated in figure 3, I already have a lot of requests received and printed by my Flask app, from the tests I did for this tutorial. Note that the printed content corresponds to the parsed data sent in the previous request. Keep in mind that you will need to reload this page to get the prints of new requests.

Figure 3 – Output of the log file of the server.

The ESP8266 code

We will start by doing some includes, so we can have all the functionality needed to connect to a WiFi network, send HTTP requests and encode JSON messages.

The ESP8266WiFi library is the one needed for our ESP8266 to connect to a WiFi network. You can check here a detailed tutorial on how to use it. The ESP8266HTTPClient library allows us to send HTTP requests with an easy to use interface. You can check here a detailed tutorial on how to make POST requests with this library. These two libraries are default libraries for the Arduino IDE ESP8266 installation.

Finally, the ArduinoJson library allows us to encode JSON messages. This library needs to be installed since is not a standard one from the ESP8266 libraries for the Arduino IDE. You can install it from the library manager of the Arduino IDE. Here is a detailed tutorial on how to use its functionality in the ESP8266.

Bellow are the imports of the mentioned libraries.

#include <ESP8266HTTPClient.h>
#include <ESP8266WiFi.h>
#include <ArduinoJson.h>

In the setup function, we will simply connect to the WiFi network, and start the serial console for debugging. The code for this is very simple and can be seen bellow.

void setup() {

  Serial.begin(115200);                             //Serial connection
  WiFi.begin("YourNetworkName", "YourPassword");    //WiFi connection

  while (WiFi.status() != WL_CONNECTED) {  //Wait for the WiFI connection

    delay(500);
    Serial.println("Waiting for connection");

  }

}

We will do remaining coding in the main loop function. We will create some dummy JSON data to post to the Flask server running on Pythonanywhere.

First, we declare an object of class StaticJsonBuffer for encoding the message. We need to specify a size that is big enough for the structure we are going to create. In this case, we will specify 300 bytes, which is more than enough.

Then, we get a reference to a JsonObject from the StaticJsonBuffer object we created, by calling the createObject method.

StaticJsonBuffer<300> JSONbuffer;
JsonObject& JSONencoder = JSONbuffer.createObject();

Now, we already have what we need to start specifying the structure of our JSON message. We will use the dummy JSON example bellow which represents a possible data structure for measurements of a temperature IoT device.

{
"sensorType": "Temperature",
"values": [20, 21, 23],
"timestamps": ["10:10", "10:20", "10:30"]
 }

To create simple name/value pairs in the JSON structure, we use the subscript operator or, in other words, we use square brackets on the reference of the JsonObject. So, as seen bellow, we use this functionality to specify the “sensorType” attribute as “Temperature”.

JSONencoder["sensorType"] = "Temperature";

Finally, we need to specify our arrays of measurement values and corresponding timestamps. To create arrays, we call the createNestedArray method on the JsonObject reference. Then we use the add method of the JsonArray reference to add the values we want on our array.

JsonArray& values = JSONencoder.createNestedArray("values");
values.add(20);
values.add(21);
values.add(23);
JsonArray& timestamps = JSONencoder.createNestedArray("timestamps");
timestamps.add("10:10");
timestamps.add("10:20");
timestamps.add("10:30");

We will now need to print the JSON structure we defined to a string, so we can send it in our request to the server. To do so, we just need to declare a char buffer to hold our data and call the prettyPrintTo method, as indicated bellow. Note that we could have used another method, called printTo, which prints the JSON content with the less overhead possible, but also less easy for a person to read.

char JSONmessageBuffer[300];
JSONencoder.prettyPrintTo(JSONmessageBuffer, sizeof(JSONmessageBuffer));
Serial.println(JSONmessageBuffer);

To finish the code, we need to do the actual HTTP request. We start by declaring an object of the HTTPClient class. Then, we call the begin method on that object, which receives as input the URL of the destination. In our case, it’s the URL of our Flask server (the one used in Postman).

In order for our request to be correctly interpreted by the server, we also specify the content-type as application/json. We do it by calling the addHeader method on the HTTPClient object.

HTTPClient http; //Declare object of class HTTPClient
http.begin("http://anteph.pythonanywhere.com/postjson"); //Specify request destination
http.addHeader("Content-Type", "application/json"); //Specify content-type header

Finally, we get the response HTTP code and the response message and print both to the serial port. We should get a 200 HTTP Code if everything is working correctly, and the response should be the message defined in the Python code. After that, we close the connection with the end method.

int httpCode = http.POST(JSONmessageBuffer); //Send the request
String payload = http.getString(); //Get the response payload
Serial.println(httpCode); //Print HTTP return code
Serial.println(payload); //Print request response payload
http.end(); //Close connection

The full code can be seen bellow.

#include <ESP8266HTTPClient.h>
#include <ESP8266WiFi.h>
#include <ArduinoJson.h>

void setup() {

  Serial.begin(115200);                            //Serial connection
  WiFi.begin("YourNetworkName", "YourPassword");   //WiFi connection

  while (WiFi.status() != WL_CONNECTED) {  //Wait for the WiFI connection completion

    delay(500);
    Serial.println("Waiting for connection");

  }

}

void loop() {

  if (WiFi.status() == WL_CONNECTED) { //Check WiFi connection status

    StaticJsonBuffer<300> JSONbuffer;   //Declaring static JSON buffer
    JsonObject& JSONencoder = JSONbuffer.createObject(); 

    JSONencoder["sensorType"] = "Temperature";

    JsonArray& values = JSONencoder.createNestedArray("values"); //JSON array
    values.add(20); //Add value to array
    values.add(21); //Add value to array
    values.add(23); //Add value to array

    JsonArray& timestamps = JSONencoder.createNestedArray("timestamps"); //JSON array
    timestamps.add("10:10"); //Add value to array
    timestamps.add("10:20"); //Add value to array
    timestamps.add("10:30"); //Add value to array

    char JSONmessageBuffer[300];
    JSONencoder.prettyPrintTo(JSONmessageBuffer, sizeof(JSONmessageBuffer));
    Serial.println(JSONmessageBuffer);

    HTTPClient http;    //Declare object of class HTTPClient

    http.begin("http://anteph.pythonanywhere.com/postjson");      //Specify request destination
    http.addHeader("Content-Type", "application/json");  //Specify content-type header

    int httpCode = http.POST(JSONmessageBuffer);   //Send the request
    String payload = http.getString();                                        //Get the response payload

    Serial.println(httpCode);   //Print HTTP return code
    Serial.println(payload);    //Print request response payload

    http.end();  //Close connection

  } else {

    Serial.println("Error in WiFi connection");

  }

  delay(30000);  //Send a request every 30 seconds

}

Testing the final code

Finally, we just need to upload the code to the ESP8266 and run it. We should get an output similar to figure 4, where the response from the server is highlighted.

Figure 4 – Output of the ESP8266 program.

You can also test if the Flask server is receiving the correct data by reloading the log file of the server in Pythonanywhere, as explained before.

Final notes

Although this tutorial is slightly more complicated than the previous ones, we can see that with the tools we have available right now we can already make a device talk with a remote webserver, hosted on the cloud.

More than that, we don’t need a lot of complicated code, since we are using very useful libraries and frameworks that hide the complicated implementation details from us.

The natural step to take from here is to start exploring the database functionalities of Pythonanywhere, in order to store and process the information and latter present it to a user. Of course that database and frontend technologies to present data to a user are a different paradigm that we have seen before.

Note that we could have used other cloud host to our Flask application, but the architecture would be similar. In this previous post, I presented a proof of concept of an ESP8266 application that talks with a server hosted on Openshift.

Other important note is that we are communicating with the cloud using HTTP. This means there are no security measures and our data is sent in plain text. So, it can be intercepted by others. Security is also a critical aspect of an IoT application that needs to be taken in consideration.

Related Posts

• Flask: Parsing JSON data
• Flask: Controlling HTTP methods allowed
• Python anywhere: Deploying a Flask server on the cloud
• ESP8266: HTTP POST Requests
• ESP8266: HTTP GET Requests
• ESP8266: Connecting to a WiFi Network
• ESP8266: uploading code from Arduino IDE