Introduction
In this tutorial we will learn how to render a .jpg image on a ILI9341 display, using the ESP32 and the Arduino core.
For a tutorial on how to wire the ESP32 to the ILI9341 display and render some text, please check here. The code shown in the sections below assumes the same wiring from the mentioned tutorial. If you are using a different wiring configuration, please make sure to adapt the code pin mappings.
We are going to upload the image to the ESP32 SPIFFS file system beforehand, so we can read it from there in our program and send it to the display to be rendered.
In order for us to be able to render the image, we will need to be able to decode it and access its uncompressed bitmap, to be sent to the display. To do it, we will use the TJpg_Decoder library, which has an API that allows to decode a .jpg image block by block, already supporting reading it from the ESP32 SPIFFS file system. This library can be installed from the Arduino IDE library manager.
For reference, the Arduino_GFX library we are going to use to interact with the display also has an example on how to render a .jpg image (code here), although it uses a different decoder library.
Uploading the image to the file system
To upload the image to the file system, we are going to use the Arduino IDE plugin covered on this tutorial.
For testing purposes, we are going to use this image from the TJpg_Decoder library examples. This image has the exact dimensions of the ILI9341 display (240 x 320 pixels), which avoids the need for us to worry about resizing the image to fit the display.
In short, in order to upload the image using the mentioned plugin, we simply need to go to the Arduino sketch folder and create a folder called “data“. Inside this folder, we paste the image and then, with the ESP32 already connected to a computer, we go to the Arduino IDE and click Tools -> ESP32 Sketch Data Upload. This should start the upload of the file to the ESP32 file system.
Once the procedure is finished, we should have the image in the file system, with the same name we have assigned to the file. In my case I’ve named it test.jpg, so it should be accessible on the path “/test.jpg“.
Decoding the jpg image
In this section we are going to focus on how the TJpg_Decoder library works, before we worry about sending the actual decoded image to the display. So, we are going to analyze its API and how it handles the decoding of the image in blocks.
We will start our code by the library includes. First, we will need the TJpg_Decoder.h we have installed, to be able decode the .jpg file. Note that this include will expose to us an extern variable called TJpgDec, which is an object of class TJpg_Decoder. We will interact with this extern variable in our code.
#include <TJpg_Decoder.h>
We will also need to include the SPIFFS.h library, so we can mount the file system and access it.
#include <SPIFFS.h>
We will also define a string that contains the path to our image in the file system.
const char* filePath = "/test.jpg";
To finalize the global variables, we will also define a variable called nBlocks, which we will use to count how many blocks of the image were already decoded. Its usage will become more clear below.
int nBlocks = 0;
Moving on to the Arduino setup, we will start by opening a serial connection, to be able to output results from our program.
Serial.begin(115200);
After that, we will mount the SPIFFS file system. This procedure is needed before we can interact with it. This is done with a call to the begin method on the SPIFFS extern variable, which returns true in case the procedure is successful and false otherwise. Naturally, in case the initialization fails, the rest of the program won’t work correctly, so we simply print a message to the serial port and finish the Setup function.
if (!SPIFFS.begin()) {
Serial.println("SPIFFS init failed");
return;
}
Serial.println("SPIFFS init finished");
Assuming the initialization of the file system is performed correctly, we will now obtain the width and the height of the image. To do so, we will first declare two variables to hold these values.
uint16_t imgWidth, imgHeight;
Then we will call the getFsJpgSize method on the TJpgDec object. This method receives as input the following arguments:
- The address of a uint16_t variable, where the width of the image will be written;
- The address of a uint16_t variable, where the height of the image will be written;
- The path to the file, as a string.
Note that this method call returns an enum of type JRESULT that should be equal to JDR_OK in case of success. For simplicity, we are not going to check for this value but, in a real application scenario, you should do an error check and act accordingly.
TJpgDec.getFsJpgSize(&imgWidth, &imgHeight, filePath);
Assuming everything went right, we will print the values we have obtained for the image width and height.
Serial.println("\nImage dimensions:");
Serial.printf("width = %d, height = %d\n", imgWidth, imgHeight);
Next we are going to call the setCallback method on the TJpgDec variable to register a callback function that will be executed during the decoding of the .jpg file. This callback function needs to follow the signature defined here. We will call it onDecodeBlock and check the implementation below.
TJpgDec.setCallback(onDecodeBlock);
Then, to begin the decoding of the image stored in the file system and have access to all the information we need to start displaying it, we simply need to call the drawFsJpg method.
Note however that the name of this method might be misleading because it doesn’t actually draw anything in any display. Instead, it will start the decoding of the image and invoke the callback function every time a block is decoded, passing as input of the function the parameters for that block. Nonetheless, it will be our responsibility to actually draw each of those blocks (we are not going to do that yet on this first code section).
As input, this method receives the following parameters:
- x position where the image should start to be drawn in the screen.
- y position where the image should start to be drawn in the screen.
- The path to the file in the file system, as a string.
Like mentioned before, the function won’t draw the image in the screen and thus the x and y parameters are used simply to offset the x and y positions of each decoded block. In our case, we are assuming that the image is the same size of the screen, so we will start at coordinates x = 0 and y = 0.
This method also returns a JRESULT enum value, which should be checked for errors in a real application scenario.
TJpgDec.drawFsJpg(0, 0, filePath);
The whole setup function can be seen in the snipet below.
void setup() {
Serial.begin(115200);
if (!SPIFFS.begin()) {
Serial.println("SPIFFS init failed");
return;
}
Serial.println("SPIFFS init finished");
uint16_t imgWidth, imgHeight;
TJpgDec.getFsJpgSize(&imgWidth, &imgHeight, filePath);
Serial.println("\nImage dimensions:");
Serial.printf("width = %d, height = %d\n", imgWidth, imgHeight);
TJpgDec.setCallback(onDecodeBlock);
TJpgDec.drawFsJpg(0, 0, filePath);
}
To finalize, we will check the signature of the callback function, which is invoked every time a block is decoded. Naturally, it is expected that a single image will be composed by multiple blocks, which means that this function will be invoked multiple times.
This function should return a Boolean value and receive the following parameters:
- x coordinate of the block, regarding the offset of x passed as input of the drawFsJpg method;
- y coordinate of the block, regarding the offset of y passed as input of the drawFsJpg method;
- Width of the block, in pixels;
- Height of the block, in pixels;
- The bitmap of the block.
bool onDecodeBlock(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap){
// Implementation of the callback function
}
In the implementation of this function we will simply increment the current block number and print it together with the x and y positions, width and height of the block.
nBlocks = nBlocks + 1;
Serial.println("\n----------");
Serial.printf("Nº of blocks %d\n", nBlocks);
Serial.printf("x = %d, y = %d\n", x, y);
Serial.printf("width = %d, height = %d\n", w, h);
Serial.flush();
We should return the value 1 (true) from this callback, so it keeps processing the remaining blocks. If we instead return 0 (false), the processing of the remaining blocks will stop. This feature is helpful since it allows us to stop the processing if we are trying to render an image bigger than our screen size, which we can check by comparing the x or the y coordinates of the block against the width and the height of the screen. For simplicity, we are assuming that the image we are going to render fits the screen and thus always return 1.
return 1;
The complete callback can be seen below.
bool onDecodeBlock(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap)
{
nBlocks = nBlocks + 1;
Serial.println("\n----------");
Serial.printf("Nº of blocks %d\n", nBlocks);
Serial.printf("x = %d, y = %d\n", x, y);
Serial.printf("width = %d, height = %d\n", w, h);
Serial.flush();
return 1;
}
The complete code can be seen below.
#include <TJpg_Decoder.h>
#include <SPIFFS.h>
const char* filePath = "/test.jpg";
int nBlocks = 0;
bool onDecodeBlock(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap)
{
nBlocks = nBlocks + 1;
Serial.println("\n----------");
Serial.printf("Nº of blocks %d\n", nBlocks);
Serial.printf("x = %d, y = %d\n", x, y);
Serial.printf("width = %d, height = %d\n", w, h);
Serial.flush();
return 1;
}
void setup() {
Serial.begin(115200);
if (!SPIFFS.begin()) {
Serial.println("SPIFFS init failed");
return;
}
Serial.println("SPIFFS init finished");
uint16_t imgWidth, imgHeight;
TJpgDec.getFsJpgSize(&imgWidth, &imgHeight, filePath);
Serial.println("\nImage dimensions:");
Serial.printf("width = %d, height = %d\n", imgWidth, imgHeight);
TJpgDec.setCallback(onDecodeBlock);
TJpgDec.drawFsJpg(0, 0, filePath);
}
void loop() {}
To test the code, simply compile it and upload it to your device, using the Arduino IDE. Upon running the code, open the IDE serial monitor. You should get an output similar to figure 1.
As can be seen, we have obtained the correct image dimensions (240 x 320 pixels) and multiple blocks information was printed. If we scroll through the blocks, we can check that a total of 300 are printed. Since the original image had 76800 pixels (240 x 230) and each block has 256 pixels (16 x 16), the division result is 300, which matches what we have obtained.
We can also check that the first block starts at x = 0 and y = 0 and then these coordinates are incremented by the block size in each direction at each time. This means that no block will overlap, as expected.
Rendering the jpg image on the display
We will start the code by the library includes. We will need our TJpg_Decoder.h, the SPIFFS.h (already used in the previous section) and the Arduino_GFX_Library.h, to be able to interact with the display.
#include <TJpg_Decoder.h>
#include <SPIFFS.h>
#include <Arduino_GFX_Library.h>
After the includes, we will define the constants that will hold the numbers of the pins of the ESP32 connected to the display. Naturally, these will depend to how you have wired your ESP32 to your display, so the values I’m going to use are the correct for my wirings (covered on this previous tutorial). If you have a different wiring, you should adapt these lines of code.
#define TFT_SCK 18
#define TFT_MOSI 23
#define TFT_MISO 19
#define TFT_CS 22
#define TFT_DC 21
#define TFT_RESET 17
Next, we will take care of initializing a display data bus. Since we are using the ESP32, we need to create a bus of class Arduino_ESP32SPI. The constructor of this class receives the number of most of the ESP32 pins connected to the display (except the reset pin).
Arduino_ESP32SPI *bus = new Arduino_ESP32SPI(TFT_DC, TFT_CS, TFT_SCK, TFT_MOSI, TFT_MISO);
Now that we have initialized our data bus, we are going to create an object of class Arduino_ILI9341. As input, the constructor of this class receives a pointer to our data bus and the number of the ESP32 pin connected to the reset pin of the display.
Arduino_ILI9341 *display = new Arduino_ILI9341(bus, TFT_RESET);
Moving on to the Arduino setup, we will start by opening a Serial connection. We will only need it for debugging purposes, to print a message indicated if SPIFFS was correctly initialized or not.
Serial.begin(115200);
Then we are going to take care of the SPIFFS initialization.
if(!SPIFFS.begin()) {
Serial.println("SPIFFS init failed");
return;
}
Serial.println("SPIFFS init finished");
After this we will do the initialization of our display. This is done with a call to the begin method on our Arduino_ILI9341 object. Note that since we have a pointer to the object, we need to use the -> operator.
display->begin();
After this we will register the callback function that will be executed whenever a block of the image is decoded, like we have checked in the previous section. We will analyze its implementation below, as it will be different this time.
TJpgDec.setCallback(onDecode);
Like before, we finish the Setup function with a call to the drawFsJpg method, passing both the offsets of x and y to be equal to zero and the path to the file on the SPIFFS file system.
TJpgDec.drawFsJpg(0, 0, "/test.jpg");
The complete setup function can be seen below.
void setup() {
Serial.begin(115200);
if(!SPIFFS.begin()) {
Serial.println("SPIFFS init failed");
return;
}
Serial.println("SPIFFS init finished");
display->begin();
TJpgDec.setCallback(onDecode);
TJpgDec.drawFsJpg(0, 0, "/test.jpg");
}
To finalize the code, we will check the implementation of the onDecode function. Its implementation will consist on writing each decoded block to the display, with a call to the draw16bitRGBBitmap method on our display object. It receives the following parameters:
- x coordinate of the top left corner where to start drawing;
- y coordinate of the top left corner where to start drawing;
- Byte array with a 16-bit color bitmap;
- Width of the bitmap in pixels;
- Height of the bitmap in pixels.
These map directly to the variables that are passed as input of the callback function.
display->draw16bitRGBBitmap(x, y, bitmap, w, h);
The complete callback can be seen below. Note that we are returning 1 because we know that the image fits our display, so we can process all the blocks of the image.
bool onDecode(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap)
{
display->draw16bitRGBBitmap(x, y, bitmap, w, h);
return 1;
}
The complete code can be seen below.
#include <TJpg_Decoder.h>
#include <SPIFFS.h>
#include <Arduino_GFX_Library.h>
#define TFT_SCK 18
#define TFT_MOSI 23
#define TFT_MISO 19
#define TFT_CS 22
#define TFT_DC 21
#define TFT_RESET 17
Arduino_ESP32SPI *bus = new Arduino_ESP32SPI(TFT_DC, TFT_CS, TFT_SCK, TFT_MOSI, TFT_MISO);
Arduino_ILI9341 *display = new Arduino_ILI9341(bus, TFT_RESET);
bool onDecode(int16_t x, int16_t y, uint16_t w, uint16_t h, uint16_t* bitmap)
{
display->draw16bitRGBBitmap(x, y, bitmap, w, h);
return 1;
}
void setup() {
Serial.begin(115200);
if(!SPIFFS.begin()) {
Serial.println("SPIFFS init failed");
return;
}
Serial.println("SPIFFS init finished");
display->begin();
TJpgDec.setCallback(onDecode);
TJpgDec.drawFsJpg(0, 0, "/test.jpg");
}
void loop() {}
Once again, to test the code, simply compile it and upload it to your ESP32, after having it wired to the display. You should get an output similar to figure 2. As can be seen, the .jpg image was rendered as expected.
Hi,
may I know what version of library you used?
I got error of: undefined reference to `BYTECLIP’
thanks
summer