Specifically, it is to perform JPEG decompression and RGB888 → RGB565 conversion at once (not analyzed.

The source code released this time is written using general arduino and M5 Stack library calls.įor example, if you write the process performed by M5.Lcd.drawJpg () yourself, you may be able to speed up a little more. Please check how the frame rate changes depending on the video quality. In the sample program, pressing the left button allows the video quality to be switched in the order of 50%, 80%, 95%. In practice, JPEG decompression time is extended and the frame rate is reduced due to the effect of increasing the video quality. Looking at this image, you may misunderstand that increasing the video quality (even increasing the data size of one frame) does not affect the frame rate because Core1 has room. The above image is the result of measuring the processing time with an oscilloscope when the video quality is 50% (the data size of one frame is about 9KB). The complete source code is in “ThetaPreview”. Refer to the file “ThetaPreview.ino” that contains the main routine of each core to understand the overall process flow.

Of the dual cores, Core1 is dedicated to communication(command transmission and response waiting by HTTPClient, data reading), and Core0 is dedicated to key input reading and display(JPEG decompression, RGB888 → RGB565 conversion and TFT output are performed by M5.Lcd.drawJpg() ).

Double buffering makes it possible to utilize the dual core of ESP32. In addition, you can now have a static double buffer in RAM. In addition, reception time and JPEG decompression time have been reduced. This measure eliminates the need for M5Stack resizing.

When requesting one frame of data, you can specify the pixel size and compression quality of the image.

This measure eliminates the need for M5Stack to read streams that cannot be processed. The measures implemented and their effects are as follows. In the case of complete darkness, it is about 2 to 4 fps.) In the case of slightly dark and low contrast, it is about 10 to 12 fps. (The result varies depending on the scene. In addition, by utilizing the dual core of ESP32, the live-preview can be displayed at around 9.5fps in most cases.

To solve this task, I used this THETA plug-in that extended the live-preview command. In addition, it is difficult to maintain the live preview for a long time. (I think this also requires processing time on the order of tens of ms.)Įven if you can write M5Stack code well, it will only be displayed at 2fps. Although measurement was not possible, processing time on the order of tens of ms is necessary.) (Data of 640 x 320 pixels is converted to data of 320 x 160 pixels. (It depends on the original data and compression ratio, but it takes about 200ms) If the reception is delayed for a long time, THETA will stop the live-preview.) It is actually a stream, so you need to keep reading data even if you can not display it. (Even if you just read one frame, it takes about 100ms. The specific processing contents and processing time are estimated as follows. Considering the Wi-Fi communication speed, CPU clock, and RAM capacity of M5 Stack (ESP32), it turns out that it is difficult to display live-preview at a practical frame rate. The data size of one frame in this case is about 46KB in most cases. Live-preview output by THETA in the product state is MOTION JPEG of 640 x 320 pixels 8fps even if you choose to minimize the load on the receiving side. Even if the shooting operation is performed on the THETA side, the state is reflected on the M5 Stack side.ĭetails on THETA live-preview Task to solve.Center button can be done shooting instruction to the THETA.Right button can change LCD brightness.Left button can change the video quality (compression ratio).With the new M5 Stack Core, the battery level is displayed.If you have already paired with THETA and have activated the THETA Plug-in that supports extended live-preview, the connection will be established within a few seconds.This sample program also has the following features. A sample program that displays a live-preview at a practical frame rate on the M5 Stack by using this THETA Plug-in together.

Framer js arduino