Send display framebuffer over USB as "camera"

neutron7

Well-known member
Is there some way to send the frame buffer of say, ILI9341 driver over USB as a "camera"? It would be a big help making an instructional video for my device, instead of trying to record with a real camera.
 
Yes, the ILI9341_t3 driver has a function readRect(x, y, width, height, buffer), which reads width×height uint16_t's from the framebuffer in RGB 565 (red most significant 5 bits, blue least significant 5 bits, green the middle 6 bits) format.

However, don't think of this as "camera", but as a framebuffer capture as a file.

I would use the USB Serial and the simple NetPBM PPM/P6 format for this –– but I use Linux and not Windows.

(Serial) evaluates to true when an application as the serial port open in Linux, and false when not. I would use this as the capture trigger.

I would use a global 15360-byte array, uint16_t fbcap[7680]; and a boolean flag bool fbcapped; (initialized to false) to indicate if the last iteration of the loop captured the framebuffer or not. The 7680 is "magic", because it is the smallest number divisible by 512, 320, and 240, so it can be used in either orientation of the display, and the buffer will contain an integral number of USB 2.0 datagrams.

In the main loop, the added code would probably look like (UNTESTED!)
Code:
    if (Serial) {
        if (!fbcapped) {
            fbcapped = true;
            if (tft.width() == 240 && tft.height() == 320) {
                Serial.write("P6\n240 320\n255\n", 15); // 15 bytes
                for (int yc = 0; yc < 320; yc += 32) {
                    tft.readRect(0, yc, 240, 32, fbcap);
                    for (int i = 0; i < 240*32; i++) {
                        Serial.write((unsigned char)((1053 * ((fbcap[i] >> 11) & 31)) >> 7));  // = 255*((fbcap[i]>>11)&31)/31
                        Serial.write((unsigned char)((4145 * ((fbcap[i] >> 5) & 63)) >> 10));  // = 255*((fbcap[i]>>5)&63)/63
                        Serial.write((unsigned char)((1053 * (fbcap[i] & 31)) >> 7));  // = 255*(fbcap[i]&31)/31
                    }
                    Serial.flush();
                }
            } else
            if (tft.width() == 320 && tft.height() == 240) {
                Serial.write("P6\n320 240\n255\n", 15); // 15 bytes
                for (int yc = 0; yc < 240; yc += 24) {
                    tft.readRect(0, yc, 320, 24, fbcap);
                    for (int i = 0; i < 320*24; i++) {
                        Serial.write((1053 * ((fbcap[i] >> 11) & 31)) >> 7);  // = 255*((fbcap[i]>>11)&31)/31
                        Serial.write((4145 * ((fbcap[i] >> 5) & 63)) >> 10);  // = 255*((fbcap[i]>>5)&63)/63
                        Serial.write((1053 * (fbcap[i] & 31)) >> 7);  // = 255*(fbcap[i]&31)/31
                    }
                    Serial.flush();
                }
            }
        }
    } else {
        fbcapped = false;
    }
Each capture is a NetPBM PPM image, 320×240 or 240×320, with a 15-byte header; thus 15+3×320×240 = 230415 bytes long.
After the data has been written to Serial, the program reading the USB Serial device in Linux has to close it (so Serial will evaluate to false), and a program open it again, for a new capture to occur. The no-program-has-it-open interval has to be long enough for two full iterations of the main loop, though, or the capture trigger will be missed.

Division by 31 or 63 is a bit hard for the compiler to optimize, so I replaced 255*rb/31 with 1053*rb/128, and 255*g/63 with 4145*g/1024, which are easy to optimize for the compiler, but provide the exact same results for rb=0..31 and g=0..63. This is just to scale the color components to the full 0..255 range.

If /dev/ttyUSB0 is the Teensy USB Serial port device in Linux, then running
    dd if=/dev/ttyUSB0 bs=230415 count=1 | pnmtopng -compress 9 > imagename.png
captures the tft framebuffer, saving it as imagename.png (lossless full-color PNG file).
(You need to have the netpbm package installed in linux, for the pnmtopng utility.)

Or, I could run e.g.
Code:
I=0; while read dummy ; do ((I++)); name=$(printf '%05f.png' $I); dd if=/dev/ttyUSB0 bs=230415 count=1 | pnmtopng -compress 9 > $name ; echo "Saved $I" ; done
which would save 00001.png, 00002.png, and so on, taking a capture every time I press Enter. Ctrl-D would stop the captures.
 
Back
Top