Take a ScreenShot from ILI9341 Display

Status
Not open for further replies.

DIYLAB

Well-known member
Hi all,

I would very much like to take a screenshot of my display and send it serially to my PC for further processing with C#.
I use the ILI9341_t3n library and frame buffer that I set myself:

Code:
// Frame Buffer
DMAMEM uint16_t frontBuffer[240 * 320] = { 0 };

Then in the setup:

Code:
   // Display
    tft.begin();
    tft.setFrameBuffer(frontBuffer);
    tft.useFrameBuffer(true);

I would then send the contents of the buffer via serial:

Code:
Serial.write((uint8_t*)&frontBuffer, sizeof(frontBuffer));

Is this the right way or am I completely wrong?
 
Hi,
transferring the screen in one piece didn't work unfortunately, but line by line works very well.
So far no lost bytes, every shot a hit.

Code:
void ScreenShot() {
    uint8_t color[2 * 240]; // Line Buffer

     for (uint16_t y = 0; y < 320; y++) {
        tft.readRect(0, y, 240, 1, (uint16_t*)color);
        Serial.write(color, 2 * 240);
        delay(2);
    }
}

 
TWhat did you use on the PC side to work the image data?

Thank you for your interest!

In my program there is a lot of serial communication, there are about 300 properties in both directions, therefore here only for understanding.

I use as a basis for communication TeensySharp: https://github.com/luni64/TeensySharp
After the port is opened, I do not subscribe to the event that provides the data, but use the Win API BaseStream according to this article by Ben Voigt: https://www.sparxeng.com/blog/software/must-use-net-system-io-ports-serialport

This method is called once right after the port is opened and then returns a permanent stream of data:
Code:
       /// <summary>
        /// Continuous Read from Serial BaseStream
        /// </summary>
        private void ContinuousRead() {
            byte[] buffer = new byte[4096];
            Action kickoffRead = null;
            kickoffRead = () => port.BaseStream.BeginRead(buffer, 0, buffer.Length, delegate (IAsyncResult ar) {
                int count = port.BaseStream.EndRead(ar);
                byte[] dst = new byte[count];
                Buffer.BlockCopy(buffer, 0, dst, 0, count);
                this.Invoke((MethodInvoker)delegate { BytesReceived(dst); });
                kickoffRead();
            }, null); kickoffRead();
        }

Now the data is processed further.
I distinguish between text messages from the Teensy, which are wrapped in "<text>" and raw data, which are wrapped in "* ... #" packets. For the screenshot of course only the section "RAW Data Bytes" is interesting:

Code:
        /// <summary>
        /// Handle Serial Byte Stream
        /// </summary>
        /// <param name="dataBlock"></param>
        private void BytesReceived(byte[] dataBlock) {
            // Emcoding current byteblock and clean up.
            string readStr = Encoding.Default.GetString(dataBlock).Replace("\r", "").Replace("\n", "");

            // RAW Data Bytes
            if (readStr.StartsWith("*")) { addSerialBytes = true; listSerialBytes.Clear(); }
            if (addSerialBytes) listSerialBytes.AddRange(dataBlock);
            if (addSerialBytes && readStr.EndsWith("#")) HandleBytes(listSerialBytes);

            // Text Message
            if (readStr.StartsWith("<")) { addSerialMessage = true; sbSerialMessage.Clear(); }
            if (addSerialMessage) sbSerialMessage.Append(readStr);
            if (addSerialMessage && readStr.EndsWith(">")) WasMessage(sbSerialMessage.ToString());
        }

For text messages, continue here:

Code:
       /// <summary>
        /// Serial Data was a Message
        /// </summary>
        /// <param name="msg"></param>
        private void WasMessage(string msg) {
            addSerialMessage = false;

            string str = msg.Replace("<", "");
            string[] items = str.Split(new[] { ">" }, StringSplitOptions.RemoveEmptyEntries);

            foreach (string s in items) {
// Do something with the text
            }
        }

And if bytes come as a screenshot and the length fits, then the image is assembled here and displayed in a PictureBox:

Code:
        /// <summary>
        /// Handle Byte Stream from Serial
        /// </summary>
        /// <param name="dataBlock"></param>
        private void HandleBytes(List<byte> lst) {
            addSerialBytes = false;

            lst.RemoveAt(0); // Remove first Element '*'
            lst.RemoveAt(listSerialBytes.Count - 1); // Remove last Element '#'

            if (lst.Count == 240 * 320 * 2) {
                byte[] bytes = lst.ToArray();

                Bitmap bmp = new Bitmap(240, 320, PixelFormat.Format24bppRgb);
                BitmapData data = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
                int stride = data.Stride;
                unsafe {
                    byte* ptr = (byte*)data.Scan0;
                    int count = 0;
                    for (int y = 0; y < 320; y++) {
                        for (int x = 0; x < 240; x++) {
                            Color c = Utility.RGB565toRGB(BitConverter.ToUInt16(bytes, count));
                            ptr[(x * 3) + y * stride] = c.B;
                            ptr[(x * 3) + y * stride + 1] = c.G;
                            ptr[(x * 3) + y * stride + 2] = c.R;
                            if (count < 153588) count += 2;
                        }
                    }
                }
                bmp.UnlockBits(data);
                PictureBoxScreenShot.BackgroundImage = bmp;
            } else {
                WriteControllerLog("Sorry, screenshot is not possible.", Logging.LogType.error);
            }
        }

Here is the method to convert the 16bit colors:

Code:
        /// <summary>
        /// Converts RGB565 color to RGB color.
        /// </summary>
        /// <param name="rgb565"></param>
        /// <returns></returns>
        public static Color RGB565toRGB(ushort rgb565) {
            byte b = (byte)((rgb565 & 0x001F) << 3);
            byte g = (byte)((rgb565 & 0x07E0) >> 3);
            byte r = (byte)((rgb565 & 0xF800) >> 8);
            return Color.FromArgb(r |= (byte)(r >> 5), g |= (byte)(g >> 6), b |= (byte)(b >> 5));
        }

In the class you still need these lists and variables, which must be outside the methods:

Code:
      // Serial Handling
        private List<byte> listSerialBytes = new List<byte>();
        private StringBuilder sbSerialMessage = new StringBuilder();
        private bool addSerialMessage = false;
        private bool addSerialBytes = false;

So if you send the Teensy the command to execute a screenshot, it should call this method (teensy side):

Code:
/// <summary>
/// Take a screenshot from display.
/// </summary>
void ScreenShot() {
    uint8_t color[2 * 240]; // Line Buffer

    _LockScreenUpdate = true;
    Serial.print("*");
    for (uint16_t y = 0; y < 320; y++) {
        tft.readRect(0, y, 240, 1, (uint16_t*)color);
        Serial.write(color, 2 * 240);
        delay(2);
    }
    Serial.print("#");
    _LockScreenUpdate = false;
}

_LockScreenUpdate (bool) I use in Teensy to block the complete redraw until the screenshot is ready.

Have fun trying
Bruno
 
Status
Not open for further replies.
Back
Top