Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 12 of 12

Thread: Uncanny Eyes is getting expensive

  1. #1
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,275

    Cool Uncanny Eyes is getting expensive

    As some/many of you know, I have this thing for the Adafruit uncanny eyes project, tracking it from using it on a Teensy 3.2, 3.5, 3.6, Teensy 4.0, and Teensy 4.1. In the 3.x era, I used the 128x128 TFT and OLED displays (both from Adafruit and elsewhere). With the Teensy 4.0 and 4.1, I now use the 240x240 IPS displays, both from Adafruit and elsewhere.

    Recently there was discussion about the round eye displays, and needless to say, I needed to get some. KurtE and mjs513 (maybe defragster also) have modified the ILI9341_t3 (and ST7789_t3/ST7789_t3n) library to work with the waveshare round eye displays. So naturally I needed to update my set of displays. I got it, and it works well (creepy of course, but that is the nature of the beast).

    I'm in the middle of modding a staff from Spirit Halloween to have an interactive display. While at the moment, I'm using a Hallowing M4 for the project (due to the eye that I was using is not yet ported to Teensy), I would like to move it to the Teensy and the round eye display. I looked around ebay for suppliers to buy an additional round didplay. It looks like the stock of US suppliers is down from when I ordered in the past. But I discovered a Canadian supplier, which had a new variant of the display. So I ordered another 2 displays from them.

    I decided I need to combine the pin-outs for the various SPI displays so that I could solder up a prototype board and easily move from display to display without having to dedicate a Teensy for each display (or change the pin connections). Unfortunately in moving the connections around, I mis-plugged one of the Adafruit 240x240 square displays, and wound up burning it out. So I went to Adafruit and ordered a replacement.

    Finally after finalizing the sketch for the square display, I decided to do the round display next. In setting it up, I noticed I had cracked one of the displays, so I destroyed two displays in one day (or more likely, I had cracked the round display earlier, and I just noticed it today).

    If you don't know what I'm talking about, the original uncanny eyes for the Teensy 3.2 is at:


    Note, the author has stopped working with Teensies and is now concentrating on other processors (after the Teensy he went to the Raspberry Pi, and then onto the Adafruit M4 processors, including the Hallowing M0 with the 128x128 display and Hallowing M4/Monster M4SK with the 240x240 display).

    After the Teensy 4.0 release, the above people reworked the uncanny eyes for the Teensy 4.x processors. The ST7735_t3 library has both the original uncanny Eyes (uncannyEyes7735) for 128x128 displays as well as the Teensy 4.x version for 240x240 displays (uncannyEyes_async_st7789_240x240). The GC9A010A driver on github has its version of the program (uncannyEyes_GC9A01A).

    Thread that announced the new driver:


    <edit>
    In terms of USA suppliers, unlike 2-3 years ago, none of the cheap 240x240 square displays (without the CS pin) seem to be sold by USA dealers. In pre-supply chain shortages, these were fairly plentiful. It looks like you can still get the displays if you are willing to order from China and wait for the shipping. You can get the Adafruit square displays (the 1.3" and 1.54") from Adafruit and the distributors.

    When I ordered the round displays previously in June, they were also more plentiful, but now the stock seems to be tighter.
    Last edited by MichaelMeissner; 09-10-2022 at 07:50 AM.

  2. #2
    I run a pair of Uncanny Eyes on a 10" LCD (ER-TFTM10-1) with an RA8876 controller using a Teensy 3.6. I'm getting about 50 frames a second for both 128x128 eyes by only outputting rows that have data. When building a eye's row I mark a boolean true if a pixel is non-zero. When sending out the data to the controller I output the line only if the boolean is true or if the boolean was true the last time the same line & eye was written. I then save the boolean in an array, one boolean per row per eye. If you have gobs of memory you could save the scanline after outputting and compare it next time through, outputting the new scanline only if it has changed. That doubles the memory needed for the eye, so I thought one boolean per scanline was a good compromise.

  3. #3
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,275

    Cool

    @dundakitty: Sounds like a reasonable optimization for the driver (I assume you did it in the driver and not in the uncanny eyes code). You could use less memory by instead of using a single byte for the boolean for each row, use an uint8_t variable and do shifting/masking to make each byte hold 8 rows modified status.

    Of course it depends how much each row changes from display to display.

    On the chance people are interested in what my pinout scheme is, there were some design considerations:
    • Each pinout has 10 pins: Ground, power, clock, MOSI, MISO, main CS pin, D/C pin, reset pin, secondary CS pin, blink pin. I put the 10 pins in a standard layout. I use a display specific cable to hook up the display to those pins so switching displays is a matter of switching just a cable. The blink pin is often just wired to 3.3v.
    • I color code the wires, fortunately there are 10 standard wire colors.
    • The secondary CS pin should be able to do PWM, analog read, as well as being a serial TX pin to run WS2812Serial on it (the intention is if you aren't reading data from a micro SD card, you might want to hook up a servo, potentiometer, and/or neopixel to that pin.
    • The first display should only use pins 0..23, so that you can use a Teensy 4.0 without soldering wires underneath when using a single display;
    • The second display should use pins common to the Teensy 4.0 and 4.1 (pins 0..33). This means using pin 1 as the MISO pin for the secondary display and not pin 39 as the alternate MISO pin. It assumes that a Teensy 4.0 being used has pins 24..33 in the same position as with the Teensy 4.1.
    • I avoided pins used by the audio adapter other than the standard I2C and SPI pins (6, 7, 8, 10, 20, 21, 23).
    • I avoided pin 3 since I often use that as a common push button pin.
    • I avoided pin 17/A3 since I often use that as my neopixel pin.
    • I avoided pin 2 since sometimes I have used that as the I2C interrupt pin (the prop shield also uses pin 2 in this fashion).
    • I left 2 standard pins for use with analog read (pins 15/A1 and 16/A2) -- note the audio adapter also uses pin 15/A1 for analog inputs to allow a potentiometer to be soldered to the board.
    • I left 2 pins for one Serial UART (pins 28 and 29 for Serial7). Serial1 (pins 0 and 1) could also be used if you are just using one display.


    The pins I chose are:
    • Power (red wire): Typically 3.3v (optionally VIN)
    • Ground (black wire): Ground
    • Clock (green wire): pin 13 and pin 27/A13 (required)
    • MOSI (purple wire): pin 11 and pin 26/A12 (required)
    • MISO (gray wire): pin 12 and pin 1 (pin 1 is required for Teensy 4.0 operation)
    • Standard CS (yellow wire): pin 4 and pin 0
    • D/C (brown wire): pin 5 and pin 22/A8
    • Reset (blue wire): pin 9 and pin 25/A11
    • Secondary CS (white wire): pin 14/A0 and pin 24/A10
    • Blink (orange wire): Typically hard wired to 3.3v.


    In the potential designs for protoboard layout, many of the pins can be overridden by jumper wires, so I'm not locked into those defaults. The power can be switched between 3.3v and VIN. The standard clock, MOSI, and first display MISO are fixed, but the second display MISO can be jumpered to pin 39.
    Last edited by MichaelMeissner; 09-10-2022 at 07:05 PM.

  4. #4
    Oh wow, this looks like a fun project, thanks for bringing it to my attention. I might order a couple of these and see how it goes. I don't suppose anyone's tried adding a camera and face detection via e.g. OpenCV so the eyes can follow people's faces? Imagine a costume/mask with couple of these eyes out on stalks covered with these that could look at people...

  5. #5
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,275
    Quote Originally Posted by chris.nz View Post
    Oh wow, this looks like a fun project, thanks for bringing it to my attention. I might order a couple of these and see how it goes. I don't suppose anyone's tried adding a camera and face detection via e.g. OpenCV so the eyes can follow people's faces? Imagine a costume/mask with couple of these eyes out on stalks covered with these that could look at people...
    The Adafruit learning guide has a howto on how to add PIR (infrared heat sensor) support to the original uncanny eyes (that supported the Teensy 3.2) to build a skull where the eyes tracks you:


    And as I've said elsewhere, now that we have the MTP support, it would be nice if we could retrofit the new code (that uses flash memory to hold the bitmaps to be used, and you can change which eye to use without recompiling it). The trouble is the new code changed how the display is updated to use the M4 zero DMA support, and for the Teensy 4.x support, I would imagine the DMA is quite different.

  6. #6
    Quote Originally Posted by MichaelMeissner View Post
    @dundakitty: Sounds like a reasonable optimization for the driver (I assume you did it in the driver and not in the uncanny eyes code). You could use less memory by instead of using a single byte for the boolean for each row, use an uint8_t variable and do shifting/masking to make each byte hold 8 rows modified status.

    Of course it depends how much each row changes from display to display.
    ...
    My project is a Literary Clock, see https://www.youtube.com/watch?v=YcSkkDHdfg0
    On Halloween there is a one-in-ten chance that the uncanny eyes code is activated. The text is drawn first, then a few seconds later a skull is drawn in the center of the display and two eyes are animated for the remainder of the minute. The screen is then cleared and a new literary quote is displayed, possibly followed by the skull.

    The optimization is in the uncanny eye code, not the RA8876 driver. This allows the optimization to run separately per eye, as each eye is only updating a portion of the 1024 x 600 10" display.
    The following code should look familiar:
    Code:
    typedef struct {        // Struct is defined before including config.h --
      int8_t  wink;         // and wink button (or -1 if none) specified there,
      uint8_t rotation;     // also display rotation.
      int16_t x_off;
      int16_t y_off;
    } eyeInfo_t;
    
    #include "eye_config.h"
    #define NUM_EYES (sizeof eyeInfo / sizeof eyeInfo[0])
    
    // A simple state machine is used to control eye blinks/winks:
    #define NOBLINK 0       // Not currently engaged in a blink
    #define ENBLINK 1       // Eyelid is currently closing
    #define DEBLINK 2       // Eyelid is currently opening
    typedef struct {
      uint8_t  state;       // NOBLINK/ENBLINK/DEBLINK
      uint32_t duration;    // Duration of blink state (micros)
      uint32_t startTime;   // Time (micros) of last state change
    } eyeBlink;
    
    struct {                // One-per-eye structure
      eyeBlink     blink;   // Current blink/wink state
    } eye[NUM_EYES];
    
    boolean lineNotBlank[NUM_EYES][SCREEN_HEIGHT];
    
    void drawEye( // Renders one eye.  Inputs must be pre-clipped & valid.
      uint8_t  e,       // Eye array index; 0 or 1 for left/right
      uint16_t iScale,  // Scale factor for iris (0-1023)
      uint16_t  scleraX, // First pixel X offset into sclera image
      uint16_t  scleraY, // First pixel Y offset into sclera image
      uint8_t  uT,      // Upper eyelid threshold value
      uint8_t  lT) {    // Lower eyelid threshold value
    
      uint8_t  screenX, screenY, er;
      boolean notBlank;
      uint16_t scleraXsave;
      int16_t  irisX, irisY;
      uint16_t p, a;
      uint32_t d;
    
      uint16_t colors[2][SCREEN_WIDTH];
      uint16_t *bp;
    
      uint32_t irisThreshold = (SCREEN_WIDTH * (1023 - iScale) + 512) / 1024;
      uint32_t irisScale     = IRIS_MAP_HEIGHT * 65536 / irisThreshold;
    
      // Set up raw pixel dump to entire screen.  Although such writes can wrap
      // around automatically from end of rect back to beginning, the region is
      // reset on each frame here in case of an SPI glitch.
    
      scleraXsave = scleraX; // Save initial X value to reset on each line
      irisY       = scleraY - (SCLERA_HEIGHT - IRIS_HEIGHT) / 2;
      er = eyeInfo[e].rotation;
      notBlank = false;
      for (screenY = 0; screenY < SCREEN_HEIGHT; screenY++, scleraY++, irisY++) {
        bp = colors[screenY & 1];
        scleraX = scleraXsave;
        irisX   = scleraXsave - (SCLERA_WIDTH - IRIS_WIDTH) / 2;
        for (screenX = 0; screenX < SCREEN_WIDTH; screenX++, scleraX++, irisX++) {
          if ((lower[screenY][screenX] <= lT) ||
              (upper[screenY][screenX] <= uT)) {             // Covered by eyelid
            p = 0;
          } else if ((irisY < 0) || (irisY >= IRIS_HEIGHT) ||
                     (irisX < 0) || (irisX >= IRIS_WIDTH)) { // In sclera
            p = sclera[scleraY][scleraX];
          } else {                                          // Maybe iris...
            p = polar[irisY][irisX];                        // Polar angle/dist
            d = p & 0x7F;                                   // Distance from edge (0-127)
            if (d < irisThreshold) {                        // Within scaled iris area
              d = d * irisScale / 65536;                    // d scaled to iris image height
              a = (IRIS_MAP_WIDTH * (p >> 7)) / 512;        // Angle (X)
              p = iris[d][a];                               // Pixel = iris
            } else {                                        // Not in iris
              p = sclera[scleraY][scleraX];                 // Pixel = sclera
            }
          }
    
          if (er != 0) {
            bp[(SCREEN_WIDTH-1)-screenX] = p;
          } else {
            bp[screenX] = p;
          }
          if (p != 0) notBlank = true;
        } // end column
        if (notBlank || lineNotBlank[e][screenY]) {
          while(tft.activeDMA) delayMicroseconds(20);
          tft.writeRect(eyeInfo[e].x_off, eyeInfo[e].y_off+screenY, SCREEN_WIDTH, 1, bp);
        }
        lineNotBlank[e][screenY] = notBlank;
      } // end scanline
    }
    I agree, if you're animating many eyes it would be more space efficient to use a uint8_t (or uint16_t if needed) instead of a boolean for lineNotBlank, then use a bitmask per eye (1>>eye).

  7. #7
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,275
    For the record, I paid for the round eyes from the Canadian supplier on Thursday September 1st, and they arrived today (Monday September 12th). It might have shaved a few days off the delivery if there had been a USA supplier, but it still beats the average Asian suppliers in terms of delivery speed..

    Both eyes look great and work well. For debugging, it is nice that the eyes have 7 male pins that I can plug directly into a breadboard with 36 rows of pins. Ultimately of course, I will need to use cables to install the displays, but it is nice for debugging to keep things altogether. The waveshare board had a 2mm connector on the board, and 8 separate female 0.1" pins. From a mounting perspective, it is better to have two holes in the new display that I can easily drill through, rather than having M2 screws that I have to line up when drilling the mounting holes.

  8. #8
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,275

    Cool

    And my luck is continuing. I now have moved Uncanny Eyes code into a separate library that supports both the ST7789 (240x240 square display) and GC9A01A (240x240 round display). After I got the 2nd Adafruit display and I verified both of my Adafruit displays now work, I dug out the 2 non-CS boards that I had and I attached them, changing the CS to be -1.

    One display works great. The other display corrupts the bottom 1/2 of the screen. But with the working display, I verified that it does work also.

    So I now have separate .ino sketches that pull in the library:

    • 2 GC9A01A round displays;
    • 1 GC9A01A round display;
    • 2 ST7789 square displays, both with a CS pin;
    • 2 ST7789 square displays, one with a CS pin, one without;
    • 2 ST7789 square displays, both without a CS pin;
    • 1 ST7789 square display with a CS pin; (and)
    • 1 ST7789 square display without a CS pin.


    I discovered that the arguments to GC9A01A's displayType constructor are in a different order to the ST7789 displayType constructor. This means if you call the ST7789 class constructor with GC9A01A argument order, it will crash when initializing the displays:

    Code:
      // Initialize eye objects based on eyeInfo list in config.h:
      for (e = 0; e < NUM_EYES; e++) {
        Serial.print("Create display #"); Serial.println(e);
    #if USE_GC9A01A
        //eye[e].display     = new displayType(&TFT_SPI, eyeInfo[e].cs,
        //                       DISPLAY_DC, -1);
        //for SPI
        //(TFT_CS, TFT_DC, TFT_MOSI, TFT_SCLK, TFT_RST);
        eye[e].display = new displayType(eyeInfo[e].cs, eyeInfo[e].dc, eyeInfo[e].rst,
                                         eyeInfo[e].mosi, eyeInfo[e].sck);
    #endif
    
    #if USE_ST7789
        //eye[e].display     = new displayType(&TFT_SPI, eyeInfo[e].cs,
        //                       DISPLAY_DC, -1);
        //for SPI
        //(TFT_CS, TFT_DC, TFT_MOSI, TFT_SCLK, TFT_RST);
        eye[e].display = new displayType(eyeInfo[e].cs, eyeInfo[e].dc,
                                         eyeInfo[e].mosi, eyeInfo[e].sck, eyeInfo[e].rst);
    
        // ...
      }
    Similarly, how you start up the display is different:

    Code:
      // After all-displays reset, now call init/begin func for each display:
      for (e = 0; e < NUM_EYES; e++) {
    
    #if USE_GC9A01A
        eye[e].display->begin();
        Serial.printf("Init GC9A01A display #%d, rotation %d\n",
    		  e, eyeInfo[e].rotation);
    #endif
    
    #if USE_ST7789
        // Try to handle the ST7789 displays without CS PINS.
        if (eyeInfo[e].cs < 0)
          eye[e].display->init(240, 240, SPI_MODE2);
        else
          eye[e].display->init();
    
        Serial.printf("Init ST7789 display #%d, rotation %d\n",
    		  e, eyeInfo[e].rotation);
    #endif
    
        eye[e].display->setRotation(eyeInfo[e].rotation);
      }
      Serial.println("done");
    Originally in creating the library, I moved the main parts of the code into separate .cpp files. But I discovered it doesn't work, because a lot of the font functions will get duplicate function messages from the linker, as each of the drivers wants to pull in the library functions, and each has the same name. Of course, I'm probably the one person crazy enough to want two separate displays in a program that have different drivers. So I just made the code into a .h file, and the .ino sketch using #include to bring it in.

    One other thing I learned (which is obvious once I thought about it). Originally, I had all of the constants (sizes, pin numbers, etc.) as 'extern const' instead of just using '#define' or normal const so that I could set these in the .ino file and call library. But it makes the code a little slower, since the compiler can't do the constant optimization. So when I moved the drivers back to being in .h files, and removed the 'extern const' declarations, it ran a little faster (by 1-2 fps).

    The GC9A01A display is faster than the ST7789 display. I suspect this is due to the GC9A01a driver not sending pixels that won't be displayed.

    Another thing that I noticed is if you run 2 eyes, the fps rate is about double as with 1 eye, That is because on the Teensy 4.x it uses asynchronous updates with DMA, and the Teensy can go and work on the setting up for the second eye display while the data is still being transferred to the first display. With a single eye, it has to wait for the eye to finish before returning from loop.

  9. #9
    Have you tried the "skip blank scanline" optimization? I'm curious what speed up you see.

  10. #10
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,275
    Quote Originally Posted by dundakitty View Post
    Have you tried the "skip blank scanline" optimization? I'm curious what speed up you see.
    No unless the ST7789_t3 or GC9A01A libraries do it. I haven't really dug into the code, I've mostly just been re-packaging it so that I could more easily change which pins are used, etc. The code I'm using right now just does a drawpixel and lets the library handle updating the display via DMA. The original Uncanny Eyes code for the 128x128 displays did use the lower level details to optimize things via SPI.

    Do you have a pointer to the optimization?

    There is a complete rewrite of the code for the Adafruit M4 processors (specifically the Hallowing M4 and the Monster M4SK boards) that uses the M4 Zero DMA code to build the display. Each eye is handled separately, rather than doing both eyes, and waiting until the transfer is done. Ultimately, it would be nice if we could use the high level code, and possibly switch to the Teensy libraries. But I'm not sure I want to dive into at that level.

  11. #11
    The M4-specific code is much more complex than the original. The blank-scanline optimization helps during eye blinks. The optimization is already partially present in the M4 version, handling the upper eyelid. I'll look at adding additional code for the lower eyelid and for the case of eyelid color not black.

  12. #12
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,275
    Quote Originally Posted by dundakitty View Post
    The M4-specific code is much more complex than the original. The blank-scanline optimization helps during eye blinks. The optimization is already partially present in the M4 version, handling the upper eyelid. I'll look at adding additional code for the lower eyelid and for the case of eyelid color not black.
    Great, thanks.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •