Teensy 4.1 : Code priority advice. processing/lcd gui

Status
Not open for further replies.

AndyCap

Well-known member
Hi Guys,

I wonder if anyone can help me.

I am building a MIDI router/processor, supporting DIN (serial), USB Host and USB Device.

I also want an LCD for the GUI, currently using an ILI9341/XPT2046.

The MIDI routing needs to run with minimal latency and jitter, it is very high priority, the GUI is much lower priority.

So I started out with Kurts wonderfull ILI9341_t3n lib which provides async api to the display, this gets rid of any blocking there.

Then I looked at finding some pre-rolled gui stuff and started looking at LGVL.

I have LGVL working on the Teensy 4.1 using ILI9341_t3n and async SPI, example code included at bottom.

The issue is the call to process LVGL can take quite a while, this time is not caused by the flushing to the frame buffer in DisplayFlush() which is very quick but elsewhere in the LVGL lib and there doesn't seem to be much you can do about this.

So what I am looking at is a way of processing my MIDI at a higher priority than the call to lv_task_handler(), I'm guessing using interrupts is a no go so are there any other ways?

It would be nice to use LGVL, but if I cannot get over this I will need to roll my own :(

Cheers

Andy

Code:
#include <Arduino.h>
#include <lvgl.h>
#include <ILI9341_t3n.h>
#include <XPT2046_Touchscreen.h>

#define TFT_DC 9
#define TFT_CS 10
#define TS_CS_PIN 8
#define TS_TIRQ_PIN 2
#define THREAD_TIME 100
#define FB_BUF_SIZE (320*240)
#define LVGL_BUF_SIZE (320*10)

// touchscreen
XPT2046_Touchscreen g_touchScreen(TS_CS_PIN, TS_TIRQ_PIN); 
ILI9341_t3n         g_tft = ILI9341_t3n(TFT_CS, TFT_DC);
DMAMEM uint16_t     fbTft[FB_BUF_SIZE];

// LVGL
lv_disp_buf_t       g_lvDispBuf;
DMAMEM lv_color_t   g_lvColorBuffer[LVGL_BUF_SIZE];
lv_disp_drv_t       g_lvDisplayDriver;
lv_obj_t            *g_plvBtn1 = NULL;
lv_obj_t            *g_plvLeftLed = NULL;
lv_obj_t            *g_plvRightLed = NULL;

// True if left LED is lit
bool g_bSignalLeft = true;

////////////////////////////////////////////////////////////////////////////////////
// Flush the LVGL data to the framebuffer
////////////////////////////////////////////////////////////////////////////////////
void DisplayFlush(lv_disp_drv_t *pDisplayDriver, const lv_area_t *pArea, lv_color_t *pColor)
{
  uint32_t w = (pArea->x2 - pArea->x1 + 1);
  uint32_t h = (pArea->y2 - pArea->y1 + 1);

  g_tft.waitUpdateAsyncComplete(); // just incase!

  g_tft.writeRect(pArea->x1, pArea->y1, w, h, &pColor->full);

  lv_disp_flush_ready(pDisplayDriver);
}

////////////////////////////////////////////////////////////////////////////////////
// Handle the touch screen
////////////////////////////////////////////////////////////////////////////////////
bool HandleTouchscreen(lv_indev_drv_t *pIndevDriver, lv_indev_data_t *pData)
{
  boolean touched = g_touchScreen.tirqTouched() && g_touchScreen.touched();

  if (!touched)
  {
    pData->state = LV_INDEV_STATE_REL;
  }
  else
  {
    pData->state = LV_INDEV_STATE_PR;

    TS_Point p = g_touchScreen.getPoint();
    uint16_t touchX = p.x / 12;
    uint16_t touchY = p.y / 16;

    if (touchX > 320 || touchY > 240)
    {
      Serial.printf("Outside bounds (%u, %u)\n", touchX, touchY);
    }
    else
    {
      /*Set the coordinates*/
      pData->point.x = touchX;
      pData->point.y = touchY;
    }
  }

  return false; 
}

////////////////////////////////////////////////////////////////////////////////////
// Handle LGVL events
// When button pressed swap lit LED.
////////////////////////////////////////////////////////////////////////////////////
void event_handler(lv_obj_t *pObj, lv_event_t event)
{
  if (event == LV_EVENT_CLICKED)
  {
    g_bSignalLeft = !g_bSignalLeft;

    if (g_bSignalLeft)
    {
      lv_led_on(g_plvLeftLed);
      lv_led_off(g_plvRightLed);
    }
    else
    {
      lv_led_on(g_plvRightLed);
      lv_led_off(g_plvLeftLed);
    }
  }
}

////////////////////////////////////////////////////////////////////////////////////
// Setup
////////////////////////////////////////////////////////////////////////////////////
void setup()
{
  Serial.begin(115200);

  g_touchScreen.begin();
  g_touchScreen.setRotation(1);

  g_tft.begin(100000000);
  g_tft.setRotation(1); 
  g_tft.fillScreen(ILI9341_BLACK);
  g_tft.useFrameBuffer(true);
  g_tft.setFrameBuffer(fbTft);
  g_tft.updateChangedAreasOnly(true);

  lv_init();
  lv_disp_buf_init(&g_lvDispBuf, g_lvColorBuffer, NULL, LVGL_BUF_SIZE);
 
  lv_disp_drv_init(&g_lvDisplayDriver);
  g_lvDisplayDriver.hor_res = 320;
  g_lvDisplayDriver.ver_res = 240;
  g_lvDisplayDriver.flush_cb = DisplayFlush;
  g_lvDisplayDriver.buffer = &g_lvDispBuf;
  lv_disp_drv_register(&g_lvDisplayDriver);

  lv_indev_drv_t indexDriver;
  lv_indev_drv_init(&indexDriver);
  indexDriver.type = LV_INDEV_TYPE_POINTER;
  indexDriver.read_cb = HandleTouchscreen;
  lv_indev_drv_register(&indexDriver);

  g_plvBtn1 = lv_btn_create(lv_scr_act(), NULL);
  lv_obj_set_event_cb(g_plvBtn1, event_handler);
  lv_obj_align(g_plvBtn1, NULL, LV_ALIGN_CENTER, 0, 0);

  lv_obj_t *label = lv_label_create(g_plvBtn1, NULL);
  lv_label_set_text(label, "Switch LEDs");

  g_plvLeftLed = lv_led_create(lv_scr_act(), NULL);
  lv_obj_align(g_plvLeftLed, NULL, LV_ALIGN_CENTER, -80, -80);
  lv_led_on(g_plvLeftLed);

  g_plvRightLed = lv_led_create(lv_scr_act(), NULL);
  lv_obj_align(g_plvRightLed, NULL, LV_ALIGN_CENTER, 80, -80);
  lv_led_off(g_plvRightLed);
}

void loop()
{
  static elapsedMillis elapsedMS;

  // do not process LGVL when DMA is active
  if(!g_tft.asyncUpdateActive())
  {
    elapsedMicros m;
    lv_task_handler(); 

    if(m>10)
      Serial.printf("lv_task_handler %uus\n", (uint32_t)m); Serial.flush();
  }

  // 30 ms refresh rate
  if(elapsedMS > 30)
  {
    g_tft.updateScreenAsync();
    elapsedMS = 0;
  }
}


Sample output:
Code:
lv_task_handler 86us
lv_task_handler 86us
lv_task_handler 5271us
lv_task_handler 41us
lv_task_handler 911us
lv_task_handler 871us
lv_task_handler 870us
lv_task_handler 870us
lv_task_handler 870us
lv_task_handler 580us
lv_task_handler 5114us
lv_task_handler 972us
lv_task_handler 2545us
lv_task_handler 925us
lv_task_handler 885us
lv_task_handler 888us
lv_task_handler 872us
lv_task_handler 871us
lv_task_handler 871us
lv_task_handler 872us
 

Attachments

  • lgvl.jpg
    lgvl.jpg
    75.1 KB · Views: 85
I'm guessing using interrupts is a no go so are there any other ways?

Not necessarily. You could use an IntervalTimer to periodically call your midi handler. Just set the priority to 255 (lowest). It will still interrupt your display code but can be interrupted by any system interrupt. Thus, it won't do any harm if it runs for some significant time. Just make sure to set the IntervalTimer call period larger than the execution time to give the 'non interrupt' code (which basically is only the code in loop() & yield()) a chance to execute.
 
Last edited:
Hi,

Thanks for the reply.

I was just a little worried about calling all the midi functions from within an interrupt, I will give it a go though and see how it goes...

Cheers

Andy
 
In case anyone is interested it looks like the code is able to run nicely at 50fps on the Teensy4:
 
Very cool - is that with the MIDI code running in _isr()?

As long as only the _isr() touches the midi it seems it should have a chance - not knowing midi or if there is anything involved that would try to use the same hardware if just UART serial?
 
No the midi code is not there, its just a demo from the LGVL examples.

I did test Luni's idea of using the interval timer, just to see it was called correctly and the LGVL stuff still worked.

Next test is to check that the calls to the midi stuff work correctly from the isr, there are three areas:

usbMidi : Midi message to and from the computer
SerialMIDI/MidiInterface : Uart based Midi.
MIDIDevice : Midi messages to and from devices plugged into the USB Host port.

If that all works from the isr for a simple example all will be good I think.

I have a another question, but first some background:

A midi port contains 16 midi channels.

There will be 8 ports for each of the three types mentioned above, so 24 ports each with 16 channels of midi. We have one set of these for inputs and one set for outputs.

So for basic routing these input port/channels can be connected to one or many output port/channels.

So if I had a Midi Keyboard plugged onto one of the DIN ports sending on channel 1, I could route that data stream out to a Synthesiser on the USB host on channel 2 and also to the computer for recording.

Added to this are the idea of inserting "processing modules" to the path, so for instance one that may transform one type of midi into another, or one that allows you to filter out particular types of midi messages.

So each import port/channel can be routed to multiple modules, which can then be routed to multiple modules, which will end up at an output. So for each import port/channel we have a directed graph that the midi stream passes through.

The Gui will allow you to set all this up, handle saving/loading setups.

The isr will do the input/output and processing of data.


So now we have the thorny issue of sharing this data between the two.

I thought maybe having a ping/pong setup where there are two sets of data, at any time the isr and gui are using different data.

If the gui changes anything I then disable interrupts, swap the pointers, copy changes and then enable the interrupts.

Does this sound like a sound way of doing it?
 
If the gui changes anything I then disable interrupts, swap the pointers, copy changes and then enable the interrupts.

Sounds reasonable in principle but I would try to keep the time with disabled interrupts small.

Gui changes anything -> copy changes to the 'second' set -> disable timer interrupt -> swap pointers -> enable timer interrupt. Since copying a pointer is atomic, using this pattern you probably don't need to disable interrupts at all.
 
Sorry maybe a little late to the party...

But if it were me, I would be curious and want more data.
Things like: you show some times:
Code:
lv_task_handler 86us
lv_task_handler 86us
lv_task_handler 5271us
lv_task_handler 41us

What percentage of the calls did not take a long time?

Also can you deduce in the longer run versions where the time is spent during the longer delays?
For example is it being spent in the touchscreen code? (i.e. in your function call back HandleTouchscreen)

Or maybe what it then calls after that...

Again if it were me, I might try looking at some of these and see if these could be improved.

good luck
 
Hi Kurt,

I did put some timing code in the flush code and a small % was only used there, I didn't put any in the touchscreen handling though, i will check that.
 
Ok, I put some timing in.

This is for the demo code from the video above, so probably worst case scenario for screen updating!:

lv_task_handler = total time in lv_task_handler();
flush = total time in flush callback, using Kurts super lib to update the framebuffer
touchscreen = total time in touchscreen callback.

Code:
lv_task_handler 7399us, flush 207us, touchscreen 0us
lv_task_handler 7186us, flush 210us, touchscreen 0us
lv_task_handler 7236us, flush 216us, touchscreen 0us
lv_task_handler 7478us, flush 210us, touchscreen 0us
lv_task_handler 7212us, flush 207us, touchscreen 0us
lv_task_handler 7178us, flush 211us, touchscreen 0us
lv_task_handler 6911us, flush 209us, touchscreen 0us
lv_task_handler 6948us, flush 212us, touchscreen 0us
lv_task_handler 6609us, flush 207us, touchscreen 0us
lv_task_handler 6342us, flush 210us, touchscreen 1us
lv_task_handler 6119us, flush 211us, touchscreen 0us
lv_task_handler 6079us, flush 208us, touchscreen 0us
lv_task_handler 5813us, flush 209us, touchscreen 0us
lv_task_handler 5652us, flush 208us, touchscreen 1us
lv_task_handler 5537us, flush 204us, touchscreen 0us
lv_task_handler 5241us, flush 220us, touchscreen 0us


And this is from the original button swapping LEDs with me pressing the button once, there is also some animation on the button going on here:

I added a 10ms delay() in the loop for this:

Code:
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1004us, flush 23us, touchscreen 82us
lv_task_handler 864us, flush 24us, touchscreen 0us
lv_task_handler 2502us, flush 57us, touchscreen 39us
lv_task_handler 865us, flush 26us, touchscreen 0us
lv_task_handler 866us, flush 25us, touchscreen 0us
lv_task_handler 864us, flush 23us, touchscreen 0us
lv_task_handler 864us, flush 25us, touchscreen 1us
lv_task_handler 864us, flush 25us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 865us, flush 25us, touchscreen 0us
lv_task_handler 864us, flush 24us, touchscreen 0us
lv_task_handler 864us, flush 24us, touchscreen 0us
lv_task_handler 865us, flush 25us, touchscreen 0us
lv_task_handler 864us, flush 24us, touchscreen 0us
lv_task_handler 864us, flush 25us, touchscreen 0us
lv_task_handler 864us, flush 26us, touchscreen 0us
lv_task_handler 863us, flush 25us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us

So the touchscreen stuff is using some time but not a great percentage, I will look into that though.
 
Also just had a thought those timings above are using 20 ms/50fps

Here we are with 30 ms/33.33 fps as in the original posts:

Code:
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 5071us, flush 273us, touchscreen 82us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 2521us, flush 60us, touchscreen 39us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 919us, flush 25us, touchscreen 39us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 918us, flush 25us, touchscreen 39us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 881us, flush 25us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 865us, flush 24us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 866us, flush 24us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 865us, flush 25us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 864us, flush 24us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 864us, flush 23us, touchscreen 1us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
lv_task_handler 0us, flush 0us, touchscreen 0us
lv_task_handler 1us, flush 0us, touchscreen 0us
 
Hi Kurt,

One thing I am looking into is that updateScreenAsync() always seems to send the entire screen (around 16ms at 100Mhz) even with a call to updateChangedAreasOnly(true);

Is this what you would expect?

Cheers

Andy
 
Yep - I have never updated the Async stuff to work with just the updated rectangle. Have thought about it, would would require some additional work like setup the bounding rectangle and the DMA chain
And let alone if then try to turn on continuous updates...

Would not be hard to do, but never needed it as when I am doing async, most of the time, don't need to touch the screen for a long while...
 
Thanks for the info Kurt.

Now I know it's not me doing things wrong :)

I was wondering if the longer LVGL times were caused by not calling lv_task_handler() regularly enough due to the fact that I block the call based on the DMA transfer state, I have tested this though and it is nothing to do with that, it is totally dependent on the FPS setting in LVGL and nothing to do with how quickly you call lv_task_handler().

As it stands I am going to go for the midi processing in the timer isr and see what happens...
 
Status
Not open for further replies.
Back
Top