MIDI dropouts when I use readMidi with FrameBuffer ILI9341_t3n

Rolfdegen

Rolfdegen

Well-known member
I have MIDI dropouts when I use readMidi with FrameBuffer ILI9341_t3n :(

Code:

#include <Arduino.h>
#include "Audio.h"
#include <MIDI.h>
#include "ILI9341_t3n.h"

elapsedMillis screenTimer = 0;

setup:
My FrameBuffer settings are:
tft.useFrameBuffer(true);
tft.updateChangedAreasOnly(true);

loop() {
readMidi();

if (screenTimer = 27) {
tft.fillScreen(ILI9341_BLACK);
tft.updateScreen();
screenTimer = 0; }
 
I use FrameBuffer to draw menu functions and sound curves on the display. When playing MIDI notes there are short interruptions in the sound. If I switch the FrameBuffer to continuous image output the sound interruptions are gone. But the image output is not clean.
 
I watched your video, not sure I followed exactly what you were doing but are you sure it is midi drop out i.e. actually missing midi messages or is the timing just being thrown off ? I see you print to serial note on - do you see the right sequence of notes.

Cheers, Paul
 
Hello Paul..
Thanks for your answer. Thank you for your tip. I checked the notes. It seems to be a timing problem. When I call up a menu page with fillScreen() there is a short interruption in the sound. No midi in note is lost (see pic from seriel monitor).


Screenshot 2025-02-06 220734.png
 
I would time add some elapsedMicros timers around fillScreen() and the other items in your Task_Timer routine to characterise how long they take (average and also max). Depending on results you may be able tot optimise them or make midi.read() occur more deterministically

Cheers Paul
 
I checked the following. I drew a menu page with fillScreen() and did not do a tft.updateScreen. There was no audio interruption. I think it only happens when the frameBuffer sends the data to the display.
 
I also tried this.. The problem remains

C: 
// fill Rectangle with readMidi --------------------------------
void fillScreen_query()
{
    uint8_t count = 240;
    for (size_t i = 0; i < count; i++)
    {
        tft.drawFastHLine(0,i,320, ILI9341_BLACK);
        readMidi();
    }
    
    
}
void fillRect_query(uint16_t x1, uint8_t y1, uint16_t w, uint8_t h, uint16_t color)
{
    for (size_t i = 0; i < h; i++)
    {
        tft.drawFastHLine(x1, y1 + i, w, color);
        readMidi();
    }
    
}
 
I do not know the library but I would time these transactions, the library has lots of comments like //Not sure if should or not or do like main code and break up into transactions... which suggests they may take some time

Cheers, Paul
 
The frame buffer needs max 40 ms to write to the display (Wire clock speed is 30MHz). Since this works with DMA, reading from MIDI In should not be a problem !?
 
If dma then control should return to your loop but I would still measure the time take in you loop() to be sure.

I don't know if your library has a dual buffer but is there a chance that you're modifying the framebuffer while DMA is still transferring?
 
had Quick Look at library, it has a dmabusy so maybe something like
Code: 
if (!tft.dmaBusy()) { tft.updateScreen}
 
If you use tft.updateScreen(), timing wise, more or less the same as tft.fillScreen(some_color)...

And yes the comments about
houtson said:
//Not sure if should or not or do like main code and break up into transactions...
Click to expand...
As mentioned, we could add an endTransaction()... beingTransaction()... after each scan line or the like, maybe with a yield() call between.
In probably 95%+ cases this will only cause the calls to take longer. As it will not return from this call until the whole (or partial) screen
is redrawn. And releasing the transactions only is of value, if potentially during a call to yield you try to use the same SPI buss for something
different. Now if we were running on a core which was setup for threading and this allowed other threads to run, than maybe.

UpdateScreenAsync - will do most of the operation using DMA. Most in that, the call will do the stuff to setup the transfer, like
output the position data, and then the write Memory command, and then it will start the DMA transfer to write out the contents
of the display.

Other options that can speed things up:
Is to experiment with the call: tft.updateChangedAreasOnly(bool)
The setting for this I believe is off by default, when turned on, it remembers the minimum rectangle the includes all of the
graphic commands that were output to the frame buffer and only redraws that region.

But if your draw code, starts off with something like: fillScreen(BLACK);
It does not help as the whole screen has changed...

If however you know for example that you are only redrawing lets say one of your level meter like things, than you can draw that region
and do an update screen which will only draw that region, which the fewer pixels you have to output the faster it will complete.

Similarly, you can use the setClipRect(x, y, w, h) for a region and all output to the frame buffer will be restricted to that region.
if the clipping rectangle is still set when you call updateScreen, it will only output in that area...
 
Hello Kurt. Thank you for your comments. I will test some of what you said. For the frame buffer I use the function "updateChangedAreasOnly". This reduces the data transfer for small blocks, e.g. voice level display (this works without audio interruption). What is the difference between updateScreen() and updateScreenAsync(false)?
 
Update Screen sync works without sound interruption but with display disruptions (image stripes). With UpdateScreen I have a good display image but with sound interruptions. I don't know ?
 
Hallo Teensy friends :)
Thanks for your help. I found a solution. I write the display data when asyncUpdateActive() == false. In a time loop of 51ms I update the screen with updateScreenAsync(false). This works very well without any sound interruptions. Watch my video.. and SDA timing on scope.

C: 
//********************************************************************
// Sammy   Polyhonic DIY Sampler (Degnerator 2)
// (c) Rolf Degen 11.2024 Version 1.00-01
// Teensy 4.1 600MHz
//********************************************************************

#include <Arduino.h>
#include "Audio.h"
#include <MIDI.h>
#include <SD.h>
#include <Wire.h>
#include <avr/interrupt.h>
#include "SPI.h"
#include "TeensyVariablePlayback.h"
#include "flashloader.h"
#include "ILI9341_t3n.h"
// #include <ILI9341_t4.h>
#include "Font/font_DroidSans.h"
#include "Adafruit_FT6206.h"
#include "touch_button.h"
#include "config.h"
#include "screensaver.h"
#include "GUI.h"
#include "filemanager.h"
#include "audioPatching.h"
#include "ILI9341_t3_PrintScreen.h"
#include "Font/font_LiberationSans.h"

extern "C" uint8_t external_psram_size;
MIDI_CREATE_INSTANCE(HardwareSerial, Serial1, MIDI);
const int Midi_channel = 1;

// Initialize tft display
DMAMEM uint16_t screenBuffer[320 * 240];
// EXTMEM uint16_t screenBuffer[320 * 240];
ILI9341_t3n tft(TFT_CS, TFT_DC, TFT_RST, TFT_MOSI, TFT_SCK, TFT_MISO);
// The FT6206 (touch pannel) uses hardware I2C (SCL/SDA)
Adafruit_FT6206 ts = Adafruit_FT6206();

// Rotary encoder
#define encoder1_addr 48
#define encoder2_addr 49
#define encoder3_addr 50
#define encoder4_addr 51
#define encoder5_addr 52
#define encoder6_addr 53
#define PIN_ENC_INT 25

// Setup -------------------------------------------------------
void setup()
{
    // init Serial for debug
    Serial.begin(9600);
    // init Midi
    MIDI.begin();
    MIDI.setHandleNoteOn(myNoteOn);
    MIDI.setHandleNoteOff(myNoteOff);
    SPI.begin();
    // init Display
    tft.begin();
    tft.setFrameBuffer(screenBuffer); // Initialize Frame Buffer
    tft.useFrameBuffer(true);         // Use Frame Buffer
    // tft.waitUpdateAsyncComplete();
    tft.updateChangedAreasOnly(true);
    tft.invertDisplay(1);
    tft.setRotation(3);
    // init TouchScreen
    ts.begin(10);
 

    // init Wire1 for Encoders
    Wire1.begin();
    Wire1.setClock(400000UL); // I2C speed 400KHz
    encoder_set(encoder_addr + 0, 0, 127, 1, 0, 0);
    encoder_set(encoder_addr + 1, 0, 127, 1, 0, 0);
    encoder_set(encoder_addr + 2, 0, 127, 1, 0, 0);
    encoder_set(encoder_addr + 3, 0, 127, 1, 0, 0);
    encoder_set(encoder_addr + 4, 0, 127, 1, 0, 0);
    encoder_set(encoder_addr + 5, 0, 127, 1, 0, 0);
    attachInterrupt(digitalPinToInterrupt(PIN_ENC_INT), Encoders_Interrupt, RISING);

    // init SDCARD
    Serial.print("Initializing SD card...");
    while (!SD.begin(BUILTIN_SDCARD))
    {
        Serial.println("initialization failed!");
        delay(1000);
    }
    Serial.println("SD card ok");

    // init Audio
    AudioMemory(128);
    // Audio Patching
    outMixer1.gain(0, 0.5);
    outMixer1.gain(1, 0.5);
    outMixer1.gain(2, 0.5);
    outMixer1.gain(3, 0.5);
    outMixer2.gain(0, 0.5);
    outMixer2.gain(1, 0.5);
    outMixer2.gain(2, 0.5);
    outMixer2.gain(3, 0.5);
    outMixer3.gain(0, 0.5);
    outMixer3.gain(1, 0.5);
    outMixer3.gain(2, 0.5);
    outMixer3.gain(3, 0.5);
    outMixer4.gain(0, 0.5);
    outMixer4.gain(1, 0.5);
    outMixer4.gain(2, 0.5);
    outMixer4.gain(3, 0.5);
    MainOutMixerL.gain(0, 0.5f);
    MainOutMixerL.gain(1, 0.5f);
    MainOutMixerR.gain(0, 0.5f);
    MainOutMixerR.gain(1, 0.5f);

    // init AmpEnvelope
    const int8_t envelopeType = -4; // 0 linear, -8 fast exponential, -8 slow exponential
    ampEnv[0].setEnvType(envelopeType);
    ampEnv[1].setEnvType(envelopeType);
    ampEnv[2].setEnvType(envelopeType);
    ampEnv[3].setEnvType(envelopeType);
    ampEnv[4].setEnvType(envelopeType);
    ampEnv[5].setEnvType(envelopeType);
    ampEnv[6].setEnvType(envelopeType);
    ampEnv[7].setEnvType(envelopeType);
    ampEnv[0].delay(ampEnvValue[0]);
    ampEnv[1].delay(ampEnvValue[0]);
    ampEnv[2].delay(ampEnvValue[0]);
    ampEnv[3].delay(ampEnvValue[0]);
    ampEnv[4].delay(ampEnvValue[0]);
    ampEnv[5].delay(ampEnvValue[0]);
    ampEnv[6].delay(ampEnvValue[0]);
    ampEnv[7].delay(ampEnvValue[0]);
    ampEnv[0].attack(ampEnvValue[1]);
    ampEnv[1].attack(ampEnvValue[1]);
    ampEnv[2].attack(ampEnvValue[1]);
    ampEnv[3].attack(ampEnvValue[1]);
    ampEnv[4].attack(ampEnvValue[1]);
    ampEnv[5].attack(ampEnvValue[1]);
    ampEnv[6].attack(ampEnvValue[1]);
    ampEnv[7].attack(ampEnvValue[1]);
    ampEnv[0].hold(ampEnvValue[2]);
    ampEnv[1].hold(ampEnvValue[2]);
    ampEnv[2].hold(ampEnvValue[2]);
    ampEnv[3].hold(ampEnvValue[2]);
    ampEnv[4].hold(ampEnvValue[2]);
    ampEnv[5].hold(ampEnvValue[2]);
    ampEnv[6].hold(ampEnvValue[2]);
    ampEnv[7].hold(ampEnvValue[2]);
    ampEnv[0].decay(ampEnvValue[3]);
    ampEnv[1].decay(ampEnvValue[3]);
    ampEnv[2].decay(ampEnvValue[3]);
    ampEnv[3].decay(ampEnvValue[3]);
    ampEnv[4].decay(ampEnvValue[3]);
    ampEnv[5].decay(ampEnvValue[3]);
    ampEnv[6].decay(ampEnvValue[3]);
    ampEnv[7].decay(ampEnvValue[3]);
    ampEnv[0].sustain(ampEnvValue[4]);
    ampEnv[1].sustain(ampEnvValue[4]);
    ampEnv[2].sustain(ampEnvValue[4]);
    ampEnv[3].sustain(ampEnvValue[4]);
    ampEnv[4].sustain(ampEnvValue[4]);
    ampEnv[5].sustain(ampEnvValue[4]);
    ampEnv[6].sustain(ampEnvValue[4]);
    ampEnv[7].sustain(ampEnvValue[4]);
    ampEnv[0].release(ampEnvValue[5]);
    ampEnv[1].release(ampEnvValue[5]);
    ampEnv[2].release(ampEnvValue[5]);
    ampEnv[3].release(ampEnvValue[5]);
    ampEnv[4].release(ampEnvValue[5]);
    ampEnv[5].release(ampEnvValue[5]);
    ampEnv[6].release(ampEnvValue[5]);
    ampEnv[7].release(ampEnvValue[5]);
    sampleOsc[0].enableInterpolation(true);
    sampleOsc[1].enableInterpolation(true);
    sampleOsc[2].enableInterpolation(true);
    sampleOsc[3].enableInterpolation(true);
    sampleOsc[4].enableInterpolation(true);
    sampleOsc[5].enableInterpolation(true);
    sampleOsc[6].enableInterpolation(true);
    sampleOsc[7].enableInterpolation(true);
    // Load Sample
    newdigate::flashloader loader;
    File root;
    root = SD.open("/Samples");
    const char *_filename = "/Samples/SAMPLE12.WAV"; // Sine 440Hz
    Sample = loader.sampleOsc(_filename);
    sample_busy_flag = true;
    // draw Main page
    pageNo = 0;
    draw_menu_page(pageNo);
    init_Encoder();
}

// Loop ---------------------------------------------------
void loop()
{
    readMidi();

    if (tft.asyncUpdateActive() == false)
    {
        draw_Peak();
        readEncoder();
        readTouchscreen();
        check_Symbols();
    }

    if (updateScreen_Timer >= 51) // 51ms
    {
        tft.updateScreenAsync(false);
        updateScreen_Timer = 0;
    }
}

Video

Scope timing from serial Display data
RigolDS1.png
 
I set the display clock speed to 60MHz and it works stable. But to be on the safe side I set it to 50MHz.

C: 
inits
// Initialize tft display
DMAMEM uint16_t screenBuffer[320 * 240];
//EXTMEM uint16_t screenBuffer[320 * 240];
ILI9341_t3n tft(TFT_CS, TFT_DC, TFT_RST, TFT_MOSI, TFT_SCK, TFT_MISO);

setup:
// init Display
    tft.begin(50000000UL);  // Clock speed 50MHz
    tft.setFrameBuffer(screenBuffer); // Initialize Frame Buffer
    tft.useFrameBuffer(true);         // Use Frame Buffer
    // tft.waitUpdateAsyncComplete();
    tft.updateChangedAreasOnly(true);
    tft.invertDisplay(1);
    tft.setRotation(3);
    // init TouchScreen
    ts.begin(10);
 
Last edited:
You must log in or register to reply here.
Back
Top