Custom audio object causes lock-ups

C

Cougar

Member
So i decided to create an object myself that would give me direct access to the audio data from the main sketch. It works but it causes lock-ups so something isn't peachy and i can't seem to figure out the issue. When i run this sketch it can run for a couple of minutes but sometimes it even locks up after seconds. If anyone has a clue what could be wrong please let me know.

Running on a Teensy 4.1 + Audio board

Sketch and CustomFilter files attached. The watchdog used is this one:
https://github.com/tonton81/WDT_T4
 

Attachments

  • _AudioCustomFilter.cpp
    309 bytes · Views: 15
  • _AudioCustomFilter.h
    772 bytes · Views: 21
  • CustomFilterTest.ino
    1.9 KB · Views: 16
In your update function...

Code: 
void AudioCustomFilter::update(void)
{
    audio_block_t *block;
    block = receiveWritable();
    if (!block) return;
    if (enabled)
    {    if (proc)
        {    __disable_irq();
            proc(block);
            __enable_irq();
        }
    }
    if (sendBlock) transmit(block);
    release(block);
}

Why are you disabling and enabling the IRQ? In the standard formulation of a AudioStream class, this shouldn't be necessary. What problem are you trying to solve with it?

Also, as a separate question, have you tried removing your watchdog system from the sketch? If you remove the watchdog, does the system work correctly or does it still lock up?
 
If you check other objects (AudioAnalyzePeak e.g.) you see quite a few enabling and disabling of the IRQ when variables are changed. This was an attempt to prevent the lock-ups.
The watchdog runs fine for hours in a row in another sketch, i doubt that would be the issue. I'm using it because i don't have a reset on my Teensy and cutting the power cuts power to other stuff as well. It's an easy auto reset when things freeze and also just cool (i think).
 
You have a non-deterministic failure in your system. The debugging technique that I'm suggesting is to keep removing elements of your program until you've got the most minimal example that still has the issue. At that point, the problem usually becomes obvious.

My two suggestions where aimed at trying to see if you are yet at the minimal example:
  • Since disable_irq() and enable_irq() don't make the problem go away, why have them? Removing them gets you closer to the minimal example
  • The watchdog might run fine in another program, but there might be a bad interaction with this program. Just once, take it out. Does it still lock-up? If it runs fine, that tells you that there is indeed a bad interaction. Or, if it still locks up, you can put it back into your program to ease your continued debugging
If you don't want to do these things, that's fine. I'm just trying to help you get to the minimal example, which is usually the best debugging technique.

Another two suggestions:
  • Try not using the "proc" function pointer...instead, just paste the code right there in update function instead of using "proc". Yes, this defeats the whole point of what you're trying to do. I get it. But if you do this temporary substitution, does the system still lock up? If it still locks up even with this simpler code, you've taken another step towards a minimal example.
  • Maybe the system is leaking audio memory? Maybe it's not being released correctly, even though it looks right in your update() function. To test this, in your main loop, periodically print out the usage of the audio memory blocks (Serial.println(AudioMemoryUsage()). Watch it while your system runs to see if your memory usage creeps up. If your system runs out of memory blocks and then locks up, that points to a whole different kind of problem.
 
Last edited:
Thanks but that's all kinda what i'm doing here. Of course i tried without enabling or disabling the IRQ. I made another object exactly doing what you're proposing, using the code from the main sketch directly in the object's code. That sketch also locks-up. In case you didn't notice i was already printing the audio memory usage in the sketch. Maybe you should try running the sketch first? :)
 
Cougar said:
If you check other objects (AudioAnalyzePeak e.g.) you see quite a few enabling and disabling of the IRQ when variables are changed. This was an attempt to prevent the lock-ups.
The watchdog runs fine for hours in a row in another sketch, i doubt that would be the issue. I'm using it because i don't have a reset on my Teensy and cutting the power cuts power to other stuff as well. It's an easy auto reset when things freeze and also just cool (i think).
Click to expand...
You can turn any pin into a reset pin. See the code below:
Code: 
const byte ledPin = 13;
const byte interruptPin = 2;
volatile byte state = LOW;

void setup() {
  pinMode(ledPin, OUTPUT);
  pinMode(interruptPin, INPUT_PULLUP);
  attachInterrupt(digitalPinToInterrupt(interruptPin), reset, CHANGE);
}

void loop() {
  digitalWrite(ledPin, state);
}

void reset() {
    SCB_AIRCR = 0x05FA0004;
}
 
Thanks! Tried that instead of the watchdog, it didn't work but it was worth trying (again) without the watchdog. Unfortunately it still locks up with and without the enabling-disabling IRQ in _AudioCustomFilter.cpp.

Last sketch tested attached.

So all this is just part of a larger sketch. In that larger sketch i use an audioin, audioout, freeverb, chorus, flanger, delay and 2 mixers. That runs pretty smooth (great!) but as soon as i add a custom audio object i'm getting these freezes. It's almost like something is lacking in these custom objects that cause these freeze issues.
 

Attachments

  • CustomFilterTest.ino
    2.1 KB · Views: 13
Last edited:
Cougar said:
If you check other objects (AudioAnalyzePeak e.g.) you see quite a few enabling and disabling of the IRQ when variables are changed. This was an attempt to prevent the lock-ups.
Click to expand...
That's because they are not in the update method - update runs as a low priority software interrupt IIRC, so you need a critical section to change the variables that affect it atomically, but interrupt handlers are automatically atomic critical sections so you don't do anything special inside them.
 
Thanks! Been a while since i was dealing with interrupts. Guess the enable IRQ could've caused some serious issues since it was within the IRQ handler basically. (Even though that wasn't the problem)
 
An issue with the audio library perhaps? I had it last updated august 9th, then i disabled the auto update feature.

E: Doesn't look like much has changed lately in that library...
 
Last edited:
Same optimisation and CPU speed here. USB is set to Serial+MIDI, as it happens, but given that USB Audio isn't involved I can't see it's relevant.

Bit of a mystery, this one. Sorry I can't help more ... if something occurs to me I'll be back :)
 
Ok thanks. Yeah it is a mystery, not crucial for the project though so i guess i'll try again in a week or so... see if it suddenly works out of the blue :)
 
What i forgot to mention is that when running these custom objects from the larger sketch that at some point, after minutes usually, the serial output stops. Would that indicate a memory leak or stack overflow perhaps?
 
Could be. The trouble with a large sketch is that there are so many different ways it can go wrong, and it’s super hard for anyone else to review. You’re really reduced to putting in heap and stack monitors, plus anything else you can think of, to try to eliminate possible causes.

I would say that apart from minor tweaks suggested by others above, which won’t make a difference but will reduce code clutter, your existing principle should work OK, as your cut-down example did. But if your real processing code introduces a problem, that’s different…
 
Well since Serial stopped working anyway, and who needs it when you got an Oled display attached, i removed a whole bunch of Serial.prints in the bigger sketch and now everything seems more stable.. or even.... stable!!
 
Ah yes ... you do sometimes have to be a bit careful with Serial.print calls. A decent 'scope is very helpful, especially for timing-critical stuff.
 
You must log in or register to reply here.
Back
Top