Retroactive Midi Looper Crashing

jaecexd · Nov 22, 2024

I'm trying to develop a retroactive midi looper on the teensy 4.1 that basically listens for a clock signal and then starts recording all notes (played via an array of velocity sensitive hall-effect sensor key switches) into a circular buffer of 32 bars. When the user plays something they like, they can trigger a loop length (1 bar, 2 bars, 4 bars, 8 bars) to start retroactively looping the last x bars of material they just played. I have the functionality working generally speaking, the loops are being recorded and are able to be played back with mostly correct timing, but I am regularly experiencing system failures/crashing during loop playback. The loop will be playing back fine and then it will just stop, after a couple of failures/crashes, the teensy will become unresponsive and need to be power cycled to resume functionality. Is there anyone who is willing to help me diagnose/fix these crashes? Disclaimer: I am very new to C++ and programming in general and I am heavily utilizing AI tools to help code this project, which I understand is pretty much guaranteed to generate somewhat buggy code at this point. I just don't have the skills yet to have implemented this on my own. According to Claude, these are the most likely causes of the crashing:

Race Conditions in Event Processing
MIDI clock interrupts might be colliding with event processing
Events could be getting processed multiple times or skipped entirely
State variables (like positions and counters) might be changing mid-processing
Memory/Buffer Issues
The circular buffer implementation might have edge cases where it overflows
Event timestamps might be wrapping around in unexpected ways
Buffer state (writePos, count) might become inconsistent
Timing Drift
Accumulated timing errors between MIDI clock and event timestamps
Position calculations might be drifting over time
Loop length and position calculations might have precision issues
State Management
Note states might become desynchronized from the actual MIDI output
Loop state transitions (start/stop/reset) might leave the system in an inconsistent state
Multiple state variables that should be atomic might be getting out of sync
Hitting hardware watchdog resets

C++:

#include <MIDIUSB.h>
#include <Adafruit_NeoPixel.h>

// Pin definitions
int keyPins[] = {A0, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11};
const int numKeys = sizeof(keyPins) / sizeof(keyPins[0]);
int joystickXPin = A12;
int joystickYPin = A13;
int joystickSwitchPin = A14;

// Key calibration thresholds
const int keyThresholds[12][3] = {
    {800, 825, 950},  // Key 1
    {786, 820, 926},  // Key 2
    {792, 811, 907},  // Key 3
    {810, 835, 965},  // Key 4
    {748, 760, 928},  // Key 5
    {777, 801, 950},  // Key 6
    {798, 823, 950},  // Key 7
    {805, 830, 960},  // Key 8
    {807, 832, 946},  // Key 9
    {765, 790, 945},  // Key 10
    {808, 832, 947},  // Key 11
    {807, 827, 937}   // Key 12
};

// MIDI settings
const int midiNotes[12] = {60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71};
const int midiChannel = 0;

// System definitions
const int MAX_EVENTS = 1024;
const int BARS_IN_BUFFER = 32;
const int PULSES_PER_BAR = 24 * 4;

// MIDI event structure
struct MidiEvent {
    unsigned long timestamp;
    byte type;
    byte note;
    byte velocity;
    bool active;
};

// NeoPixel and switch definitions
#define NEOPIXEL_PIN 4
#define NUM_PIXELS 4
#define SWITCHA_PIN 0
#define SWITCHB_PIN 1
#define SWITCHC_PIN 2
#define SWITCHD_PIN 3

// Core timing and state
volatile bool loopStateChanged = false;
volatile unsigned long switchDebounceTimer = 0;
const unsigned long DEBOUNCE_DELAY = 50;

// Essential state tracking
volatile bool clockReceived = false;
const unsigned long CLOCK_TIMEOUT = 1000;
unsigned long lastClockTime = 0;
unsigned long totalClocks = 0;
bool isPlaying = false;
bool isLooping = false;
int currentLoopBars = 0;
bool noteStates[128] = {false};
bool keyStates[12] = {false};  // Track states for the 12 keys

// Event buffer
struct CircularBuffer {
    MidiEvent events[MAX_EVENTS];
    int writePos;
    int count;
} eventBuffer;

// Essential loop control
int currentEvent = 0;
unsigned long loopStartTime = 0;
unsigned long loopLength = 0;

// Display and UI state
Adafruit_NeoPixel strip = Adafruit_NeoPixel(NUM_PIXELS, NEOPIXEL_PIN, NEO_GRB + NEO_KHZ800);
bool switchStates[4] = {false};
bool loopActive[4] = {false};
int currentBeat = 0;


void setup() {
    Serial.begin(115200);
    
    // Pin setup
    for (int i = 0; i < numKeys; i++) {
        pinMode(keyPins[i], INPUT);
    }
    pinMode(joystickXPin, INPUT);
    pinMode(joystickYPin, INPUT);
    pinMode(joystickSwitchPin, INPUT_PULLUP);
    
    // Initialize event buffer
    eventBuffer.writePos = 0;
    eventBuffer.count = 0;
    for (int i = 0; i < MAX_EVENTS; i++) {
        eventBuffer.events[i].active = false;
    }
    
    // Initialize note states
    memset(noteStates, 0, sizeof(noteStates));

    // NeoPixel setup
    strip.begin();
    strip.setBrightness(25);
    strip.show();
    
    // Switch setup
    pinMode(SWITCHA_PIN, INPUT_PULLUP);
    pinMode(SWITCHB_PIN, INPUT_PULLUP);
    pinMode(SWITCHC_PIN, INPUT_PULLUP);
    pinMode(SWITCHD_PIN, INPUT_PULLUP);
}

void loop() {
    // Handle MIDI messages
    midiEventPacket_t rx;
    do {
        rx = MidiUSB.read();
        if (rx.header != 0) {
            switch (rx.byte1) {
                case 0xF8:  // MIDI Clock
                    if (isPlaying) handleClock();
                    break;
                    
                case 0xFA:  // MIDI Start
                    // Reset all state first
                    isPlaying = true;
                    clockReceived = true;
                    lastClockTime = millis();
                    currentBeat = 0;
                    totalClocks = 0;
                    
                    // Clear all note states and send note-offs
                    for (byte note = 0; note < 128; note++) {
                        if (noteStates[note]) {
                            noteOff(midiChannel, note, 0);
                            noteStates[note] = false;
                        }
                    }
                    
                    // Only handle loop if we actually have one
                    if (isLooping && currentEvent > 0) {
                        loopStartTime = 0;
                    } else {
                        // Make sure loop state is clean if we don't have a loop
                        isLooping = false;
                        currentEvent = 0;
                        eventBuffer.count = 0;
                        eventBuffer.writePos = 0;
                    }
                    
                    updatePixels();
                    break;
                    
                case 0xFC:  // MIDI Stop
                    handleStop();
                    break;
            }
        }
    } while (rx.header != 0);

    // Check for clock timeout
    if (isPlaying && (millis() - lastClockTime > CLOCK_TIMEOUT)) {
        handleStop();
    }

    // Main processing
    if (isPlaying) {
        checkSwitches();
        readKeysAndRecord();
        
        if (loopStateChanged) {
            initializeLoop(currentLoopBars);
            loopStateChanged = false;
        }
    }
}
// MIDI note handling
void noteOn(byte channel, byte note, byte velocity) {
    if (!MidiUSB.available()) {
        midiEventPacket_t noteOn = {0x09, uint8_t(0x90 | (channel & 0x0F)), note, velocity};
        MidiUSB.sendMIDI(noteOn);
        MidiUSB.flush();
        noteStates[note] = true;
    }
}

void noteOff(byte channel, byte note, byte velocity) {
    if (!MidiUSB.available()) {
        midiEventPacket_t noteOff = {0x08, uint8_t(0x80 | (channel & 0x0F)), note, velocity};
        MidiUSB.sendMIDI(noteOff);
        MidiUSB.flush();
        noteStates[note] = false;
    }
}

void resetAllNotes() {
    for (byte note = 0; note < 128; note++) {
        if (noteStates[note]) {
            noteOff(midiChannel, note, 0);
        }
    }
}

void recordEvent(bool isNoteOn, byte note, byte velocity) {
    if (!isPlaying || eventBuffer.count >= MAX_EVENTS) return;
    
    int nextPos = (eventBuffer.writePos + 1) % MAX_EVENTS;
    MidiEvent& evt = eventBuffer.events[eventBuffer.writePos];
    
    evt.timestamp = totalClocks;
    evt.type = isNoteOn ? 0x90 : 0x80;
    evt.note = note;
    evt.velocity = velocity;
    evt.active = true;
    
    eventBuffer.writePos = nextPos;
    eventBuffer.count++;
}

void readKeysAndRecord() {
    static unsigned long lastReadTime = 0;
    unsigned long currentTime = millis();
    
    // Rate limiting to 200Hz
    if (currentTime - lastReadTime < 5) return;
    lastReadTime = currentTime;

    for (int i = 0; i < numKeys; i++) {
        int sensorValue = analogRead(keyPins[i]);
        
        // Key pressed
        if (sensorValue > keyThresholds[i][1]) {  // Above minimum threshold
            if (keyStates[i] == 0) {  // Was released
                byte velocity = calculateVelocity(
                    sensorValue,
                    keyThresholds[i][1],  // Minimum threshold
                    keyThresholds[i][2]   // Maximum threshold
                );
                
                noteOn(midiChannel, midiNotes[i], velocity);
                if (isPlaying) {
                    recordEvent(true, midiNotes[i], velocity);
                }
                keyStates[i] = 1;
            }
        }
        // Key released
        else if (keyStates[i] == 1) {
            noteOff(midiChannel, midiNotes[i], 0);
            if (isPlaying) {
                recordEvent(false, midiNotes[i], 0);
            }
            keyStates[i] = 0;
        }
    }
}

byte calculateVelocity(int sensorValue, int minThreshold, int maxThreshold) {
    // Constrain the sensor value to our valid range
    sensorValue = constrain(sensorValue, minThreshold, maxThreshold);
    
    // Calculate velocity on a curve for more natural response
    // Subtract minimum to start from 0
    float normalized = (float)(sensorValue - minThreshold) / (maxThreshold - minThreshold);
    
    // Apply a curve for more musical response (squared curve)
    // This makes soft presses easier and maintains dynamic range for harder presses
    normalized = normalized * normalized;
    
    // Scale to MIDI velocity range (1-127, leaving 0 for note off)
    return 1 + (byte)(normalized * 126.0);
}

// MODIFIED: Switch handling with proper loop timing
void checkSwitches() {
    unsigned long now = millis();
    if (now - switchDebounceTimer < DEBOUNCE_DELAY) return;
    
    for (int i = 0; i < 4; i++) {
        bool currentState = !digitalRead(SWITCHA_PIN + i);
        
        if (currentState != switchStates[i]) {
            switchDebounceTimer = now;
            
            if (currentState) {  // Switch pressed
                int bars = 1 << i;  // 1, 2, 4, or 8 bars
                
                if (loopActive[i]) {
                    // Clear this loop
                    strip.setPixelColor(i, strip.Color(255, 0, 0));
                    strip.show();
                    
                    stopLoop();  // Use stopLoop instead of direct state changes
                    loopActive[i] = false;
                } else {
                    // Clear any other active loops
                    for (int j = 0; j < 4; j++) {
                        if (loopActive[j]) {
                            loopActive[j] = false;
                            stopLoop();  // Properly stop any active loops
                        }
                    }
                    
                    // Calculate loop timing
                    unsigned long currentPosition = totalClocks;
                    unsigned long positionInBar = currentPosition % PULSES_PER_BAR;
                    
                    // Always align to current bar start
                    loopStartTime = currentPosition - positionInBar;
                    currentLoopBars = bars;
                    
                    loopActive[i] = true;
                    isLooping = true;
                    loopStateChanged = true;
                }
                
                updatePixels();
            }
            switchStates[i] = currentState;
        }
    }
}

// MODIFIED: Handle MIDI Stop with complete buffer clear
void handleStop() {
    isPlaying = false;
    isLooping = false;
    currentBeat = 0;
    
    // Reset all loop states
    for (int i = 0; i < 4; i++) {
        loopActive[i] = false;
    }
    
    loopLength = 0;
    eventBuffer.writePos = 0;
    eventBuffer.count = 0;
    currentEvent = 0;
    
    resetAllNotes();
    
    strip.clear();
    strip.show();
}

void playLoopEvents() {
    static unsigned long lastProcessedClock = 0;
    static unsigned long lastRelativePosition = 0;
    static bool isProcessing = false;
    
    // Prevent re-entry and duplicate processing
    if (isProcessing || totalClocks == lastProcessedClock) return;
    
    isProcessing = true;
    lastProcessedClock = totalClocks;
    
    // Basic state checks
    if (!isLooping || !isPlaying || loopLength == 0) {
        isProcessing = false;
        return;
    }
    
    // Calculate loop position
    unsigned long clocksSinceStart = totalClocks - loopStartTime;
    unsigned long relativePosition = clocksSinceStart % loopLength;
    
    // Process events
    for (int i = 0; i < currentEvent; i++) {
        if (!eventBuffer.events[i].active) continue;
        
        unsigned long eventTime = eventBuffer.events[i].timestamp % loopLength;
        bool shouldPlay = false;
        
        // Handle loop wraparound
        if (relativePosition < lastRelativePosition) {
            shouldPlay = (eventTime > lastRelativePosition) || (eventTime <= relativePosition);
        } else {
            shouldPlay = (eventTime > lastRelativePosition && eventTime <= relativePosition);
        }
        
        if (shouldPlay) {
            MidiEvent* evt = &eventBuffer.events[i];
            if (evt->type == 0x90) {
                noteOn(midiChannel, evt->note, evt->velocity);
            } else {
                noteOff(midiChannel, evt->note, 0);
            }
        }
    }
    
    // Combined loop boundary and stuck note handling
    if (relativePosition < lastRelativePosition || relativePosition < 24) {  // At loop boundary or first beat
        for (byte note = 0; note < 128; note++) {
            if (noteStates[note]) {
                bool hasActiveNoteOn = false;
                for (int i = 0; i < currentEvent; i++) {
                    if (eventBuffer.events[i].active &&
                        eventBuffer.events[i].note == note &&
                        eventBuffer.events[i].type == 0x90 &&
                        ((eventBuffer.events[i].timestamp % loopLength) <= relativePosition ||
                         relativePosition < lastRelativePosition)) {  // Account for wraparound
                        hasActiveNoteOn = true;
                        break;
                    }
                }
                if (!hasActiveNoteOn) {
                    noteOff(midiChannel, note, 0);
                    noteStates[note] = false;
                }
            }
        }
    }
    
    lastRelativePosition = relativePosition;
    isProcessing = false;
}

void updatePixels() {
    static unsigned long lastPixelUpdate = 0;
    unsigned long now = millis();
    
    // Limit update rate to 50Hz
    if (now - lastPixelUpdate < 20) return;
    lastPixelUpdate = now;
    
    strip.clear();
    
    if (isPlaying) {
        for (int i = 0; i < 4; i++) {
            if (i == currentBeat) {
                strip.setPixelColor(i, strip.Color(64, 64, 64));  // Beat indicator
            } else if (loopActive[i]) {
                strip.setPixelColor(i, strip.Color(0, 32, 0));    // Active loop
            }
        }
    }
    
    strip.show();
}

void initializeLoop(int bars) {
    if (!isPlaying || bars <= 0) return;
    
    loopLength = PULSES_PER_BAR * bars;
    unsigned long currentPosition = totalClocks;
    
    // Align to bar boundary
    unsigned long positionInBar = currentPosition % PULSES_PER_BAR;
    loopStartTime = currentPosition - positionInBar;
    
    // Calculate event window
    unsigned long windowStart = (loopStartTime >= loopLength) ?
        loopStartTime - loopLength : 0;
    unsigned long windowEnd = loopStartTime + PULSES_PER_BAR;
    
    // Create temporary buffer for valid events
    MidiEvent tempBuffer[MAX_EVENTS];
    int validEvents = 0;
    
    // Copy valid events with corrected timestamps
    for (int i = 0; i < eventBuffer.count && validEvents < MAX_EVENTS; i++) {
        int pos = (eventBuffer.writePos - eventBuffer.count + i + MAX_EVENTS) % MAX_EVENTS;
        MidiEvent* evt = &eventBuffer.events[pos];
        
        if (evt->timestamp >= windowStart && evt->timestamp < windowEnd) {
            tempBuffer[validEvents] = *evt;
            
            // Correct timestamp for loop position
            tempBuffer[validEvents].timestamp = (evt->timestamp >= loopStartTime) ?
                evt->timestamp - loopStartTime :
                (evt->timestamp - windowStart) % loopLength;
            
            tempBuffer[validEvents].active = true;
            validEvents++;
        }
    }
    
    // Copy valid events back to main buffer
    currentEvent = validEvents;
    for (int i = 0; i < validEvents; i++) {
        eventBuffer.events[i] = tempBuffer[i];
    }
}

void handleClock() {
    static unsigned long lastHandledClock = 0;
    unsigned long now = millis();

    // Prevent multiple processing in same millisecond
    if (now == lastHandledClock) return;
    lastHandledClock = now;
    
    totalClocks++;
    clockReceived = true;
    lastClockTime = now;
    
    // Update beat counter every quarter note (24 MIDI clocks)
    if (totalClocks % 24 == 0) {
        currentBeat = (currentBeat + 1) % 4;
        updatePixels();
    }
    
    if (isLooping) {
        playLoopEvents();
    }
}

void recoverFromError() {
    static unsigned long lastRecoveryTime = 0;
    unsigned long now = millis();
    
    // Prevent multiple recoveries in quick succession
    if (now - lastRecoveryTime < 100) return;
    lastRecoveryTime = now;
    
    // Find current musical position
    unsigned long clocksInBar = totalClocks % PULSES_PER_BAR;
    
    // Check for stuck notes without stopping them
    for (byte note = 0; note < 128; note++) {
        if (noteStates[note]) {
            bool foundInLoop = false;
            for (int i = 0; i < currentEvent; i++) {
                if (eventBuffer.events[i].active &&
                    eventBuffer.events[i].note == note &&
                    eventBuffer.events[i].type == 0x90) {
                    foundInLoop = true;
                    break;
                }
            }
            // Only stop notes that aren't part of the current loop
            if (!foundInLoop) {
                noteOff(midiChannel, note, 0);
                noteStates[note] = false;
            }
        }
    }
    
    // Realign loop if timing is off
    if (isLooping) {
        unsigned long expectedPosition = (totalClocks - loopStartTime) % loopLength;
        if (abs((long)(expectedPosition - clocksInBar)) > 24) {  // More than 1 beat off
            // Realign to next bar
            loopStartTime = totalClocks + (PULSES_PER_BAR - clocksInBar);
        }
    }
    
    // Clean up event buffer without clearing it
    if (eventBuffer.count > MAX_EVENTS || eventBuffer.writePos >= MAX_EVENTS) {
        // Keep the most recent valid events
        int validEvents = min(MAX_EVENTS, eventBuffer.count);
        for (int i = 0; i < validEvents; i++) {
            eventBuffer.events[i] = eventBuffer.events[eventBuffer.writePos - validEvents + i];
        }
        eventBuffer.count = validEvents;
        eventBuffer.writePos = validEvents;
    }
    
    updatePixels();
}

void stopLoop() {
    isLooping = false;
    
    // Clear any active notes from the loop
    for (int i = 0; i < currentEvent; i++) {
        if (eventBuffer.events[i].active && eventBuffer.events[i].type == 0x90) {
            byte note = eventBuffer.events[i].note;
            if (noteStates[note]) {
                noteOff(midiChannel, note, 0);
                noteStates[note] = false;
            }
        }
    }
    
    // Reset buffer
    currentEvent = 0;
    eventBuffer.count = 0;
    eventBuffer.writePos = 0;
}

joepasquariello · Nov 22, 2024

I am very new to C++ and programming in general and I am heavily utilizing AI tools to help code this project, which I understand is pretty much guaranteed to generate somewhat buggy code at this point. I just don't have the skills yet to have implemented this on my own. According to Claude, these are the most likely causes of the crashing:

Yikes. Have you asked Claude for a debugging procedure?

jaecexd · Nov 22, 2024

Yeah, at one point the code was twice as long as it is now, with about half of it being debug logs and safety checks. It was fairly inconclusive, because the crashes were occuring before the logs could print most of the time. I've wrestled it for maybe 12 hours total by now, with it suggesting debug logs and various solutions, me testing them, feeding the serial logs back into the chat, more suggestions, etc. It led to the code becoming extremely verbose and bloated with debug log statements, which eventually added up to the point where I feared that all the serial printing was exacerbating the problem by overwhelming the cpu. This is what led me to post in here, because I figured that debugging with the AI model that created the problems in the first place was leading to an endless loop of non-solutions, and that a human who actually understands C++ would have some insight that I can't get from AI at this point.

joepasquariello · Nov 22, 2024

A hypothetical non-artificial developer would probably break the problem down into functional pieces, implement and test each one separately, and then go about combining them into the more complex whole.

MarkT · Nov 22, 2024

I think validEvents could reach MAX_EVENTS, then get copied to currentEvent, which is indexing outside the eventBuffer. Then it will trample some other variables and lead to mayhem...

Retroactive Midi Looper Crashing

jaecexd

New member

joepasquariello

Well-known member

jaecexd

New member

joepasquariello

Well-known member

MarkT

Well-known member