Occasional problems creating a new file with SerialFlash/Teensy 3.1/Audioshield

Status
Not open for further replies.

a_guy_called_tom

Active member
Hi,

I'm building a drummachine/sampler using a teensy 3.1, audioshield and a W25Q128FV Flash chip to play samples from and record samples to. Im using the SerialFlash Library (version 0.3) to access the flash chip. From time to time, i can no longer create new files on the flash chip (i'm always using the same 6 filenames with a fixed file size, so space on the flash chip should not be the issue). When this problem occurs, SerialFlash.create(filename, SAMPLEBUFFER_LENGTH)) no longer returns true (Line 273 in the code, in the function startRecording()).
After this happens, I can no longer create any new files, even with different filenames. The only thing that helps is to completely wipe the flash chip (using the SerialFlash->EraseEverything Sketch). After that i can again create new files, play files and overwrite existing files.

Any ideas what could cause this problems or how i could debug this? Is there anything i need to take special care when working with files on the flash chip? My instrument allows to record to a file and its possible that the file with the same name is also playing at the same time. Is it possible that this somehow messes up the file allocation table on the flash chip?

Many thanks for your inputs and help and let me know if you need any other details.

Tom


Code:
// needs to be set, otherwise fastled doesnt work together with the audio lib
#define FASTLED_ALLOW_INTERRUPTS 0

#include "Bounce2.h"
#include "FastLED.h"
#include "SequencerTrack.h"
#include "SequencerStep.h"
#include "AudioPlayPitchedSerialflashRaw.h"
#include "Sequencer.h"


#include <Audio.h>
#include <SerialFlash.h>
#include <SPI.h>
#include <Wire.h>
#include <SD.h>


/* 
 * Width of pulse to trigger the shift register to read and latch.
 */
#define PULSE_WIDTH_USEC 5

#define SHIFT_IN_PLOAD_PIN 0 // 2  // Connects to Parallel load pin the 165
#define SHIFT_IN_DATA_PIN 1 // 4 // Connects to the Q7 pin the 165
#define SHIFT_IN_CLOCK_PIN 2 //5 // Connects to the Clock pin the 165

//pin used to send the serial data to the array of leds (via fastLED)
#define DATA_PIN 16

#define POTI_PIN_1 21
#define POTI_PIN_2 20

//number of bytes in sample buffer
#define SAMPLEBUFFER_LENGTH 131072

#define MAX_ENV_HOLD_TIME 1024

// GUItool: begin automatically generated code

AudioInputI2S            i2s2;           //xy=215,423
AudioRecordQueue         queue1;         //xy=376,417

AudioPlayPitchedSerialflashRaw          playMem1;       //xy=385,243
AudioPlayPitchedSerialflashRaw          playMem2;       //xy=385,342
AudioPlayPitchedSerialflashRaw          playMem3;       //xy=386,192
AudioPlayPitchedSerialflashRaw          playMem4;
AudioPlayPitchedSerialflashRaw          playMem5;
AudioPlayPitchedSerialflashRaw          playMem6;

AudioEffectEnvelope      envelope1;
AudioEffectEnvelope      envelope2;
AudioEffectEnvelope      envelope3;
AudioEffectEnvelope      envelope4;
AudioEffectEnvelope      envelope5;
AudioEffectEnvelope      envelope6;

//xy=387,294
AudioMixer8              mixer1;
AudioMixer8              mixer2;//xy=553,273
AudioOutputI2S           i2s1;           //xy=710,272
AudioConnection          patchCord1(i2s2, 0, queue1, 0);
AudioConnection          patchCord2(playMem1, envelope1);
AudioConnection          patchCord3(playMem2, envelope2);
AudioConnection          patchCord4(playMem3, envelope3);
AudioConnection          patchCord5(playMem4, envelope4);
AudioConnection          patchCord6(playMem5, envelope5);
AudioConnection          patchCord7(playMem6, envelope6);
AudioConnection          patchCord8(envelope1, 0, mixer1, 0);
AudioConnection          patchCord9(envelope2, 0, mixer1, 1);
AudioConnection          patchCord10(envelope3, 0, mixer1, 2);
AudioConnection          patchCord11(envelope4, 0, mixer1, 3);
AudioConnection          patchCord12(envelope5, 0, mixer1, 4);
AudioConnection          patchCord13(envelope6, 0, mixer1, 5);
AudioConnection          patchCord14(envelope1, 0, mixer2, 0);
AudioConnection          patchCord15(envelope2, 0, mixer2, 1);
AudioConnection          patchCord16(envelope3, 0, mixer2, 2);
AudioConnection          patchCord17(envelope4, 0, mixer2, 3);
AudioConnection          patchCord18(envelope5, 0, mixer2, 4);
AudioConnection          patchCord19(envelope6, 0, mixer2, 5);
AudioConnection          patchCord20(mixer1, 0, i2s1, 0);
AudioConnection          patchCord21(mixer2, 0, i2s1, 1);

// GUItool: end automatically generated code

AudioPlayPitchedSerialflashRaw *players [NUMBER_OF_INSTRUMENTTRACKS] = {
    &playMem1, &playMem2, &playMem3, &playMem4, &playMem5, &playMem6
};


AudioEffectEnvelope *envelopes [NUMBER_OF_INSTRUMENTTRACKS] = {
    &envelope1, &envelope2, &envelope3, &envelope4, &envelope5, &envelope6
};

AudioControlSGTL5000 audioShield;

unsigned long last_step_time = millis();

unsigned int step_length = 25;

//flash file used to record stuff into
SerialFlashFile toRecordFile;

Sequencer sequencer;

void error(const char *message) {
    while (1) {
        Serial.println(message);
        delay(2500);
    }
}

void spaces(int num) {
    for (int i=0; i < num; i++) {
        Serial.print(" ");
    }
}

void printFlashChipFiles() {
    // wait for Arduino Serial Monitor
    while (!Serial) ;
    delay(100);
    Serial.println("All Files on SPI Flash chip:");
    
    if (!SerialFlash.begin()) {
        error("Unable to access SPI Flash chip");
    }
    
    SerialFlash.opendir();
    while (1) {
        char filename[64];
        unsigned long filesize;
        
        if (SerialFlash.readdir(filename, sizeof(filename), filesize)) {
            Serial.print("  ");
            Serial.print(filename);
            spaces(20 - strlen(filename));
            Serial.print("  ");
            Serial.print(filesize);
            Serial.print(" bytes");
            Serial.println();
        } else {
            break; // no more files
        }
    }
    
}


void setup() {
    Serial.begin(9600);
    
    //uncomment these if using Teensy audio shield
    SPI.setSCK(14);  // Audio shield has SCK on pin 14
    SPI.setMOSI(7);  // Audio shield has MOSI on pin 7
    
    SerialFlash.begin();
    //printFlashChipFiles();
    
    // tact switches input shift register
    pinMode(SHIFT_IN_PLOAD_PIN, OUTPUT);
    //pinMode(inClockEnablePin, OUTPUT);
    pinMode(SHIFT_IN_CLOCK_PIN, OUTPUT);
    pinMode(SHIFT_IN_DATA_PIN, INPUT);
    // init input shift register
    digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
    digitalWrite(SHIFT_IN_PLOAD_PIN, HIGH);
    
    // Audio connections require memory to work.  For more
    // detailed information, see the MemoryAndCpuUsage example
    AudioMemory(20);
    
    // turn on the output
    audioShield.enable();
    audioShield.inputSelect(AUDIO_INPUT_MIC);
    audioShield.volume(0.5);
    
    // by default the Teensy 3.1 DAC uses 3.3Vp-p output
    // if your 3.3V power has noise, switching to the
    // internal 1.2V reference can give you a clean signal
    //dac1.analogReference(INTERNAL);
    
    // reduce the gain on mixer channels, so more than 1
    // sound can play simultaneously without clipping
    mixer1.gain(0, 0.3f);
    mixer1.gain(1, 0.3f);
    mixer1.gain(2, 0.1f);
    mixer1.gain(3, 0.3f);
    mixer1.gain(4, 0.1f);
    mixer1.gain(5, 0.3f);
    
    mixer2.gain(0, 0.3f);
    mixer2.gain(1, 0.3f);
    mixer2.gain(2, 0.3f);
    mixer2.gain(3, 0.1f);
    mixer2.gain(4, 0.3f);
    mixer2.gain(5, 0.1f);
    
    //sequencer = Sequencer();
    //Serial.print("sampleMode is:");
    //Serial.println(sequencer.sampleMode);
    
    for (int i=0; i<NUMBER_OF_INSTRUMENTTRACKS; i++){
        envelopes[i]->decay(5);
        envelopes[i]->sustain(0.0f);
    }
    
    FastLED.addLeds<WS2812B, DATA_PIN, GRB>(sequencer.leds, NUM_LEDS);
    FastLED.setBrightness(100);
    
    Serial.println("--Setup end");
    
    
    for (int i = 0; i < NUM_LEDS; i++){
        for (int j = 0; j < NUM_LEDS; j++){
            sequencer.leds[j] = CRGB::Black;
        }
        sequencer.leds[i] = CRGB::Red;
        FastLED.show();
        delay(20);
    }
    for (int i = NUM_LEDS-1; i >= 0; i--){
        for (int j = 0; j < NUM_LEDS; j++){
            sequencer.leds[j] = CRGB::Black;
        }
        sequencer.leds[i] = CRGB::Red;
        FastLED.show();
        delay(20);
    }
    sequencer.leds[0] = CRGB::Black;
    FastLED.show();
    
    
}






static int recordBufferOffset = 0;

void startRecording() {
    Serial.println("startRecording");
    const char *filename = "0.RAW";
    switch(sequencer.getSelectedTrack()){
        case 0:
            filename = "0.RAW";
            break;
        case 1:
            filename = "1.RAW";
            break;
        case 2:
            filename = "2.RAW";
            break;
        case 3:
            filename = "3.RAW";
            break;
        case 4:
            filename = "4.RAW";
            break;
        case 5:
            filename = "5.RAW";
            break;
        default:
            return;
    }
    
    if (SerialFlash.exists(filename)) {
        SerialFlash.remove(filename);
        Serial.println("removed old file");
    }
    if (SerialFlash.create(filename, SAMPLEBUFFER_LENGTH)) {
        toRecordFile = SerialFlash.open(filename);
        //Serial.println("created and opened new file");
        if (toRecordFile){
            Serial.println("toRecordFile is true");
            queue1.begin();
            sequencer.sampleMode = SamplingMode::RECORDING;
            recordBufferOffset = 0;

        } else {
            Serial.println("cannot record..");
            sequencer.sampleMode = SamplingMode::STOPPED;
        }
    } else {
        Serial.print("cannot create file");
        Serial.println(filename);
        sequencer.sampleMode = SamplingMode::STOPPED;
    }
}



void continueRecording() {
    //Serial.println("continueRecording()");
    while (queue1.available() >= 1) {
        //Serial.println("queue contains data");
        
        if (toRecordFile){
            //Serial.println("file still there");
            if (recordBufferOffset >= SAMPLEBUFFER_LENGTH){
                //Serial.println("finished recording 7.RAW");
                sequencer.sampleMode = SamplingMode::STOPPED;
                queue1.clear();
                queue1.end();
                toRecordFile.close();
                //printFlashChipFiles();
            } else {
                //Serial.println("writing 256 bytes of recording, offset is");
                toRecordFile.write(queue1.readBuffer(), 256);
                queue1.freeBuffer();
                recordBufferOffset += 256;
                //Serial.println(recordBufferOffset);
            }
        } else {
            //Serial.println("file not there");
        }
    }
}

CRGB colorForStepState(uint8_t state){
    switch (state) {
        case 1:
            //trigger on / plock rec off
            return CRGB::CornflowerBlue;
        case 2:
            //trigger off / plock rec on
            return CRGB::Green;
        case 3:
            //trigger on / plock rec on
            return CRGB::DarkOrange;
        default:
            //trigger off / plock rec off
            return CRGB::Black;
    }
}

/*
 * updates all button states by reading in the values from the shift registers
 */
void readButtonStates(){
    // Trigger a parallel Load to latch the state of the data lines in the input shift register.
    digitalWrite(SHIFT_IN_PLOAD_PIN, LOW);
    delayMicroseconds(PULSE_WIDTH_USEC);
    digitalWrite(SHIFT_IN_PLOAD_PIN, HIGH);
    
    // read track buttons states from shift register
    for (int i = NUMBER_OF_TRACKBUTTONS-1;  i >= 0; i--){
        sequencer.trackButtons[i].update(digitalReadFast(SHIFT_IN_DATA_PIN));
        //Pulse the Clock (rising edge shifts the next bit).
        digitalWrite(SHIFT_IN_CLOCK_PIN, HIGH);
        //delayMicroseconds(PULSE_WIDTH_USEC);
        digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
    }
    
    // read step button states from shift register
    for (int i = NUMBER_OF_STEPBUTTONS-1;  i >= 0; i--){
        sequencer.stepButtons[i].update(digitalReadFast(SHIFT_IN_DATA_PIN));
        //Pulse the Clock (rising edge shifts the next bit).
        digitalWrite(SHIFT_IN_CLOCK_PIN, HIGH);
        //delayMicroseconds(PULSE_WIDTH_USEC);
        digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
    }
    
    // read step button states from shift register
    for (int i = NUMBER_OF_FUNCTIONBUTTONS-1;  i >= 0; i--){
        sequencer.functionButtons[i].update(digitalReadFast(SHIFT_IN_DATA_PIN));
        //Pulse the Clock (rising edge shifts the next bit).
        digitalWrite(SHIFT_IN_CLOCK_PIN, HIGH);
        //delayMicroseconds(PULSE_WIDTH_USEC);
        digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
    }
    
    
}



void updateAudio() {
    
    if (millis() - last_step_time >= step_length) {
        
        sequencer.tick();
        last_step_time = millis();
        
        uint16_t sensorValue1 = (uint16_t)analogRead(POTI_PIN_1);
        uint16_t sensorValue2 = (uint16_t)analogRead(POTI_PIN_2);
        
        //Serial.print("sensor1 :");
        //Serial.print(sensorValue1);
        //Serial.print("sensor2 :");
        //Serial.println(sensorValue2);
        
        if (sequencer.pulseCount == 0){
            
            //handle tempo track...
            //TODO should not be handled separately
            sequencer.tracks[6].doStep();
            SequencerStep * step = sequencer.tracks[6].getCurrentStep();
            
            if (step->isParameterLockOn()){
                step->parameter1 = sensorValue1;
            }
            if (step->isTriggerOn()){
                step_length = map(step->parameter1, 0, 1024, 0, 128);
            }
            
            // trigger tracks if step is set to on
            for (int i = 0; i < NUMBER_OF_INSTRUMENTTRACKS; i++) {
                
                //advance one step;
                sequencer.tracks[i].doStep();
                // checks if the bit at position of the current_step is set to 1 in the step on/off integer
                SequencerStep * step = sequencer.tracks[i].getCurrentStep();
                
                if (step->isParameterLockOn()){
                    step->parameter1 = sensorValue1;
                    step->parameter2 = sensorValue2;
                }
            }
            
            if (sequencer.sampleMode == SamplingMode::ARMED && sequencer.tracks[sequencer.getSelectedTrack()].currentStep == 0){
                startRecording();
            }
        }
        
        // punch in style recording of the trigger pattern for the currently selected track
        uint8_t triggerPattern = 0b00000001;
        if (sequencer.functionButtons[5].read()){
            if (sensorValue1 < 200){
                triggerPattern = 0b00000001;
            } else if (sensorValue1 < 400){
                triggerPattern = 0b00000101;
            } else if (sensorValue1 < 600){
                triggerPattern = 0b00000111;
            } else if (sensorValue1 < 800){
                triggerPattern = 0b00010101;
            } else {
                triggerPattern = 0b00111111;
            }
            sequencer.tracks[sequencer.getSelectedTrack()].getCurrentStep()->triggerPattern = triggerPattern;
        }
        
        
        // trigger tracks if step is set to on
        for (int i = 0; i < NUMBER_OF_INSTRUMENTTRACKS; i++) {
            SequencerStep * step = sequencer.tracks[i].getCurrentStep();
            
            if (!sequencer.tracks[i].isMuted() && step->isTriggerOn()
                //check if the bit for the n-th pulse of this step is set
                && (step->triggerPattern & (1 << sequencer.pulseCount))){
                if (sequencer.functionButtons[4].read()){
                    players[i]->frequency(sensorValue1);
                    envelopes[i]->hold(map(sensorValue2, 0, 1024, 0, MAX_ENV_HOLD_TIME));
                } else {
                    players[i]->frequency((float)step->parameter1);
                    envelopes[i]->hold(map(step->parameter2, 0, 1024, 0, MAX_ENV_HOLD_TIME));
                }
                envelopes[i]->noteOn();
                switch (i) {
                    case 0:
                        players[i]->play("0.RAW");
                        break;
                    case 1:
                        players[i]->play("1.RAW");
                        break;
                    case 2:
                        players[i]->play("2.RAW");
                        break;
                    case 3:
                        players[i]->play("3.RAW");
                        break;
                    case 4:
                        players[i]->play("4.RAW");
                        break;
                    case 5:
                        players[i]->play("5.RAW");
                        break;
                    default:
                        break;
                }
                
            }
        }
        
    }
    if (sequencer.sampleMode == SamplingMode::RECORDING) {
        continueRecording();
    }
}




void loop() {
    
    readButtonStates();
    sequencer.updateState();
    FastLED.show();
    
    if (sequencer.isRunning()){
        updateAudio();

        if (0){

            Serial.print("CPU: ");
            Serial.print("all=");
            Serial.print(AudioProcessorUsage());
            Serial.print(",");
            Serial.print(AudioProcessorUsageMax());
            Serial.print("    ");
            Serial.print("Memory: ");
            Serial.print(AudioMemoryUsage());
            Serial.print(",");
            Serial.print(AudioMemoryUsageMax());
            Serial.println();

        }
    }
    
}
 
Is there a tool to walk the storage structures to see the state of the system and show free space? Number of files and size and walk through them showing the length and key details - i.e. chkdsk?

I see "Example / Listfiles" that may indicate where the problem lies. Not sure if it will indicate anything about your problem. You seem to have some part of this in your sketch.

I find this interesting - but have not looked at it yet in any way - though I did buy chips and memory boards.

To continue to use a file per the GitHub readme - it seems you should createErasable and then to start it anew would have to "SerialFlash.remove(filename);"?

Each byte may be written only once. Files created as erasable may be fully erased, to allow new data to be written.

Paul or Frank would know - but I guess adding the same named file just abandons the old copy and starts fresh - and it can only do this until the flash fills - which is I think what you are seeing and why a full erase lets you work again.

This would seem like a common use case and an Example to that effect would be instructive.
 
Hi, thanks a lot for your answer.
Indeed it seems that the flash chip is filling up and finally running out of space. I could verify this by counting the number of successful writes before the problem starts to occur. Its always the same number and the number is consistent with my sample file size and the overall size of the flash (131072 bytes * 127 writes is around 16777216 bytes total flash memory). This is good to know. However the bad news is that changing the code to SerialFlash.createErasable did not solve the problem. Still running out of space, even though the debug log output indicates that when an old file with the same name exists, it is sucessfully removed, before a new file with the same name is created and filled with the recorded samples.
This is very strange.. any other ideas?

Edit: i tried listing all files and cannot see any problems in that output:

All Files on SPI Flash chip:
0.RAW 131072 bytes
1.RAW 131072 bytes
2.RAW 131072 bytes
3.RAW 131072 bytes
4.RAW 131072 bytes
5.RAW 131072 bytes
--Setup end
 
Last edited:
@a_guy_called_tom: If you can create a skeleton of your sketch that would be easier to work with using one/some instances of - createErasable - write to file(s) then - SerialFlash.remove() then Paul or IIRC FrankB might look into it to resolve it.

I'd suggest starting with a minimal existing example - remove anything not needed then put in this functionality in some reasonable form that might make another good example for general use.

<edit>: And post that sample here: code.PNG
 
Last edited:
Heres a minimal sample that reproduces the problem. With a freshly erased flash memory, the code in the loop method executed 128 times. After that, a new file with the same name cannot be created anymore, even though the log output indicates, that the previous files with the same name were removed.

Any help is appreciated.

Code:
#include <SerialFlash.h>
#include <SPI.h>

#define SAMPLEBUFFER_LENGTH 131072

static int iteration = 0;



void setup() {
    Serial.begin(9600);
 
    SPI.setSCK(14);  // Audio shield has SCK on pin 14
    SPI.setMOSI(7);  // Audio shield has MOSI on pin 7
    
    SerialFlash.begin();
    printFlashChipFiles();
    
    Serial.println("--Setup end");       
}

void loop() {
      iteration++;
      Serial.printf("Starting iteration: %i \n", iteration);
   
      
      const char *filename = "0.RAW";
      SerialFlashFile flashFile;

    
      // check for existing file with the same name, and remove it is there.
      if (SerialFlash.exists(filename)) {
          Serial.println("theres an existing file");
          if (SerialFlash.remove(filename)){
            Serial.println("removed old file");
          } else{
            error("Error: could not remove existing old file");
          }
      }
      // create a new file
      if (SerialFlash.createErasable(filename, SAMPLEBUFFER_LENGTH)) {
          flashFile = SerialFlash.open(filename);
          if (flashFile){
              Serial.println("flashFile is true, everything ok, will close the file now");
              // this is where the recording would normally happen and the bytes written.
              flashFile.close();
          } else {
              error("Error: flashFile is not opened");
          }
      } else {
          error("Error cannot create file 0.RAW");
      }
}

//print error message
void error(const char *message) {
    while (1) {
        Serial.println(message);
        delay(2500);
    }
}

//utility function
void spaces(int num) {
    for (int i=0; i < num; i++) {
        Serial.print(" ");
    }
}

// list filenames and sizes of the flash memory chip
void printFlashChipFiles() {
    // wait for Arduino Serial Monitor
    while (!Serial) ;
    delay(100);
    Serial.println("All Files on SPI Flash chip:");
    
    if (!SerialFlash.begin()) {
        error("Unable to access SPI Flash chip");
    }
    
    SerialFlash.opendir();
    while (1) {
        char filename[64];
        unsigned long filesize;
        
        if (SerialFlash.readdir(filename, sizeof(filename), filesize)) {
            Serial.print("  ");
            Serial.print(filename);
            spaces(20 - strlen(filename));
            Serial.print("  ");
            Serial.print(filesize);
            Serial.print(" bytes");
            Serial.println();
        } else {
            break; // no more files
        }
    }
    
}

This outputs (last few lines):

...
Starting iteration: 125
theres an existing file
removed old file
flashFile is true, everything ok, will close the file now
Starting iteration: 126
theres an existing file
removed old file
flashFile is true, everything ok, will close the file now
Starting iteration: 127
theres an existing file
removed old file
flashFile is true, everything ok, will close the file now
Starting iteration: 128
theres an existing file
removed old file
Error cannot create file 0.RAW
Error cannot create file 0.RAW
Error cannot create file 0.RAW
Error cannot create file 0.RAW
 
Space is not reused when you remove a file. SerialFlash gives you something that looks like a filesystem, but it's really just a very thin layer over the flash memory. It's not a full filesystem.

Every time you remove the old file, more of the flash chip is used. Eventually you'll run out of space. I'm pretty sure that's what's happening here.
 
Thanks Paul - I was under the impression a file 'createErasable ' would be selectively made available for re-use with SerialFlash.remove() - based on the function names and the size boundary creation detail on erasable "Files created for erasing automatically increase in size to the nearest number of erasable blocks, resulting in a file that may be 4K to 128K larger than requested."

Also this text seemed to speak to this usage case:
Delete A File
SerialFlash.remove(filename);
The actual space used by the file is not reclaimed. However, a new file with this name may be created after the original is deleted.

However reading that in the context of your post - the first part takes precedence when it comes to space exhaustion. The odd thing is that it seemed to be working that same way for '...._tom' when he was not using the 'erasable' creation.
 
Hi all, many thanks for your answers.

I need to have a fixed number of samples on the flash memory and they all have the same fixed size (eg. 8 samples, 256KB each). But i need to be able to change the contents of the files or overwrite them, without running out of space eventually (each of my 8 sequencer tracks plays one specific sample that can be changed by recording a new sample).
After reading Pauls answer, i'm not even sure if this is possible using the SerialFlash Library? If not, any hints on how this could be achieved (possibliy without diving into the real low-level territory which i'm not really familiar with)?

Thanks,
thomas
 
As noted - not having looked at this in any way before - I assumed space from an erased file would get eraseBlock()'d and get put back into a free pool. In your case that is exactly what would work - as long as you could suffer the downtime of the block erase cycles - since you'd just re-use that block later in the same size it wouldn't even fragment the space.

Perhaps this is planned for a future version as some groundwork would seem to be there. I see this Winbond chip has 32KB and 64KB block erase commands. The 64KB eraseBlock function is exercised in the example/rawhardwaretest. The problem would be the directory structure entry being 'write twice'? - once to create a file - and once again to zero it to show it is no longer a pointer to an active file. File directory block itself would have to be erased to support new file entries - or some other scheme using the same filename forcing the block erase of the data before continuing [i.e. REcreateErasable()] - or more robustly if the directory area allocated supports it: to migrate used dir entries to a new block and then erase the old one when more space was needed over time. Would be non-trivial but do-able with more down time for a 'defrag' to run. I didn't read the code far enough to see how big the directory area is - if it has multiple 32KB blocks - two half full or a free block it could be done while in use up to the block erase part stopping writes for the duration. If the directory data is given fixed space or less than a 32KB block - this would be a bigger change - like the fun of going from FAT16 to FAT32 was.
 
@a_guy_called_tom: without a software change it seems like you would need to add a 2nd flash chip and make sure the other is empty and ready to migrate to and then clean and prep the other one behind you.
 
I need to have a fixed number of samples on the flash memory and they all have the same fixed size (eg. 8 samples, 256KB each). But i need to be able to change the contents of the files or overwrite them, without running out of space eventually

Ok. Just create them as erasable files. When you need to overwrite the file, just erase it. But never delete the file with remove().

Actually, you might pre-erase the file. Erasing is slow, so if you need to begin recording with low latency, make sure you have a file already erased.


I assumed space from an erased file would get eraseBlock()'d and get put back into a free pool.

With SerialFlash, there's a subtle but important distinction between erased and removed/deleted. Erasing the file does not delete the file. It simply resets all the bytes in the file to 0xFF.

There is no free pool or other conventional filesystem space management with SerialFlash. The files are simply created one after the next. Files are always created with a fixed size that can never grow (or shrink). Once created, the file's physical location in the chip is fixed.

When you delete a file with remove(), the space is not reclaimed. The data isn't deleted either. It's just left there in the memory. Only the filename record is overwritten, which means all reference to the file is gone. Then you can create another file with that same name.

I see this Winbond chip has 32KB and 64KB block erase commands. The 64KB eraseBlock function is exercised in the example/rawhardwaretest.

It's also used when you call file.erase().

You must create the file with SerialFlash.createErasable(). The normal SerialFlash.create() packs the file tightly, with only page padding to support the write and erase suspend on a file granularity level. SerialFlash.createErasable() aligns and pads the file to those erasable block boundaries, so you can later use the file.erase() function.


seems like you would need to add a 2nd flash chip and make sure the other is empty and ready to migrate to and then clean and prep the other one behind you.

SerialFlash only supports a single chip.
 
Ok. Just create them as erasable files. When you need to overwrite the file, just erase it. But never delete the file with remove().

Actually, you might pre-erase the file. Erasing is slow, so if you need to begin recording with low latency, make sure you have a file already erased.

Thanks Paul. Tried this, and it works - the flash no longer runs out of memory. However as you pointed out, erase is slow and is now blocking the whole instrument for about 400ms.

1) Since erase just seems to reset the bytes to 0xFF, could i overwrite the file without erasing the old data first? Since im using raw wav data without any header, my assumption was that this might work, with the result that the first part of the file would be the new sample and the last part would still be the old sample during the write. I tried this out (simply commenting out the file.erase() before writing new data to the file, but what i hear (pretty nasty digital noise) tells me that this is not a good solution.
2) Is there a way i can do file.erase() in a non-blocking way on the teensy? One goal in my design is that everything can be done while the sequencer is running, so it would be ideal if i could solve this problem without blocking the UI and audio out

Sorry to bother you all with these questions, any help is much appreciated. (If your interested to know about the project your helping me with: https://vimeo.com/144595364)

Thomas
 
Erase is slow. Nothing will change that.

However, SerialFlash could do more to allow you to not block your program.....
 
Status
Not open for further replies.
Back
Top