Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 13 of 13

Thread: Occasional problems creating a new file with SerialFlash/Teensy 3.1/Audioshield

  1. #1

    Occasional problems creating a new file with SerialFlash/Teensy 3.1/Audioshield

    Hi,

    I'm building a drummachine/sampler using a teensy 3.1, audioshield and a W25Q128FV Flash chip to play samples from and record samples to. Im using the SerialFlash Library (version 0.3) to access the flash chip. From time to time, i can no longer create new files on the flash chip (i'm always using the same 6 filenames with a fixed file size, so space on the flash chip should not be the issue). When this problem occurs, SerialFlash.create(filename, SAMPLEBUFFER_LENGTH)) no longer returns true (Line 273 in the code, in the function startRecording()).
    After this happens, I can no longer create any new files, even with different filenames. The only thing that helps is to completely wipe the flash chip (using the SerialFlash->EraseEverything Sketch). After that i can again create new files, play files and overwrite existing files.

    Any ideas what could cause this problems or how i could debug this? Is there anything i need to take special care when working with files on the flash chip? My instrument allows to record to a file and its possible that the file with the same name is also playing at the same time. Is it possible that this somehow messes up the file allocation table on the flash chip?

    Many thanks for your inputs and help and let me know if you need any other details.

    Tom


    Code:
    // needs to be set, otherwise fastled doesnt work together with the audio lib
    #define FASTLED_ALLOW_INTERRUPTS 0
    
    #include "Bounce2.h"
    #include "FastLED.h"
    #include "SequencerTrack.h"
    #include "SequencerStep.h"
    #include "AudioPlayPitchedSerialflashRaw.h"
    #include "Sequencer.h"
    
    
    #include <Audio.h>
    #include <SerialFlash.h>
    #include <SPI.h>
    #include <Wire.h>
    #include <SD.h>
    
    
    /* 
     * Width of pulse to trigger the shift register to read and latch.
     */
    #define PULSE_WIDTH_USEC 5
    
    #define SHIFT_IN_PLOAD_PIN 0 // 2  // Connects to Parallel load pin the 165
    #define SHIFT_IN_DATA_PIN 1 // 4 // Connects to the Q7 pin the 165
    #define SHIFT_IN_CLOCK_PIN 2 //5 // Connects to the Clock pin the 165
    
    //pin used to send the serial data to the array of leds (via fastLED)
    #define DATA_PIN 16
    
    #define POTI_PIN_1 21
    #define POTI_PIN_2 20
    
    //number of bytes in sample buffer
    #define SAMPLEBUFFER_LENGTH 131072
    
    #define MAX_ENV_HOLD_TIME 1024
    
    // GUItool: begin automatically generated code
    
    AudioInputI2S            i2s2;           //xy=215,423
    AudioRecordQueue         queue1;         //xy=376,417
    
    AudioPlayPitchedSerialflashRaw          playMem1;       //xy=385,243
    AudioPlayPitchedSerialflashRaw          playMem2;       //xy=385,342
    AudioPlayPitchedSerialflashRaw          playMem3;       //xy=386,192
    AudioPlayPitchedSerialflashRaw          playMem4;
    AudioPlayPitchedSerialflashRaw          playMem5;
    AudioPlayPitchedSerialflashRaw          playMem6;
    
    AudioEffectEnvelope      envelope1;
    AudioEffectEnvelope      envelope2;
    AudioEffectEnvelope      envelope3;
    AudioEffectEnvelope      envelope4;
    AudioEffectEnvelope      envelope5;
    AudioEffectEnvelope      envelope6;
    
    //xy=387,294
    AudioMixer8              mixer1;
    AudioMixer8              mixer2;//xy=553,273
    AudioOutputI2S           i2s1;           //xy=710,272
    AudioConnection          patchCord1(i2s2, 0, queue1, 0);
    AudioConnection          patchCord2(playMem1, envelope1);
    AudioConnection          patchCord3(playMem2, envelope2);
    AudioConnection          patchCord4(playMem3, envelope3);
    AudioConnection          patchCord5(playMem4, envelope4);
    AudioConnection          patchCord6(playMem5, envelope5);
    AudioConnection          patchCord7(playMem6, envelope6);
    AudioConnection          patchCord8(envelope1, 0, mixer1, 0);
    AudioConnection          patchCord9(envelope2, 0, mixer1, 1);
    AudioConnection          patchCord10(envelope3, 0, mixer1, 2);
    AudioConnection          patchCord11(envelope4, 0, mixer1, 3);
    AudioConnection          patchCord12(envelope5, 0, mixer1, 4);
    AudioConnection          patchCord13(envelope6, 0, mixer1, 5);
    AudioConnection          patchCord14(envelope1, 0, mixer2, 0);
    AudioConnection          patchCord15(envelope2, 0, mixer2, 1);
    AudioConnection          patchCord16(envelope3, 0, mixer2, 2);
    AudioConnection          patchCord17(envelope4, 0, mixer2, 3);
    AudioConnection          patchCord18(envelope5, 0, mixer2, 4);
    AudioConnection          patchCord19(envelope6, 0, mixer2, 5);
    AudioConnection          patchCord20(mixer1, 0, i2s1, 0);
    AudioConnection          patchCord21(mixer2, 0, i2s1, 1);
    
    // GUItool: end automatically generated code
    
    AudioPlayPitchedSerialflashRaw *players [NUMBER_OF_INSTRUMENTTRACKS] = {
        &playMem1, &playMem2, &playMem3, &playMem4, &playMem5, &playMem6
    };
    
    
    AudioEffectEnvelope *envelopes [NUMBER_OF_INSTRUMENTTRACKS] = {
        &envelope1, &envelope2, &envelope3, &envelope4, &envelope5, &envelope6
    };
    
    AudioControlSGTL5000 audioShield;
    
    unsigned long last_step_time = millis();
    
    unsigned int step_length = 25;
    
    //flash file used to record stuff into
    SerialFlashFile toRecordFile;
    
    Sequencer sequencer;
    
    void error(const char *message) {
        while (1) {
            Serial.println(message);
            delay(2500);
        }
    }
    
    void spaces(int num) {
        for (int i=0; i < num; i++) {
            Serial.print(" ");
        }
    }
    
    void printFlashChipFiles() {
        // wait for Arduino Serial Monitor
        while (!Serial) ;
        delay(100);
        Serial.println("All Files on SPI Flash chip:");
        
        if (!SerialFlash.begin()) {
            error("Unable to access SPI Flash chip");
        }
        
        SerialFlash.opendir();
        while (1) {
            char filename[64];
            unsigned long filesize;
            
            if (SerialFlash.readdir(filename, sizeof(filename), filesize)) {
                Serial.print("  ");
                Serial.print(filename);
                spaces(20 - strlen(filename));
                Serial.print("  ");
                Serial.print(filesize);
                Serial.print(" bytes");
                Serial.println();
            } else {
                break; // no more files
            }
        }
        
    }
    
    
    void setup() {
        Serial.begin(9600);
        
        //uncomment these if using Teensy audio shield
        SPI.setSCK(14);  // Audio shield has SCK on pin 14
        SPI.setMOSI(7);  // Audio shield has MOSI on pin 7
        
        SerialFlash.begin();
        //printFlashChipFiles();
        
        // tact switches input shift register
        pinMode(SHIFT_IN_PLOAD_PIN, OUTPUT);
        //pinMode(inClockEnablePin, OUTPUT);
        pinMode(SHIFT_IN_CLOCK_PIN, OUTPUT);
        pinMode(SHIFT_IN_DATA_PIN, INPUT);
        // init input shift register
        digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
        digitalWrite(SHIFT_IN_PLOAD_PIN, HIGH);
        
        // Audio connections require memory to work.  For more
        // detailed information, see the MemoryAndCpuUsage example
        AudioMemory(20);
        
        // turn on the output
        audioShield.enable();
        audioShield.inputSelect(AUDIO_INPUT_MIC);
        audioShield.volume(0.5);
        
        // by default the Teensy 3.1 DAC uses 3.3Vp-p output
        // if your 3.3V power has noise, switching to the
        // internal 1.2V reference can give you a clean signal
        //dac1.analogReference(INTERNAL);
        
        // reduce the gain on mixer channels, so more than 1
        // sound can play simultaneously without clipping
        mixer1.gain(0, 0.3f);
        mixer1.gain(1, 0.3f);
        mixer1.gain(2, 0.1f);
        mixer1.gain(3, 0.3f);
        mixer1.gain(4, 0.1f);
        mixer1.gain(5, 0.3f);
        
        mixer2.gain(0, 0.3f);
        mixer2.gain(1, 0.3f);
        mixer2.gain(2, 0.3f);
        mixer2.gain(3, 0.1f);
        mixer2.gain(4, 0.3f);
        mixer2.gain(5, 0.1f);
        
        //sequencer = Sequencer();
        //Serial.print("sampleMode is:");
        //Serial.println(sequencer.sampleMode);
        
        for (int i=0; i<NUMBER_OF_INSTRUMENTTRACKS; i++){
            envelopes[i]->decay(5);
            envelopes[i]->sustain(0.0f);
        }
        
        FastLED.addLeds<WS2812B, DATA_PIN, GRB>(sequencer.leds, NUM_LEDS);
        FastLED.setBrightness(100);
        
        Serial.println("--Setup end");
        
        
        for (int i = 0; i < NUM_LEDS; i++){
            for (int j = 0; j < NUM_LEDS; j++){
                sequencer.leds[j] = CRGB::Black;
            }
            sequencer.leds[i] = CRGB::Red;
            FastLED.show();
            delay(20);
        }
        for (int i = NUM_LEDS-1; i >= 0; i--){
            for (int j = 0; j < NUM_LEDS; j++){
                sequencer.leds[j] = CRGB::Black;
            }
            sequencer.leds[i] = CRGB::Red;
            FastLED.show();
            delay(20);
        }
        sequencer.leds[0] = CRGB::Black;
        FastLED.show();
        
        
    }
    
    
    
    
    
    
    static int recordBufferOffset = 0;
    
    void startRecording() {
        Serial.println("startRecording");
        const char *filename = "0.RAW";
        switch(sequencer.getSelectedTrack()){
            case 0:
                filename = "0.RAW";
                break;
            case 1:
                filename = "1.RAW";
                break;
            case 2:
                filename = "2.RAW";
                break;
            case 3:
                filename = "3.RAW";
                break;
            case 4:
                filename = "4.RAW";
                break;
            case 5:
                filename = "5.RAW";
                break;
            default:
                return;
        }
        
        if (SerialFlash.exists(filename)) {
            SerialFlash.remove(filename);
            Serial.println("removed old file");
        }
        if (SerialFlash.create(filename, SAMPLEBUFFER_LENGTH)) {
            toRecordFile = SerialFlash.open(filename);
            //Serial.println("created and opened new file");
            if (toRecordFile){
                Serial.println("toRecordFile is true");
                queue1.begin();
                sequencer.sampleMode = SamplingMode::RECORDING;
                recordBufferOffset = 0;
    
            } else {
                Serial.println("cannot record..");
                sequencer.sampleMode = SamplingMode::STOPPED;
            }
        } else {
            Serial.print("cannot create file");
            Serial.println(filename);
            sequencer.sampleMode = SamplingMode::STOPPED;
        }
    }
    
    
    
    void continueRecording() {
        //Serial.println("continueRecording()");
        while (queue1.available() >= 1) {
            //Serial.println("queue contains data");
            
            if (toRecordFile){
                //Serial.println("file still there");
                if (recordBufferOffset >= SAMPLEBUFFER_LENGTH){
                    //Serial.println("finished recording 7.RAW");
                    sequencer.sampleMode = SamplingMode::STOPPED;
                    queue1.clear();
                    queue1.end();
                    toRecordFile.close();
                    //printFlashChipFiles();
                } else {
                    //Serial.println("writing 256 bytes of recording, offset is");
                    toRecordFile.write(queue1.readBuffer(), 256);
                    queue1.freeBuffer();
                    recordBufferOffset += 256;
                    //Serial.println(recordBufferOffset);
                }
            } else {
                //Serial.println("file not there");
            }
        }
    }
    
    CRGB colorForStepState(uint8_t state){
        switch (state) {
            case 1:
                //trigger on / plock rec off
                return CRGB::CornflowerBlue;
            case 2:
                //trigger off / plock rec on
                return CRGB::Green;
            case 3:
                //trigger on / plock rec on
                return CRGB::DarkOrange;
            default:
                //trigger off / plock rec off
                return CRGB::Black;
        }
    }
    
    /*
     * updates all button states by reading in the values from the shift registers
     */
    void readButtonStates(){
        // Trigger a parallel Load to latch the state of the data lines in the input shift register.
        digitalWrite(SHIFT_IN_PLOAD_PIN, LOW);
        delayMicroseconds(PULSE_WIDTH_USEC);
        digitalWrite(SHIFT_IN_PLOAD_PIN, HIGH);
        
        // read track buttons states from shift register
        for (int i = NUMBER_OF_TRACKBUTTONS-1;  i >= 0; i--){
            sequencer.trackButtons[i].update(digitalReadFast(SHIFT_IN_DATA_PIN));
            //Pulse the Clock (rising edge shifts the next bit).
            digitalWrite(SHIFT_IN_CLOCK_PIN, HIGH);
            //delayMicroseconds(PULSE_WIDTH_USEC);
            digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
        }
        
        // read step button states from shift register
        for (int i = NUMBER_OF_STEPBUTTONS-1;  i >= 0; i--){
            sequencer.stepButtons[i].update(digitalReadFast(SHIFT_IN_DATA_PIN));
            //Pulse the Clock (rising edge shifts the next bit).
            digitalWrite(SHIFT_IN_CLOCK_PIN, HIGH);
            //delayMicroseconds(PULSE_WIDTH_USEC);
            digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
        }
        
        // read step button states from shift register
        for (int i = NUMBER_OF_FUNCTIONBUTTONS-1;  i >= 0; i--){
            sequencer.functionButtons[i].update(digitalReadFast(SHIFT_IN_DATA_PIN));
            //Pulse the Clock (rising edge shifts the next bit).
            digitalWrite(SHIFT_IN_CLOCK_PIN, HIGH);
            //delayMicroseconds(PULSE_WIDTH_USEC);
            digitalWrite(SHIFT_IN_CLOCK_PIN, LOW);
        }
        
        
    }
    
    
    
    void updateAudio() {
        
        if (millis() - last_step_time >= step_length) {
            
            sequencer.tick();
            last_step_time = millis();
            
            uint16_t sensorValue1 = (uint16_t)analogRead(POTI_PIN_1);
            uint16_t sensorValue2 = (uint16_t)analogRead(POTI_PIN_2);
            
            //Serial.print("sensor1 :");
            //Serial.print(sensorValue1);
            //Serial.print("sensor2 :");
            //Serial.println(sensorValue2);
            
            if (sequencer.pulseCount == 0){
                
                //handle tempo track...
                //TODO should not be handled separately
                sequencer.tracks[6].doStep();
                SequencerStep * step = sequencer.tracks[6].getCurrentStep();
                
                if (step->isParameterLockOn()){
                    step->parameter1 = sensorValue1;
                }
                if (step->isTriggerOn()){
                    step_length = map(step->parameter1, 0, 1024, 0, 128);
                }
                
                // trigger tracks if step is set to on
                for (int i = 0; i < NUMBER_OF_INSTRUMENTTRACKS; i++) {
                    
                    //advance one step;
                    sequencer.tracks[i].doStep();
                    // checks if the bit at position of the current_step is set to 1 in the step on/off integer
                    SequencerStep * step = sequencer.tracks[i].getCurrentStep();
                    
                    if (step->isParameterLockOn()){
                        step->parameter1 = sensorValue1;
                        step->parameter2 = sensorValue2;
                    }
                }
                
                if (sequencer.sampleMode == SamplingMode::ARMED && sequencer.tracks[sequencer.getSelectedTrack()].currentStep == 0){
                    startRecording();
                }
            }
            
            // punch in style recording of the trigger pattern for the currently selected track
            uint8_t triggerPattern = 0b00000001;
            if (sequencer.functionButtons[5].read()){
                if (sensorValue1 < 200){
                    triggerPattern = 0b00000001;
                } else if (sensorValue1 < 400){
                    triggerPattern = 0b00000101;
                } else if (sensorValue1 < 600){
                    triggerPattern = 0b00000111;
                } else if (sensorValue1 < 800){
                    triggerPattern = 0b00010101;
                } else {
                    triggerPattern = 0b00111111;
                }
                sequencer.tracks[sequencer.getSelectedTrack()].getCurrentStep()->triggerPattern = triggerPattern;
            }
            
            
            // trigger tracks if step is set to on
            for (int i = 0; i < NUMBER_OF_INSTRUMENTTRACKS; i++) {
                SequencerStep * step = sequencer.tracks[i].getCurrentStep();
                
                if (!sequencer.tracks[i].isMuted() && step->isTriggerOn()
                    //check if the bit for the n-th pulse of this step is set
                    && (step->triggerPattern & (1 << sequencer.pulseCount))){
                    if (sequencer.functionButtons[4].read()){
                        players[i]->frequency(sensorValue1);
                        envelopes[i]->hold(map(sensorValue2, 0, 1024, 0, MAX_ENV_HOLD_TIME));
                    } else {
                        players[i]->frequency((float)step->parameter1);
                        envelopes[i]->hold(map(step->parameter2, 0, 1024, 0, MAX_ENV_HOLD_TIME));
                    }
                    envelopes[i]->noteOn();
                    switch (i) {
                        case 0:
                            players[i]->play("0.RAW");
                            break;
                        case 1:
                            players[i]->play("1.RAW");
                            break;
                        case 2:
                            players[i]->play("2.RAW");
                            break;
                        case 3:
                            players[i]->play("3.RAW");
                            break;
                        case 4:
                            players[i]->play("4.RAW");
                            break;
                        case 5:
                            players[i]->play("5.RAW");
                            break;
                        default:
                            break;
                    }
                    
                }
            }
            
        }
        if (sequencer.sampleMode == SamplingMode::RECORDING) {
            continueRecording();
        }
    }
    
    
    
    
    void loop() {
        
        readButtonStates();
        sequencer.updateState();
        FastLED.show();
        
        if (sequencer.isRunning()){
            updateAudio();
    
            if (0){
    
                Serial.print("CPU: ");
                Serial.print("all=");
                Serial.print(AudioProcessorUsage());
                Serial.print(",");
                Serial.print(AudioProcessorUsageMax());
                Serial.print("    ");
                Serial.print("Memory: ");
                Serial.print(AudioMemoryUsage());
                Serial.print(",");
                Serial.print(AudioMemoryUsageMax());
                Serial.println();
    
            }
        }
        
    }

  2. #2
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    11,948
    Is there a tool to walk the storage structures to see the state of the system and show free space? Number of files and size and walk through them showing the length and key details - i.e. chkdsk?

    I see "Example / Listfiles" that may indicate where the problem lies. Not sure if it will indicate anything about your problem. You seem to have some part of this in your sketch.

    I find this interesting - but have not looked at it yet in any way - though I did buy chips and memory boards.

    To continue to use a file per the GitHub readme - it seems you should createErasable and then to start it anew would have to "SerialFlash.remove(filename);"?

    Each byte may be written only once. Files created as erasable may be fully erased, to allow new data to be written.

    Paul or Frank would know - but I guess adding the same named file just abandons the old copy and starts fresh - and it can only do this until the flash fills - which is I think what you are seeing and why a full erase lets you work again.

    This would seem like a common use case and an Example to that effect would be instructive.

  3. #3
    Hi, thanks a lot for your answer.
    Indeed it seems that the flash chip is filling up and finally running out of space. I could verify this by counting the number of successful writes before the problem starts to occur. Its always the same number and the number is consistent with my sample file size and the overall size of the flash (131072 bytes * 127 writes is around 16777216 bytes total flash memory). This is good to know. However the bad news is that changing the code to SerialFlash.createErasable did not solve the problem. Still running out of space, even though the debug log output indicates that when an old file with the same name exists, it is sucessfully removed, before a new file with the same name is created and filled with the recorded samples.
    This is very strange.. any other ideas?

    Edit: i tried listing all files and cannot see any problems in that output:

    All Files on SPI Flash chip:
    0.RAW 131072 bytes
    1.RAW 131072 bytes
    2.RAW 131072 bytes
    3.RAW 131072 bytes
    4.RAW 131072 bytes
    5.RAW 131072 bytes
    --Setup end
    Last edited by a_guy_called_tom; 11-28-2015 at 07:17 PM.

  4. #4
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    11,948
    @a_guy_called_tom: If you can create a skeleton of your sketch that would be easier to work with using one/some instances of - createErasable - write to file(s) then - SerialFlash.remove() then Paul or IIRC FrankB might look into it to resolve it.

    I'd suggest starting with a minimal existing example - remove anything not needed then put in this functionality in some reasonable form that might make another good example for general use.

    <edit>: And post that sample here: Name:  code.PNG
Views: 398
Size:  1.7 KB
    Last edited by defragster; 11-28-2015 at 08:31 PM.

  5. #5
    Heres a minimal sample that reproduces the problem. With a freshly erased flash memory, the code in the loop method executed 128 times. After that, a new file with the same name cannot be created anymore, even though the log output indicates, that the previous files with the same name were removed.

    Any help is appreciated.

    Code:
    #include <SerialFlash.h>
    #include <SPI.h>
    
    #define SAMPLEBUFFER_LENGTH 131072
    
    static int iteration = 0;
    
    
    
    void setup() {
        Serial.begin(9600);
     
        SPI.setSCK(14);  // Audio shield has SCK on pin 14
        SPI.setMOSI(7);  // Audio shield has MOSI on pin 7
        
        SerialFlash.begin();
        printFlashChipFiles();
        
        Serial.println("--Setup end");       
    }
    
    void loop() {
          iteration++;
          Serial.printf("Starting iteration: %i \n", iteration);
       
          
          const char *filename = "0.RAW";
          SerialFlashFile flashFile;
    
        
          // check for existing file with the same name, and remove it is there.
          if (SerialFlash.exists(filename)) {
              Serial.println("theres an existing file");
              if (SerialFlash.remove(filename)){
                Serial.println("removed old file");
              } else{
                error("Error: could not remove existing old file");
              }
          }
          // create a new file
          if (SerialFlash.createErasable(filename, SAMPLEBUFFER_LENGTH)) {
              flashFile = SerialFlash.open(filename);
              if (flashFile){
                  Serial.println("flashFile is true, everything ok, will close the file now");
                  // this is where the recording would normally happen and the bytes written.
                  flashFile.close();
              } else {
                  error("Error: flashFile is not opened");
              }
          } else {
              error("Error cannot create file 0.RAW");
          }
    }
    
    //print error message
    void error(const char *message) {
        while (1) {
            Serial.println(message);
            delay(2500);
        }
    }
    
    //utility function
    void spaces(int num) {
        for (int i=0; i < num; i++) {
            Serial.print(" ");
        }
    }
    
    // list filenames and sizes of the flash memory chip
    void printFlashChipFiles() {
        // wait for Arduino Serial Monitor
        while (!Serial) ;
        delay(100);
        Serial.println("All Files on SPI Flash chip:");
        
        if (!SerialFlash.begin()) {
            error("Unable to access SPI Flash chip");
        }
        
        SerialFlash.opendir();
        while (1) {
            char filename[64];
            unsigned long filesize;
            
            if (SerialFlash.readdir(filename, sizeof(filename), filesize)) {
                Serial.print("  ");
                Serial.print(filename);
                spaces(20 - strlen(filename));
                Serial.print("  ");
                Serial.print(filesize);
                Serial.print(" bytes");
                Serial.println();
            } else {
                break; // no more files
            }
        }
        
    }
    This outputs (last few lines):

    ...
    Starting iteration: 125
    theres an existing file
    removed old file
    flashFile is true, everything ok, will close the file now
    Starting iteration: 126
    theres an existing file
    removed old file
    flashFile is true, everything ok, will close the file now
    Starting iteration: 127
    theres an existing file
    removed old file
    flashFile is true, everything ok, will close the file now
    Starting iteration: 128
    theres an existing file
    removed old file
    Error cannot create file 0.RAW
    Error cannot create file 0.RAW
    Error cannot create file 0.RAW
    Error cannot create file 0.RAW

  6. #6
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    22,276
    Space is not reused when you remove a file. SerialFlash gives you something that looks like a filesystem, but it's really just a very thin layer over the flash memory. It's not a full filesystem.

    Every time you remove the old file, more of the flash chip is used. Eventually you'll run out of space. I'm pretty sure that's what's happening here.

  7. #7
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    11,948
    Thanks Paul - I was under the impression a file 'createErasable ' would be selectively made available for re-use with SerialFlash.remove() - based on the function names and the size boundary creation detail on erasable "Files created for erasing automatically increase in size to the nearest number of erasable blocks, resulting in a file that may be 4K to 128K larger than requested."

    Also this text seemed to speak to this usage case:
    Delete A File
    SerialFlash.remove(filename);
    The actual space used by the file is not reclaimed. However, a new file with this name may be created after the original is deleted.
    However reading that in the context of your post - the first part takes precedence when it comes to space exhaustion. The odd thing is that it seemed to be working that same way for '...._tom' when he was not using the 'erasable' creation.

  8. #8
    Hi all, many thanks for your answers.

    I need to have a fixed number of samples on the flash memory and they all have the same fixed size (eg. 8 samples, 256KB each). But i need to be able to change the contents of the files or overwrite them, without running out of space eventually (each of my 8 sequencer tracks plays one specific sample that can be changed by recording a new sample).
    After reading Pauls answer, i'm not even sure if this is possible using the SerialFlash Library? If not, any hints on how this could be achieved (possibliy without diving into the real low-level territory which i'm not really familiar with)?

    Thanks,
    thomas

  9. #9
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    11,948
    As noted - not having looked at this in any way before - I assumed space from an erased file would get eraseBlock()'d and get put back into a free pool. In your case that is exactly what would work - as long as you could suffer the downtime of the block erase cycles - since you'd just re-use that block later in the same size it wouldn't even fragment the space.

    Perhaps this is planned for a future version as some groundwork would seem to be there. I see this Winbond chip has 32KB and 64KB block erase commands. The 64KB eraseBlock function is exercised in the example/rawhardwaretest. The problem would be the directory structure entry being 'write twice'? - once to create a file - and once again to zero it to show it is no longer a pointer to an active file. File directory block itself would have to be erased to support new file entries - or some other scheme using the same filename forcing the block erase of the data before continuing [i.e. REcreateErasable()] - or more robustly if the directory area allocated supports it: to migrate used dir entries to a new block and then erase the old one when more space was needed over time. Would be non-trivial but do-able with more down time for a 'defrag' to run. I didn't read the code far enough to see how big the directory area is - if it has multiple 32KB blocks - two half full or a free block it could be done while in use up to the block erase part stopping writes for the duration. If the directory data is given fixed space or less than a 32KB block - this would be a bigger change - like the fun of going from FAT16 to FAT32 was.

  10. #10
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    11,948
    @a_guy_called_tom: without a software change it seems like you would need to add a 2nd flash chip and make sure the other is empty and ready to migrate to and then clean and prep the other one behind you.

  11. #11
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    22,276
    Quote Originally Posted by a_guy_called_tom View Post
    I need to have a fixed number of samples on the flash memory and they all have the same fixed size (eg. 8 samples, 256KB each). But i need to be able to change the contents of the files or overwrite them, without running out of space eventually
    Ok. Just create them as erasable files. When you need to overwrite the file, just erase it. But never delete the file with remove().

    Actually, you might pre-erase the file. Erasing is slow, so if you need to begin recording with low latency, make sure you have a file already erased.


    Quote Originally Posted by defragster View Post
    I assumed space from an erased file would get eraseBlock()'d and get put back into a free pool.
    With SerialFlash, there's a subtle but important distinction between erased and removed/deleted. Erasing the file does not delete the file. It simply resets all the bytes in the file to 0xFF.

    There is no free pool or other conventional filesystem space management with SerialFlash. The files are simply created one after the next. Files are always created with a fixed size that can never grow (or shrink). Once created, the file's physical location in the chip is fixed.

    When you delete a file with remove(), the space is not reclaimed. The data isn't deleted either. It's just left there in the memory. Only the filename record is overwritten, which means all reference to the file is gone. Then you can create another file with that same name.

    I see this Winbond chip has 32KB and 64KB block erase commands. The 64KB eraseBlock function is exercised in the example/rawhardwaretest.
    It's also used when you call file.erase().

    You must create the file with SerialFlash.createErasable(). The normal SerialFlash.create() packs the file tightly, with only page padding to support the write and erase suspend on a file granularity level. SerialFlash.createErasable() aligns and pads the file to those erasable block boundaries, so you can later use the file.erase() function.


    Quote Originally Posted by defragster View Post
    seems like you would need to add a 2nd flash chip and make sure the other is empty and ready to migrate to and then clean and prep the other one behind you.
    SerialFlash only supports a single chip.

  12. #12
    Quote Originally Posted by PaulStoffregen View Post
    Ok. Just create them as erasable files. When you need to overwrite the file, just erase it. But never delete the file with remove().

    Actually, you might pre-erase the file. Erasing is slow, so if you need to begin recording with low latency, make sure you have a file already erased.
    Thanks Paul. Tried this, and it works - the flash no longer runs out of memory. However as you pointed out, erase is slow and is now blocking the whole instrument for about 400ms.

    1) Since erase just seems to reset the bytes to 0xFF, could i overwrite the file without erasing the old data first? Since im using raw wav data without any header, my assumption was that this might work, with the result that the first part of the file would be the new sample and the last part would still be the old sample during the write. I tried this out (simply commenting out the file.erase() before writing new data to the file, but what i hear (pretty nasty digital noise) tells me that this is not a good solution.
    2) Is there a way i can do file.erase() in a non-blocking way on the teensy? One goal in my design is that everything can be done while the sequencer is running, so it would be ideal if i could solve this problem without blocking the UI and audio out

    Sorry to bother you all with these questions, any help is much appreciated. (If your interested to know about the project your helping me with: https://vimeo.com/144595364)

    Thomas

  13. #13
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    22,276
    Erase is slow. Nothing will change that.

    However, SerialFlash could do more to allow you to not block your program.....

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •