Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 16 of 16

Thread: Can't get files on SPI flash

  1. #1
    Member
    Join Date
    Apr 2014
    Location
    Flevoland, 5 meters under water
    Posts
    32

    Can't get files on SPI flash

    hello there,

    At the moment I'm having some issues using the new SPI flash library.
    I don't know if it's a bug, as Paul still works on the library.
    A few files get flashed, but not all of them.
    Can someone help me out?

    My hardware setup is:

    Teensy 3.1 + audio board soldered together.
    A Winbond 25Q128FVSG flash chip soldered on the audio board.
    A 4GB Transcend MicroSD card, containing a few raw audio files.

    What I tried in the following order:
    eraseEverything example
    copyfromSD example.
    ListFiles example.

    This is where it goes wrong, given the output:

    Code:
    Copy all files from SD Card to SPI Flash
    
    CR3.TRW    147716
      copying..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    
    HH1CLO~1.TRW    7684
      copying...............................
    
    HH1CLO~2.TRW    9092
      error opening freshly created file!
    
    HH1HAL~1.TRW    20228
      error opening freshly created file!
    
    HH1OPEN.TRW    61316
      error opening freshly created file!
    
    K1.TRW    29700
      error opening freshly created file!
    
    K2.TRW    8836
      error opening freshly created file!
    
    SHAKE1.TRW    4484
      error opening freshly created file!
    
    R1.TRW    159748
      error opening freshly created file!
    
    R2.TRW    116996
      error opening freshly created file!
    
    RIM1.TRW    15108
      error opening freshly created file!
    
    S1.TRW    16260
      error opening freshly created file!
    to make it even funnier:
    the listFiles example lists these two files,
    but also the files that couldn't be created

    Code:
    All Files on SPI Flash chip:
      CR3.TRW               147716 bytes
      HH1CLO~1.TRW          7684 bytes
                            9092 bytes
                            20228 bytes
                            61316 bytes
                            29700 bytes
                            8836 bytes
                            4484 bytes
                            159748 bytes
                            116996 bytes
                            15108 bytes
                            16260 bytes
    After some debugging with serial prints,
    everything points to line 170 of SerialFlashDirectory.cpp.
    Code:
    else if (hashtable[i] == 0xFFFF) {
    				return file;
    			}
    Thanks in advance

  2. #2
    Member
    Join Date
    Apr 2014
    Location
    Flevoland, 5 meters under water
    Posts
    32
    Small update:

    The SerialFlash library still doesn't work,
    but the older sd2serialflash by FrankB does work.
    At least storage flash now works a little

    If anyone knows what I'm doing wrong, I would be glad to take advice.
    Not having to hardcode files would be great.

    Code:
    /*
    
       W25Q128FV Serial Flasher
       
       (c) Frank Boesing, 2014,Dec
       GNU License Version 3.
    
       Teensy Audio Shield (c) Paul Stoffregen 
       W25Q128FV - Library  (c) Pete (El Supremo) 
       Thank you both!!!!
          
       Reads directory "/SERFLASH" on SD and burns 
       all files to the optional serial flash.
      
       Version 20141227
          
    */
    
    #include <Audio.h>
    #include <SD.h>
    #include <SPI.h>
    #include <Wire.h>
    #include <flash_spi.h>
    
    #define FLASHSIZE 0x1000000
    #define PAGE      256
    #define DIRECTORY "SERFLASH"
    
    
    
    File dir;
    File entry;
    unsigned char id_tab[32];
    unsigned pos;
    unsigned page;
    int fsize = 0;
    int fcnt = 0;
    unsigned char buf[PAGE];
    
    String filename[500];
    unsigned int position[500];
    
    bool verify(void)
    {
    
    	unsigned char buf2[PAGE];
    
        fcnt = 0;
        pos = 0; 
        page = 0;     
        Serial.println("Verify.");
        dir = SD.open(DIRECTORY);
        while(1) {
          entry = dir.openNextFile();
          if (!entry) break;
          pos = page * PAGE;
          Serial.printf("%d. Verifying \"%s\" at position: 0x%07X...", fcnt+1, entry.name(), pos);
    	  filename[fcnt] = entry.name();
    	  position[fcnt] = pos;
          int rd =0;
          do {
            memset(buf, 0xff, PAGE);
            rd = entry.read(buf, PAGE);
            flash_read_pages(buf2, page, 1);
            int v = 0;
            for (int i=0; i<PAGE; i++) v+= buf[i]-buf2[i];
            if (v) {Serial.println("is not ok."); return false;}
            pos += rd;         
            page++;
          } while (rd==PAGE);          
          Serial.printf("ok.\r\n");
          entry.close(); 
          fcnt++;
        }
        return true;
    }
    
    void flash(void)
    {
    	unsigned char buf[PAGE];
        fcnt = 0;
        pos = 0; 
        page = 0;     
        dir = SD.open(DIRECTORY);
        while(1) {
          entry = dir.openNextFile();
          if (!entry) break;
          pos = page * PAGE;
          Serial.printf("%d. Flashing \"%s\" at position: 0x%07X...", fcnt+1, entry.name(), pos);
          int rd =0;
          do {
            memset(buf, 0xff, PAGE);          
            rd = entry.read(buf, PAGE);
            pos += rd;         
            flash_page_program(buf,page);
            page++;
          } while (rd==PAGE);          
          Serial.printf("done.\r\n");
          entry.close(); 
          fcnt++;
        }
    }
    
    void erase(void) {
        Serial.println("Erasing chip....");
        flash_chip_erase(true);
        Serial.println("done.");
    }
    
    void setup()
    {
      SPI.setMOSI(7);
      SPI.setSCK(14);
      // Open serial communications and wait for port to open:
      Serial.begin(9600);
       while (!Serial) {;}  
      delay(1000); 
      Serial.print("\r\n\r\nW25Q128FV Serial Flasher \r\nInitializing SD card...");
      if (!SD.begin(10)) {
        Serial.println("failed!");
        return;
      }  
      Serial.println("done.\r\n");
      dir = SD.open(DIRECTORY);
      fsize = 0;
      fcnt = 0;
      while(1) 
      {
        entry = dir.openNextFile();
        if (!entry) break;    
        int s = entry.size();   
        if ( (s & 0xff) > 0) s |= 0xff;
        Serial.printf("%s\t\t\t%d Bytes\r\n", entry.name(), s);
        fsize += s;   
        entry.close();  
        fcnt++;
      } 
      dir.close();
      
      Serial.printf("\t\t\t%d File(s), %d Bytes\r\n", fcnt, fsize);
    
      if (!fsize) goto ready;
      if (fsize < FLASHSIZE) {
          flash_init();
          int flashstatus = flash_read_status();
          flash_read_id(id_tab);
          Serial.printf("Flash Status: 0x%X, ID:0x%X,0x%X,0x%X,0x%X ", flashstatus , id_tab[0], id_tab[1], id_tab[2], id_tab[3]);   
          if (id_tab[0]!=0xef || id_tab[1]!=0x40 || id_tab[2]!=0x18 || id_tab[3]!=0x00) {Serial.println(" is not ok."); goto end;}
          Serial.printf(" is ok.\r\nFile(s) fit(s) in serial flash, %d Bytes remaining.\r\n\r\n", FLASHSIZE - fsize);
    
          Serial.print("Check flash content: ");
          if (verify()) { Serial.println("Flash content ok. Nothing to do.");goto end; }
    
          erase();
          flash();      
          verify();      
    
    end:            
          dir.close();
      }
      else Serial.println("Does not fit.");  
      
    ready:  
      Serial.println("Ready.");
      if (fsize) { 
    	Serial.printf("Copy'n paste:\r\n\r\nconst int SPIFlash[%d] = {\r\n",fcnt);
    	for (int i = 0; i<fcnt; i++) 
    		Serial.printf("\t\t0x%07X%s //\"%s\"\r\n",position[i], ((i<fcnt-1)?", ":"};"), filename[i].c_str());
    	
    	
    	Serial.printf("\r\nconst char SPIFlashFilename[%d][13] = {\r\n",fcnt);
    	for (int i = 0; i<fcnt; i++) 
    		Serial.printf("\t\t\"%s\"%s\r\n", filename[i].c_str(),((i<fcnt-1)?",":"};"));	
      }
      
    }
    
    
    
    const int SPIFlash[7] = {
                    0x0000000,  //"201159~1.RAW"
                    0x002C800,  //"82583_~1.RAW"
                    0x0053300,  //"86334_~1.RAW"
                    0x0065B00,  //"86773_~1.RAW"
                    0x01F1D00,  //"102790~1.RAW"
                    0x020A100,  //"171104~1.RAW"
                    0x0214900}; //"P2.RAW"
    
    const char SPIFlashFilename[7][13] = {
                    "201159~1.RAW",
                    "82583_~1.RAW",
                    "86334_~1.RAW",
                    "86773_~1.RAW",
                    "102790~1.RAW",
                    "171104~1.RAW",
                    "P2.RAW"};
    
    
    
    void loop()
    {}

  3. #3
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,304
    Can you post these files, or email them to me. I'll give it a try here.

  4. #4
    Member
    Join Date
    Apr 2014
    Location
    Flevoland, 5 meters under water
    Posts
    32
    Of course, I'll send the files this weekend
    Thanks in advance

  5. #5
    Member
    Join Date
    Apr 2014
    Location
    Flevoland, 5 meters under water
    Posts
    32
    trw files.zip
    @Paul,

    here is a collection of files of which some did, and some didn't work for me.

  6. #6
    Senior Member+ manitou's Avatar
    Join Date
    Jan 2013
    Posts
    2,077
    I don't have the same hardware setup, but I do have a 1MB winbond DIP SPI flash. Not using an SD, but just doing the same sort of serial flash file creates, I could not reproduce the problem on teensy 3.1. (i'll try with an SD shortly, but it still won't be the same hardware config).

    Paul, the 1MB serial flash from adafruit http://www.adafruit.com/product/1564
    seems to work with your SerialFlash library. To windbond ID check in RawHardwareTest example I added
    if (id[2] == 0x14) return "W25Q80BV";
    I'll do the "issue" thing on github.

    Edit, even without the SD logic I can get a similar failure with this sketch

    Code:
    #include <SerialFlash.h>
    #include <SPI.h>
    
    const int FlashChipSelect = 6;
    
    void dofile(char *filename, unsigned long length) {
        Serial.print(filename);
        Serial.print("    ");
        Serial.println(length);
    
        // check if this file is already on the Flash chip
        if (SerialFlash.exists(filename)) {
          Serial.println("  already exists on the Flash chip");
          Serial.println("  delete file from Flash chip");
          SerialFlash.remove(filename);
        }
      
        // create the file on the Flash chip and copy data
        if (SerialFlash.create(filename, length)) {
          SerialFlashFile ff = SerialFlash.open(filename);
          if (ff) {
            // copy data loop
            unsigned long count = 0;
            while (length) {
              char buf[256];
              unsigned int n;
    		  if (length >= 256) n =256;
    		   else n = length;
              ff.write(buf, n);
              length -= n;
              Serial.print(".");
            }
            ff.close();
            Serial.println();
          } else {
            Serial.println("  error opening freshly created file!");
          }
        } else {
          Serial.println("  unable to create file");
        }
    }
    
    
    void setup() {
      //uncomment these if using Teensy audio shield
    //  SPI.setSCK(14);  // Audio shield has SCK on pin 14
     // SPI.setMOSI(7);  // Audio shield has MOSI on pin 7
    
      // wait up to 10 seconds for Arduino Serial Monitor
      unsigned long startMillis = millis();
      while (!Serial && (millis() - startMillis < 10000)) ;
      delay(100);
      Serial.println("create some files on SPI Flash");
      SerialFlash.begin();
    
     dofile("TST.DAT",147716);
     dofile("CR3.TRW",147716);
      dofile("HH1CLO~1.TRW",7684);
      dofile("HH1CLO~2.TRW",9092);
    }
    
    void loop() {
    }
    
    void error(const char *message) {
      while (1) {
        Serial.println(message);
        delay(2500);
      }
    }
    output is
    Code:
    create some files on SPI Flash
    TST.DAT    147716
    ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    CR3.TRW    147716
    ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    HH1CLO~1.TRW    7684
    ...............................
    HH1CLO~2.TRW    9092
      error opening freshly created file!
    and the behavior of example ListFiles is
    Code:
    All Files on SPI Flash chip:
      TST.DAT               147716 bytes
      CR3.TRW               147716 bytes
      HH1CLO~1.TRW          7684 bytes
                            9092 bytes
    if i change the name of the last file to tom.TRW then all are created ??
    i'll probe a little deeper ... I suspect a buffer overflow, results change with Serial.print's inserted
    ... lots of magic numbers 8 10 12
    Last edited by manitou; 08-28-2018 at 02:45 PM.

  7. #7
    Senior Member+ manitou's Avatar
    Join Date
    Jan 2013
    Posts
    2,077
    I think the problem "error opening freshly created file!" arises from an error in rounding the string storage up to a multiple of 4 in SerialFlashDirectory.cpp.
    The calculation should be
    straddr += string_length(straddr) + 1;

  8. #8
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,304
    Yes, very likely a bug on my part in the library!

    I'm currently at Maker Farie, without access to my desktop where I keep the list of bugs to investigate and fix. Please bump this thread on Tuesday, or post an issue on this library's github, so it won't be forgotten.

  9. #9
    Senior Member+ manitou's Avatar
    Join Date
    Jan 2013
    Posts
    2,077
    maybe another bug (feature?) in readdir(). example ListFiles will show all files, even those "removed" because readdir() does not check the hash value. (hash value of 0 means file has been removed)

    might also be nice to have a file method that reports free space remaining ....

    -------
    (never mind following ....)
    After EraseEverything example (teensy 3.1), i'm also still experiencing "unable to create files", but adding a Serial.print or a delay here and there often fixes it. Enabling the print in check_signature() also fixes it, also delay(1) before SerialFlash.write(0, sig, 8); in check_signature() seems to fix it, though delayMicroseconds(1000) does NOT fix it ???
    Or adding delayMicroseconds(1); to CSRELEASE seemed to work .... maybe not ?

    on teensy 3.0 i have not experienced "unable to create files"

    (using 1.0.6 1.21)

    Edit: using 1.6.0 and 1.21, my create files tests seem to work OK on 3.1 -- i need to catch up with the new IDE releases.

    Edit 2: installed 1.6.3 and 1.23, SerialFlash tests seem to work, no problems with create on teensy 3.1
    Last edited by manitou; 05-17-2015 at 11:24 AM.

  10. #10
    Member
    Join Date
    Apr 2014
    Location
    Flevoland, 5 meters under water
    Posts
    32
    Hey Paul,

    Here is the bump you asked for.

    Have a nice day
    Last edited by Roel; 05-19-2015 at 02:13 PM. Reason: typo

  11. #11
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,304
    I'm trying this now, with the files from #5.

    Code:
    Copy all files from SD Card to SPI Flash
    
    HH1CLE.TRW    9092
      copying....................................
    
    HH1CL.TRW    7684
      copying...............................
    
    HH1HO.TRW    20228
      copying................................................................................
    
    K1.TRW    29700
      copying.....................................................................................................................
    
    K2.TRW    8836
      copying...................................
    
    R1.TRW    159748
      copying.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    
    R2.TRW    116996
      copying..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................
    
    RIM1.TRW    15108
      copying............................................................
    It seems to be working fine, at least with this set of files.

    However, the file names don't match the names in the text above. Message #1 has "HH1HAL~1.TRW" failing. None of the files from #5 are longer than 8.3 format.

    Is there another set of files I should try?

  12. #12
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,304
    Quote Originally Posted by manitou View Post
    I think the problem "error opening freshly created file!" arises from an error in rounding the string storage up to a multiple of 4 in SerialFlashDirectory.cpp.
    The calculation should be
    straddr += string_length(straddr) + 1;
    Oh, yes, it looks like string_length() isn't including the null terminator in the length calculation.

    Still, I'd really like to have a set of files that reproduces this problem before committing a fix. Anyone?!


    Edit: ok, here's a set that causes the error!
    Attached Files Attached Files
    Last edited by PaulStoffregen; 05-23-2015 at 05:26 AM.

  13. #13
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    20,304
    Ok, I've committed a fix.

    https://github.com/PaulStoffregen/Se...55d2ee6758597b

    Please give the latest code a try and let me know if it fully resolves the problems you're seeing?

  14. #14
    Senior Member+ manitou's Avatar
    Join Date
    Jan 2013
    Posts
    2,077
    Quote Originally Posted by PaulStoffregen View Post
    Still, I'd really like to have a set of files that reproduces this problem before committing a fix. Anyone?!
    I had a sketch in this thread that demonstrated the string_length problem. Your fixes look good. string_length problem fixed, my winbond chip is ID'd in the RawHardwareTest example, and the directory list (ListFiles) no longer shows removed files. Check.

  15. #15
    Member
    Join Date
    Apr 2014
    Location
    Flevoland, 5 meters under water
    Posts
    32
    Hey Paul,

    Thanks very much for your fix!
    I retried to flash even more files,
    and it copied them all without the slightest complaint

  16. #16
    Senior Member+ manitou's Avatar
    Join Date
    Jan 2013
    Posts
    2,077
    Just for the record, here is another Winbond SPI Serial flash that works with teensy SerialFlash lib. SPI flash is 2MB W25Q16BV. example RawHardwareTest reports:
    Code:
    Raw SerialFlash Hardware Test
    Read Chip Identification:
      JEDEC ID:     1 40 15
      Part Nummber: (unknown chip)
      Memory Size:  2097152 bytes
      Block Size:   65536 bytes
    Reading Chip...
    Writing 1024 signatures
    Double Checking All Signatures:
      all 1024 signatures read ok
    Checking Signature Pairs
      all 511 signature pairs read ok
    Checking Read-While-Write (Program Suspend)
      write 256 bytes at 256
      write time was 656 microseconds.
      read-while-writing: 00 00 00 00 15 F5 95 4B
      test passed, good read while writing
    Checking Read-While-Erase (Erase Suspend)
      erase time was 412833 microseconds.
      erase correctly erased 65536 bytes
      read-while-erasing: 00 00 00 00 15 F5 95 4B
      test passed, good read while erasing
    All Tests Passed  :-)
    flash chip actually resides on and tested on Adafruit circuit playground express (ARM M0+).

    Adafruit has a SPI flash library that supports FatFs and I've tried to run that Adafruit SPI flash lib on the teensy 3.2+propshield, but I get oodles of compile errors ...

    Re: FatFs on prop shield SPI serial flash
    UPDATE: The compile errors come from conflict definition of BYTE in fatfs integer.h (typedef) and teensy3/Print.h (#define). I changed integer.h to use a define, and I could compile and run the fatfs on the T3.2 with propshield SPI flash. The Adafruit lib is hardwired for the 2MB SPI flash, but a new symbol and specs could be added for the prop shield 8MB SPI flash. The teensy (T3.2@120mhz with 30mhz SPI) SPI and the resulting SPI flash IO is about 4 times faster than the Adafruit circuit playground Express. The fatfs does read/erase/update/write on the smallest erasable unit (4KB) to update files on the SPI flash.

    Of course there are a lot of arguments of why you wouldn't the overhead of FatFs on an embedded SPI serial flash ...
    Last edited by manitou; 06-25-2018 at 07:22 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •