Can't get files on SPI flash

Roel · Apr 30, 2015

hello there,

At the moment I'm having some issues using the new SPI flash library.
I don't know if it's a bug, as Paul still works on the library.
A few files get flashed, but not all of them.
Can someone help me out?

My hardware setup is:

Teensy 3.1 + audio board soldered together.
A Winbond 25Q128FVSG flash chip soldered on the audio board.
A 4GB Transcend MicroSD card, containing a few raw audio files.

What I tried in the following order:
eraseEverything example
copyfromSD example.
ListFiles example.

This is where it goes wrong, given the output:

Code:

Copy all files from SD Card to SPI Flash

CR3.TRW    147716
  copying..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

HH1CLO~1.TRW    7684
  copying...............................

HH1CLO~2.TRW    9092
  error opening freshly created file!

HH1HAL~1.TRW    20228
  error opening freshly created file!

HH1OPEN.TRW    61316
  error opening freshly created file!

K1.TRW    29700
  error opening freshly created file!

K2.TRW    8836
  error opening freshly created file!

SHAKE1.TRW    4484
  error opening freshly created file!

R1.TRW    159748
  error opening freshly created file!

R2.TRW    116996
  error opening freshly created file!

RIM1.TRW    15108
  error opening freshly created file!

S1.TRW    16260
  error opening freshly created file!

to make it even funnier:
the listFiles example lists these two files,
but also the files that couldn't be created

Code:

All Files on SPI Flash chip:
  CR3.TRW               147716 bytes
  HH1CLO~1.TRW          7684 bytes
                        9092 bytes
                        20228 bytes
                        61316 bytes
                        29700 bytes
                        8836 bytes
                        4484 bytes
                        159748 bytes
                        116996 bytes
                        15108 bytes
                        16260 bytes

After some debugging with serial prints,
everything points to line 170 of SerialFlashDirectory.cpp.

Code:

else if (hashtable[i] == 0xFFFF) {
				return file;
			}

Thanks in advance

Roel · May 7, 2015

Small update:

The SerialFlash library still doesn't work,
but the older sd2serialflash by FrankB does work.
At least storage flash now works a little

If anyone knows what I'm doing wrong, I would be glad to take advice.
Not having to hardcode files would be great.

Code:

/*

   W25Q128FV Serial Flasher
   
   (c) Frank Boesing, 2014,Dec
   GNU License Version 3.

   Teensy Audio Shield (c) Paul Stoffregen 
   W25Q128FV - Library  (c) Pete (El Supremo) 
   Thank you both!!!!
      
   Reads directory "/SERFLASH" on SD and burns 
   all files to the optional serial flash.
  
   Version 20141227
      
*/

#include <Audio.h>
#include <SD.h>
#include <SPI.h>
#include <Wire.h>
#include <flash_spi.h>

#define FLASHSIZE 0x1000000
#define PAGE      256
#define DIRECTORY "SERFLASH"



File dir;
File entry;
unsigned char id_tab[32];
unsigned pos;
unsigned page;
int fsize = 0;
int fcnt = 0;
unsigned char buf[PAGE];

String filename[500];
unsigned int position[500];

bool verify(void)
{

	unsigned char buf2[PAGE];

    fcnt = 0;
    pos = 0; 
    page = 0;     
    Serial.println("Verify.");
    dir = SD.open(DIRECTORY);
    while(1) {
      entry = dir.openNextFile();
      if (!entry) break;
      pos = page * PAGE;
      Serial.printf("%d. Verifying \"%s\" at position: 0x%07X...", fcnt+1, entry.name(), pos);
	  filename[fcnt] = entry.name();
	  position[fcnt] = pos;
      int rd =0;
      do {
        memset(buf, 0xff, PAGE);
        rd = entry.read(buf, PAGE);
        flash_read_pages(buf2, page, 1);
        int v = 0;
        for (int i=0; i<PAGE; i++) v+= buf[i]-buf2[i];
        if (v) {Serial.println("is not ok."); return false;}
        pos += rd;         
        page++;
      } while (rd==PAGE);          
      Serial.printf("ok.\r\n");
      entry.close(); 
      fcnt++;
    }
    return true;
}

void flash(void)
{
	unsigned char buf[PAGE];
    fcnt = 0;
    pos = 0; 
    page = 0;     
    dir = SD.open(DIRECTORY);
    while(1) {
      entry = dir.openNextFile();
      if (!entry) break;
      pos = page * PAGE;
      Serial.printf("%d. Flashing \"%s\" at position: 0x%07X...", fcnt+1, entry.name(), pos);
      int rd =0;
      do {
        memset(buf, 0xff, PAGE);          
        rd = entry.read(buf, PAGE);
        pos += rd;         
        flash_page_program(buf,page);
        page++;
      } while (rd==PAGE);          
      Serial.printf("done.\r\n");
      entry.close(); 
      fcnt++;
    }
}

void erase(void) {
    Serial.println("Erasing chip....");
    flash_chip_erase(true);
    Serial.println("done.");
}

void setup()
{
  SPI.setMOSI(7);
  SPI.setSCK(14);
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {;}  
  delay(1000); 
  Serial.print("\r\n\r\nW25Q128FV Serial Flasher \r\nInitializing SD card...");
  if (!SD.begin(10)) {
    Serial.println("failed!");
    return;
  }  
  Serial.println("done.\r\n");
  dir = SD.open(DIRECTORY);
  fsize = 0;
  fcnt = 0;
  while(1) 
  {
    entry = dir.openNextFile();
    if (!entry) break;    
    int s = entry.size();   
    if ( (s & 0xff) > 0) s |= 0xff;
    Serial.printf("%s\t\t\t%d Bytes\r\n", entry.name(), s);
    fsize += s;   
    entry.close();  
    fcnt++;
  } 
  dir.close();
  
  Serial.printf("\t\t\t%d File(s), %d Bytes\r\n", fcnt, fsize);

  if (!fsize) goto ready;
  if (fsize < FLASHSIZE) {
      flash_init();
      int flashstatus = flash_read_status();
      flash_read_id(id_tab);
      Serial.printf("Flash Status: 0x%X, ID:0x%X,0x%X,0x%X,0x%X ", flashstatus , id_tab[0], id_tab[1], id_tab[2], id_tab[3]);   
      if (id_tab[0]!=0xef || id_tab[1]!=0x40 || id_tab[2]!=0x18 || id_tab[3]!=0x00) {Serial.println(" is not ok."); goto end;}
      Serial.printf(" is ok.\r\nFile(s) fit(s) in serial flash, %d Bytes remaining.\r\n\r\n", FLASHSIZE - fsize);

      Serial.print("Check flash content: ");
      if (verify()) { Serial.println("Flash content ok. Nothing to do.");goto end; }

      erase();
      flash();      
      verify();      

end:            
      dir.close();
  }
  else Serial.println("Does not fit.");  
  
ready:  
  Serial.println("Ready.");
  if (fsize) { 
	Serial.printf("Copy'n paste:\r\n\r\nconst int SPIFlash[%d] = {\r\n",fcnt);
	for (int i = 0; i<fcnt; i++) 
		Serial.printf("\t\t0x%07X%s //\"%s\"\r\n",position[i], ((i<fcnt-1)?", ":"};"), filename[i].c_str());
	
	
	Serial.printf("\r\nconst char SPIFlashFilename[%d][13] = {\r\n",fcnt);
	for (int i = 0; i<fcnt; i++) 
		Serial.printf("\t\t\"%s\"%s\r\n", filename[i].c_str(),((i<fcnt-1)?",":"};"));	
  }
  
}



const int SPIFlash[7] = {
                0x0000000,  //"201159~1.RAW"
                0x002C800,  //"82583_~1.RAW"
                0x0053300,  //"86334_~1.RAW"
                0x0065B00,  //"86773_~1.RAW"
                0x01F1D00,  //"102790~1.RAW"
                0x020A100,  //"171104~1.RAW"
                0x0214900}; //"P2.RAW"

const char SPIFlashFilename[7][13] = {
                "201159~1.RAW",
                "82583_~1.RAW",
                "86334_~1.RAW",
                "86773_~1.RAW",
                "102790~1.RAW",
                "171104~1.RAW",
                "P2.RAW"};



void loop()
{}

PaulStoffregen · May 7, 2015

Can you post these files, or email them to me. I'll give it a try here.

Roel · May 7, 2015

Of course, I'll send the files this weekend
Thanks in advance

Roel · May 8, 2015

View attachment trw files.zip
@Paul,

here is a collection of files of which some did, and some didn't work for me.

manitou · May 14, 2015

I don't have the same hardware setup, but I do have a 1MB winbond DIP SPI flash. Not using an SD, but just doing the same sort of serial flash file creates, I could not reproduce the problem on teensy 3.1. (i'll try with an SD shortly, but it still won't be the same hardware config).

Paul, the 1MB serial flash from adafruit http://www.adafruit.com/product/1564
seems to work with your SerialFlash library. To windbond ID check in RawHardwareTest example I added
if (id[2] == 0x14) return "W25Q80BV";
I'll do the "issue" thing on github.

Edit, even without the SD logic I can get a similar failure with this sketch

Code:

#include <SerialFlash.h>
#include <SPI.h>

const int FlashChipSelect = 6;

void dofile(char *filename, unsigned long length) {
    Serial.print(filename);
    Serial.print("    ");
    Serial.println(length);

    // check if this file is already on the Flash chip
    if (SerialFlash.exists(filename)) {
      Serial.println("  already exists on the Flash chip");
      Serial.println("  delete file from Flash chip");
      SerialFlash.remove(filename);
    }
  
    // create the file on the Flash chip and copy data
    if (SerialFlash.create(filename, length)) {
      SerialFlashFile ff = SerialFlash.open(filename);
      if (ff) {
        // copy data loop
        unsigned long count = 0;
        while (length) {
          char buf[256];
          unsigned int n;
		  if (length >= 256) n =256;
		   else n = length;
          ff.write(buf, n);
          length -= n;
          Serial.print(".");
        }
        ff.close();
        Serial.println();
      } else {
        Serial.println("  error opening freshly created file!");
      }
    } else {
      Serial.println("  unable to create file");
    }
}


void setup() {
  //uncomment these if using Teensy audio shield
//  SPI.setSCK(14);  // Audio shield has SCK on pin 14
 // SPI.setMOSI(7);  // Audio shield has MOSI on pin 7

  // wait up to 10 seconds for Arduino Serial Monitor
  unsigned long startMillis = millis();
  while (!Serial && (millis() - startMillis < 10000)) ;
  delay(100);
  Serial.println("create some files on SPI Flash");
  SerialFlash.begin();

 dofile("TST.DAT",147716);
 dofile("CR3.TRW",147716);
  dofile("HH1CLO~1.TRW",7684);
  dofile("HH1CLO~2.TRW",9092);
}

void loop() {
}

void error(const char *message) {
  while (1) {
    Serial.println(message);
    delay(2500);
  }
}

output is

Code:

create some files on SPI Flash
TST.DAT    147716
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
CR3.TRW    147716
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
HH1CLO~1.TRW    7684
...............................
HH1CLO~2.TRW    9092
  error opening freshly created file!

and the behavior of example ListFiles is

Code:

All Files on SPI Flash chip:
  TST.DAT               147716 bytes
  CR3.TRW               147716 bytes
  HH1CLO~1.TRW          7684 bytes
                        9092 bytes

if i change the name of the last file to tom.TRW then all are created ??
i'll probe a little deeper ... I suspect a buffer overflow, results change with Serial.print's inserted
... lots of magic numbers 8 10 12

manitou · May 15, 2015

I think the problem "error opening freshly created file!" arises from an error in rounding the string storage up to a multiple of 4 in SerialFlashDirectory.cpp.
The calculation should be
straddr += string_length(straddr) + 1;

PaulStoffregen · May 16, 2015

Yes, very likely a bug on my part in the library!

I'm currently at Maker Farie, without access to my desktop where I keep the list of bugs to investigate and fix. Please bump this thread on Tuesday, or post an issue on this library's github, so it won't be forgotten.

manitou · May 16, 2015

maybe another bug (feature?) in readdir(). example ListFiles will show all files, even those "removed" because readdir() does not check the hash value. (hash value of 0 means file has been removed)

might also be nice to have a file method that reports free space remaining ....

-------
(never mind following ....)
After EraseEverything example (teensy 3.1), i'm also still experiencing "unable to create files", but adding a Serial.print or a delay here and there often fixes it. Enabling the print in check_signature() also fixes it, also delay(1) before SerialFlash.write(0, sig, 8); in check_signature() seems to fix it, though delayMicroseconds(1000) does NOT fix it ???
Or adding delayMicroseconds(1); to CSRELEASE seemed to work .... maybe not ?

on teensy 3.0 i have not experienced "unable to create files"

(using 1.0.6 1.21)

Edit: using 1.6.0 and 1.21, my create files tests seem to work OK on 3.1 -- i need to catch up with the new IDE releases.

Edit 2: installed 1.6.3 and 1.23, SerialFlash tests seem to work, no problems with create on teensy 3.1

Roel · May 19, 2015

Hey Paul,

Here is the bump you asked for.

Have a nice day

PaulStoffregen · May 22, 2015

I'm trying this now, with the files from #5.

Code:

Copy all files from SD Card to SPI Flash

HH1CLE.TRW    9092
  copying....................................

HH1CL.TRW    7684
  copying...............................

HH1HO.TRW    20228
  copying................................................................................

K1.TRW    29700
  copying.....................................................................................................................

K2.TRW    8836
  copying...................................

R1.TRW    159748
  copying.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

R2.TRW    116996
  copying..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................

RIM1.TRW    15108
  copying............................................................

It seems to be working fine, at least with this set of files.

However, the file names don't match the names in the text above. Message #1 has "HH1HAL~1.TRW" failing. None of the files from #5 are longer than 8.3 format.

Is there another set of files I should try?

PaulStoffregen · May 22, 2015

manitou said:
I think the problem "error opening freshly created file!" arises from an error in rounding the string storage up to a multiple of 4 in SerialFlashDirectory.cpp.
The calculation should be
straddr += string_length(straddr) + 1;

Oh, yes, it looks like string_length() isn't including the null terminator in the length calculation.

Still, I'd really like to have a set of files that reproduces this problem before committing a fix. Anyone?!

Edit: ok, here's a set that causes the error!

PaulStoffregen · May 22, 2015

Ok, I've committed a fix.

https://github.com/PaulStoffregen/SerialFlash/commit/b53b2910a98b9576d0f28cbf8d55d2ee6758597b

Please give the latest code a try and let me know if it fully resolves the problems you're seeing?

manitou · May 23, 2015

PaulStoffregen said:
Still, I'd really like to have a set of files that reproduces this problem before committing a fix. Anyone?!

I had a sketch in this thread that demonstrated the string_length problem. Your fixes look good. string_length problem fixed, my winbond chip is ID'd in the RawHardwareTest example, and the directory list (ListFiles) no longer shows removed files. Check.

Roel · May 24, 2015

Hey Paul,

Thanks very much for your fix!
I retried to flash even more files,
and it copied them all without the slightest complaint

manitou · Jun 22, 2018

Just for the record, here is another Winbond SPI Serial flash that works with teensy SerialFlash lib. SPI flash is 2MB W25Q16BV. example RawHardwareTest reports:

Code:

Raw SerialFlash Hardware Test
Read Chip Identification:
  JEDEC ID:     1 40 15
  Part Nummber: (unknown chip)
  Memory Size:  2097152 bytes
  Block Size:   65536 bytes
Reading Chip...
Writing 1024 signatures
Double Checking All Signatures:
  all 1024 signatures read ok
Checking Signature Pairs
  all 511 signature pairs read ok
Checking Read-While-Write (Program Suspend)
  write 256 bytes at 256
  write time was 656 microseconds.
  read-while-writing: 00 00 00 00 15 F5 95 4B
  test passed, good read while writing
Checking Read-While-Erase (Erase Suspend)
  erase time was 412833 microseconds.
  erase correctly erased 65536 bytes
  read-while-erasing: 00 00 00 00 15 F5 95 4B
  test passed, good read while erasing
All Tests Passed  :-)

flash chip actually resides on and tested on Adafruit circuit playground express (ARM M0+).

Adafruit has a SPI flash library that supports FatFs and I've tried to run that Adafruit SPI flash lib on the teensy 3.2+propshield, but I get oodles of compile errors ...

Re: FatFs on prop shield SPI serial flash

UPDATE: The compile errors come from conflict definition of BYTE in fatfs integer.h (typedef) and teensy3/Print.h (#define). I changed integer.h to use a define, and I could compile and run the fatfs on the T3.2 with propshield SPI flash. The Adafruit lib is hardwired for the 2MB SPI flash, but a new symbol and specs could be added for the prop shield 8MB SPI flash. The teensy (T3.2@120mhz with 30mhz SPI) SPI and the resulting SPI flash IO is about 4 times faster than the Adafruit circuit playground Express. The fatfs does read/erase/update/write on the smallest erasable unit (4KB) to update files on the SPI flash.

Of course there are a lot of arguments of why you wouldn't the overhead of FatFs on an embedded SPI serial flash ...

Can't get files on SPI flash

Roel

Active member

Roel

Active member

PaulStoffregen

Well-known member

Roel

Active member

Roel

Active member

manitou

Senior Member+

manitou

Senior Member+

PaulStoffregen

Well-known member

manitou

Senior Member+

Roel

Active member

PaulStoffregen

Well-known member

PaulStoffregen

Well-known member

Attachments

PaulStoffregen

Well-known member

manitou

Senior Member+

Roel

Active member

manitou

Senior Member+