LittleFS port to Teensy/SPIFlash

I am guessing for this round the later one is the better choice to wire up another T4.1...

Yes, the smaller & simpler NOR flash is the only type that's likely to work for a while...

Unless you really *want* to dive into writing NAND flash support code, probably best to leave those in the Digikey bag for a while. Those chips require quite a lot of extra work to check the error correcting code status and manage remapping bad sectors. Winbond has 2 more chips coming soon with 256M and 512M capacity, which are similar in concept but use different ECC and have slightly different commands. I did get the datasheets without signing a NDA (they're not on Winbond's website yet), so email me if you really want the details. My current feeling is to wait until early next year when I can a sample of one of those new chips, rather than writing a lot of code now and having to test adapt it to the new format only months later.

The NAND chips have larger 2K page size and 128K sector erase size. So even though the chip is much larger, it probably won't be any more useful if you store only small files, because LittleFS allocates sectors to files. Even a file with a few bytes will take up at least 128K of space on those NAND chips. Of course they will be pretty awesome for storing a small number of relatively large files like big images and sounds.
 
Yes, the smaller & simpler NOR flash is the only type that's likely to work for a while...

Unless you really *want* to dive into writing NAND flash support code, probably best to leave those in the Digikey bag for a while. Those chips require quite a lot of extra work to check the error correcting code status and manage remapping bad sectors. Winbond has 2 more chips coming soon with 256M and 512M capacity, which are similar in concept but use different ECC and have slightly different commands. I did get the datasheets without signing a NDA (they're not on Winbond's website yet), so email me if you really want the details. My current feeling is to wait until early next year when I can a sample of one of those new chips, rather than writing a lot of code now and having to test adapt it to the new format only months later.

The NAND chips have larger 2K page size and 128K sector erase size. So even though the chip is much larger, it probably won't be any more useful if you store only small files, because LittleFS allocates sectors to files. Even a file with a few bytes will take up at least 128K of space on those NAND chips. Of course they will be pretty awesome for storing a small number of relatively large files like big images and sounds.

Well, we did manage to get that NAND chip working with the T4.1, don't really do anything with ECC (yeah I know - way over my head). But you can read and write data to the chip but it is slower. The thread we discussed at is here: https://forum.pjrc.com/threads/61566-NAND-flash-support-in-1-54?highlight=nand. Paul - thanks for teaching about LUTS but still was pain in the you know what. The lib that we put together is on Github: https://forum.pjrc.com/threads/61566-NAND-flash-support-in-1-54?highlight=nand.

Found an interesting discussion on LittleFS project repository about using the W25N01GV chip with LittleFS: using LittleFs on W25N01GV 1Gb NAND Flash with FreeRTOS. From what I read I agree with Paul not sure its worth it.
The NAND chips have larger 2K page size and 128K sector erase size. So even though the chip is much larger, it probably won't be any more useful if you store only small files, because LittleFS allocates sectors to files. Even a file with a few bytes will take up at least 128K of space on those NAND chips. Of course they will be pretty awesome for storing a small number of relatively large files like big images and sounds.

Looking more forward to playing with the new Winbond chips when they come out.

Ok - now I am rambling.
 
LittleFS looks good for next Beta as far as I can see.

@Paul - thanks for the timing detail - forgot about erase time as it reuses\cycles across the media. Is there a way to do a full media format on command? Would be nice to start a test run with full disk 'ready for writing'

Larger LittleFS seems nice - though 16MB onboard not a bad start until bigger/better NAND makes it worthwhile to go there.

> Paul's DIR test sketch not seeing any shifting - would take changes to trigger them now. I could add Count updates in LFSIntegrity for File create and File remove detection - then do a no print count DirWalk ...

Started a hundred batch of iterations - funny it ended on all but 1 of 286 possible files deleted. Which verifies content with read before delete, while running files were 10K or larger with 13 alternating upper lower case copies of the filename letter and no errors found.
Did a lot of Error Free I/O : Bytes read 297,809,460, written 139,129,700
Code:
printDirectory QSPI_DISK
--------------
DIR	0_dir / 
	FILE	A_file.txt		20200
 0 dirs with 1 files of Size 20200 Bytes
DIR	1_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	2_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	3_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	4_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	5_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	6_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	7_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	8_dir / 
 0 dirs with 0 files of Size 0 Bytes
DIR	9_dir / 
 0 dirs with 0 files of Size 0 Bytes
 10 dirs with 0 files of Size 0 Bytes
 Total 1 files of Size 20200 Bytes
Bytes Used: 118784, Bytes Total:16777216
[118.79 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >

[118.79 M] Awaiting input 0123456789rdchkfvplm? loops left 0 >l 
	 Loop Count: 1169 (#fileCycle=25661), Bytes read 297809460, written 139129700
 
Thanks Paul,

I soldered on one of the smaller ones as well as my last PSRAM and it appears to be working :D
 
Is there a way to do a full media format on command?

By full media format, I'm guessing you mean erasing every block / sector in the whole chip, not just myfs.format(), right?

There isn't any public API nor any code in LittleFS.cpp for such an erase command. But of course the hardware does have full chip erase capability. For QSPI, you'd need to craft the code to program the FlexSPI LUTs and send the enable and erase command, and wait for it to complete. Does writing & testing that code could as "a way"?
 
By full media format, I'm guessing you mean erasing every block / sector in the whole chip, not just myfs.format(), right?

There isn't any public API nor any code in LittleFS.cpp for such an erase command. But of course the hardware does have full chip erase capability. For QSPI, you'd need to craft the code to program the FlexSPI LUTs and send the enable and erase command, and wait for it to complete. Does writing & testing that code could as "a way"?

Yes, that is what I meant. But asking if it was an existing unknown feature of LittleFS, which it is not.
FrankB wrote one that I included earlier in the test I wrote. Seemed to require restart after format as the LUT'ttery was tattered/altered as it was added.

BTW: Updated LFSIntegrity to do a silent DirWalk file count on each pass - it is passing on QSPI! :: unsigned int printDirectoryFilecount(File dir)
Slows things down - but I removed the LittleFS.cpp::"waited ..." prints so it looks much prettier now.

Stopped it now and ALL 286 files are present this time:
Code:
 10 dirs with 26 files of Size 425550 Bytes
 [B]Total 286 files[/B] of Size 4640750 Bytes
Bytes Used: 5144576, Bytes Total:16777216

	 Loop Count: 32 (#fileCycle=2964), Bytes read 68451920, written 21594700, [B]#Files=286[/B]

Full LFSintegrity.ino Code 438 lines:
Code:
#include <LittleFS.h>

// #define ROOTONLY // NORMAL is NOT DEFINED!

//#define TEST_RAM
//#define TEST_SPI
//#define TEST_QSPI
#define TEST_PROG

// Set for SPI usage
const int FlashChipSelect = 6; // digital pin for flash chip CS pin

#ifdef TEST_RAM
LittleFS_RAM myfs;
DMAMEM char buf[490000];	// USE DMAMEM for more memory than ITCM allows - or remove
char szDiskMem[] = "RAM_DISK";
#elif defined(TEST_SPI)
//const int FlashChipSelect = 21; // Arduino 101 built-in SPI Flash
#define FORMATSPI
//#define FORMATSPI2
LittleFS_SPIFlash myfs;
char szDiskMem[] = "SPI_DISK";
#elif defined(TEST_PROG)
LittleFS_Program myfs;
char szDiskMem[] = "PRO_DISK";
#else // TEST_QSPI
LittleFS_QSPIFlash myfs;
char szDiskMem[] = "QSPI_DISK";
#endif

File file3;

#define SUBADD 1	// bytes added each pass (*times file number)
#define BIGADD 10	// bytes added each pass - bigger will quickly consume more space
#define MAXNUM 26	// ALPHA A-Z is 26, less for fewer files
#define DELDELAY 0 	// delay before DEL files : delayMicroseconds
#define ADDDELAY 0 	// delay on ADD FILE : delayMicroseconds

const uint32_t lowOffset = 'a' - 'A';
const uint32_t lowShift = 13;
uint32_t lCnt = 0;
uint32_t LoopCnt = 0;
uint32_t rdCnt = 0;
uint32_t wrCnt = 0;
unsigned int filecount = 0;

void setup() {
	while (!Serial) ; // wait
	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);
	Serial.println("LittleFS Test : File Integrity"); delay(5);

#ifdef TEST_RAM
	if (!myfs.begin(buf, sizeof(buf))) {
#elif defined(TEST_SPI)
#ifdef FORMATSPI
	if (!myfs.begin( FlashChipSelect )) {
#elif defined(FORMATSPI2)
	pinMode(FlashChipSelect, OUTPUT);
	digitalWriteFast(FlashChipSelect, LOW);
	SPI2.setMOSI(50);
	SPI2.setMISO(54);
	SPI2.setSCK(49);
	SPI2.begin();
	if (!myfs.begin(51, SPI2)) {
#endif
#elif defined(TEST_PROG)
	if (!myfs.begin(1024*1024*4)) {
#else
	if (!myfs.begin()) {
#endif
		Serial.printf("Error starting %s\n", szDiskMem);
		checkInput( 1 );
	}
	// parseCmd( 'f' ); // ENABLE this is disk won't allow startup
	printDirectory();
	parseCmd( '?' );
#ifndef ROOTONLY // Don't make subdirs - 11/9/20 : QSPI only makes first two usable, creates third that 'goes away'
	makeRootDirs();
#endif
	checkInput( 1 );
	filecount = printDirectoryFilecount( myfs.open("/") );
	printDirectory();
}

void makeRootDirs() {
	char szDir[16];
	for ( int ii = 0; ii < 10; ii++ ) {
		sprintf( szDir, "/%c_dir", '0' + ii );
		myfs.mkdir( szDir );
	}
}

int loopLimit = 0; // -1 continuous, otherwise # to count down to 0
bool pauseDir = false;  // Start Pause on each off
bool showDir =  false;  // false Start Dir on each off
void loop() {
	char szDir[16];
	LoopCnt++;
	uint32_t chStep;
	if ( loopLimit != 0 ) {
#ifdef ROOTONLY // ii=0-9 are subdirs. #10 is Root
		for ( int ii = 10; ii < 11; ii++ )
#else
		for ( int ii = 0; ii < 11; ii++ )
#endif
		{
			if ( ii == 10 )
				sprintf( szDir, "/" );
			else
				sprintf( szDir, "/%c_dir", '0' + ii );
			chStep = fileCycle(szDir);
			while ( chStep != fileCycle(szDir) && ( loopLimit != 0 ) ) checkInput( 0 ); // user input can 0 loopLimit
		}
		checkInput( 0 );
		if ( loopLimit > 0 ) // -1 means continuous
			loopLimit--;
	}
	else
		checkInput( 1 );
}

char szInputs[] = "0123456789rdchkfvplmu?";
void checkInput( int step ) { // prompt for input without user input with step != 0
	char retVal = 0, temp;
	char *pTemp;
	if ( step != 0 ) {
		Serial.printf( "[%6.2f M] Awaiting input %s loops left %d >", millis() / 60000.0, szInputs, loopLimit );
	}
	else {
		if ( !Serial.available() ) return;
		Serial.printf( "[%6.2f] Awaiting input %s loops left %d >", millis() / 60000.0, szInputs, loopLimit );
		while ( Serial.available() ) {
			temp = Serial.read( );
			if ( (pTemp = strchr(szInputs, temp)) ) {
				retVal = pTemp[0];
				parseCmd( retVal );
			}
		}
	}
	while ( !Serial.available() );
	while ( Serial.available() ) {
		temp = Serial.read();
		if ( (pTemp = strchr(szInputs, temp)) ) {
			retVal = pTemp[0];
			parseCmd( retVal );
		}
	}
	Serial.print( '\n' );
	if ( '?' == retVal ) checkInput( 1 ); // recurse on '?' to allow command show and response
	return;
}
void parseCmd( char chIn ) { // pass chIn == '?' for help
	switch (chIn ) {
	case '?':
		Serial.printf( "%s\n", " 0, 1-9 '#' passes continue loop before Pause\n\
 'r' Restart Teensy\n\
 'd' Directory of LittleFS\n\
 'c' Continuous Loop\n\
 'h' Hundred loops\n\
 'k' Thousand loops\n\
 'f' Format: RAM Restart : Q/SPI->'myfs.format();'\n\
 'v' Verbose All Dir Prints - TOGGLE\n\
 'p' Pause after all Dir prints - TOGGLE\n\
 'l' Show count of loop()'s, Bytes Read,Written\n\
 'm' Make ROOT dirs (needed after format !ROOTONLY)\n\
 'u' Update Filecount\n\
 '?' Help list" );
		break;
	case 'r':
		Serial.print(" RESET Teensy ...");
		delay(100);
		SCB_AIRCR = 0x05FA0004;
		break;
	case '0':
	case '1':
	case '2':
	case '3':
	case '4':
	case '5':
	case '6':
	case '7':
	case '8':
	case '9':
		loopLimit = chIn - '0';
		break;
	case 'c':
		loopLimit = -1;
		break;
	case 'd':
		Serial.print( " d\n" );
		printDirectory();
		Serial.print( '\n' );
		parseCmd( 'l' );
		checkInput( 1 );
		chIn = 0;
		break;
	case 'h':
		loopLimit = 100;
		break;
	case 'k':
		loopLimit = 1000;
		break;
	case 'f': // format
#ifdef TEST_RAM
		parseCmd( 'r' );
#elif defined(TEST_SPI)
		myfs.format();
#elif defined(TEST_PROG)
		//myfs.format();
#else // TEST_QSPI
		myfs.format();
#endif
		parseCmd( 'u' );
		break;
	case 'v': // verbose dir
		showDir = !showDir;
		showDir ? Serial.print(" Verbose on: ") : Serial.print(" Verbose off: ");
		chIn = 0;
		break;
	case 'p': // pause on dirs
		pauseDir = !pauseDir;
		pauseDir ? Serial.print(" Pause on: ") : Serial.print(" Pause off: ");
		chIn = 0;
		break;
	case 'l': // Show Loop Count
		Serial.printf("\n\t Loop Count: %u (#fileCycle=%u), Bytes read %u, written %u, #Files=%u\n", LoopCnt, lCnt, rdCnt, wrCnt, filecount );
		chIn = 0;
		break;
	case 'm':
		Serial.printf("m \n\t Making Root Dirs\n" );
		makeRootDirs();
		parseCmd( 'd' );
		chIn = 0;
		break;
	case 'u': // Show Loop Count
		filecount = printDirectoryFilecount( myfs.open("/") );
		Serial.printf("u \n\t Updated filecount %u\n", filecount );
		chIn = 0;
		break;

	default:
		Serial.println( chIn ); // never see without unhandled char in szInputs[]
		break;
	}
	if ( 0 != chIn ) Serial.print( chIn );
}

uint32_t fTot, totSize;
void printDirectory() {
	fTot = 0, totSize = 0;
	Serial.printf("printDirectory %s\n--------------\n", szDiskMem);
	printDirectory(myfs.open("/"), 0);
	//Serial.println();
	Serial.printf(" %Total %u files of Size %u Bytes\n", fTot, totSize);
	Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());
}

unsigned int printDirectoryFilecount(File dir) {
	unsigned int filecnt = 0;
	while (true) {
		File entry =  dir.openNextFile();
		if (! entry) {
			// no more files
			break;
		}
		if (entry.isDirectory()) {
			filecnt+=printDirectoryFilecount(entry);
		} else {
			filecnt++;
		}
		entry.close();
	}
	return filecnt;
}

void printDirectory(File dir, int numTabs) {
	//dir.whoami();
	uint32_t fSize = 0, dCnt = 0, fCnt = 0;
	if ( 0 == dir ) {
		Serial.printf( "\t>>>\t>>>>> No Dir\n" );
		return;
	}
	while (true) {
		File entry =  dir.openNextFile();
		if (! entry) {
			// no more files
			Serial.printf("\n %u dirs with %u files of Size %u Bytes\n", dCnt, fCnt, fSize);
			fTot += fCnt;
			totSize += fSize;
			break;
		}
		for (uint8_t i = 0; i < numTabs; i++) {
			Serial.print('\t');
		}

		if (entry.isDirectory()) {
			Serial.print("DIR\t");
			dCnt++;
		} else {
			Serial.print("FILE\t");
			fCnt++;
			fSize += entry.size();
		}
		Serial.print(entry.name());
		if (entry.isDirectory()) {
			Serial.println(" / ");
			printDirectory(entry, numTabs + 1);
		} else {
			// files have sizes, directories do not
			Serial.print("\t\t");
			Serial.println(entry.size(), DEC);
		}
		entry.close();
		//Serial.flush();
	}
}

uint32_t cCnt = 0;
uint32_t fileCycle(const char *dir) {
	static char szFile[] = "_file.txt";
	char szPath[150];
	int ii;
	lCnt++;
	byte nNum = lCnt % MAXNUM;
	char chNow = 'A' + lCnt % MAXNUM;
	lfs_ssize_t resW = 1;

	if ( dir[1] == 0 )	// catch root
		sprintf( szPath, "/%c%s", chNow, szFile );
	else
		sprintf( szPath, "%s/%c%s", dir, chNow, szFile );
	if ( cCnt >= 3 && myfs.exists(szPath) ) { // DELETE ALL KNOWN FILES
		if ( nNum == 1 ) {
			Serial.print( "\n == == ==   DELETE PASS START  == == == = \n");
			if ( showDir ) {
				printDirectory();
				Serial.print( " == == ==   DELETE PASS START  == == == = \n");
			}
			delayMicroseconds(DELDELAY);
		}
	}
	Serial.printf( ":: %s ", szPath );
	if ( cCnt >= 3 && myfs.exists(szPath) ) { // DELETE ALL KNOWN FILES
		readVerify( szPath, chNow );
		myfs.remove(szPath);
		filecount--;
		Serial.printf(" %s ----DEL----", szDiskMem);
		Serial.printf(" -- %c", chNow);
		if ( showDir ) {
			Serial.print("\n");
			printDirectory(myfs.open(dir), 1);
		}
		if ( pauseDir ) checkInput( 1 );
		Serial.println();
	}
	else {
		if ( nNum == 0 ) {
			nNum = 10;
			cCnt++;
			if ( cCnt >= 5 ) cCnt = 0;
		}
		file3 = myfs.open(szPath, FILE_WRITE);
		if ( 0 == file3 ) {
			Serial.printf( "\tXXX\tXXX\tXXX\tXXX\tFail File open \n" );
			checkInput( 1 );	// PAUSE on CmdLine
		}
		delayMicroseconds(ADDDELAY);
		char mm = chNow + lowOffset;
		uint32_t jj = file3.size() + 1;
		if ( jj == 1) filecount++;
		for ( ii = 0; ii < (nNum * SUBADD + BIGADD ) && resW > 0; ii++ ) {
			if ( 0 == ((ii + jj) / lowShift) % 2 )
				resW = file3.write( &mm , 1 );
			else
				resW = file3.write( &chNow , 1 );
			wrCnt++;
			// if ( lCnt%100 == 50 ) mm='x'; // GENERATE ERROR to detect on DELETE read verify
		}
		file3.close();
		Serial.printf(" %s +++ Add +++ [sz %u add %u]", szDiskMem, jj - 1, ii);
		if (resW < 0) {
			Serial.printf( "\twrite fail %i", resW );
			checkInput( 1 );	// PAUSE on CmdLine
		}
		Serial.printf(" ++ %c ", chNow);
		readVerify( szPath, chNow );
		if ( showDir ) {
			Serial.print("\n");
			printDirectory(myfs.open(dir), 1);
		}
		if ( pauseDir ) checkInput( 1 );
		Serial.print("\n");
		delayMicroseconds(ADDDELAY);
	}
	if ( filecount != printDirectoryFilecount( myfs.open("/") ) ) {
		Serial.printf( "\tFilecount mismatch %u != %u\n", filecount, printDirectoryFilecount( myfs.open("/") ) );
		checkInput( 1 );	// PAUSE on CmdLine
	}
	return cCnt;
}

void readVerify( char szPath[], char chNow ) {
	file3 = myfs.open(szPath);
	if ( 0 == file3 ) {
		Serial.printf( "\tV\t Fail File open %s\n", szPath );
		checkInput( 1 );
	}
	char mm;
	char chNow2 = chNow + lowOffset;
	uint32_t ii = 0;
	while ( file3.available() ) {
		file3.read( &mm , 1 );
		rdCnt++;
		//Serial.print( mm ); // show chars as read
		ii++;
		if ( 0 == (ii / lowShift) % 2 ) {
			if ( chNow2 != mm ) {
				Serial.printf( "<Bad Byte!  %c! = %c [0x%X] @%u\n", chNow2, mm, mm, ii );
				checkInput( 1 );
			}
		}
		else {
			if ( chNow != mm ) {
				Serial.printf( "<Bad Byte!  %c! = %c [0x%X] @%u\n", chNow, mm, mm, ii );
				checkInput( 1 );
			}
		}
	}
	Serial.printf( "\tVerify %s bytes %u ", szPath, ii );
	if (ii != file3.size()) {
		Serial.printf( "\n\tRead Count fail! :: read %u != f.size %u",ii, file3.size() );
		checkInput( 1 );	// PAUSE on CmdLine
	}
	file3.close();
}
 
Last edited:
I've added LittleFS_Program, to create a filesystem in program memory on Teensy 4.x.

https://github.com/PaulStoffregen/LittleFS/commit/4622ed264348d4f49d4ff1177e50640abe8898f9

To use it, you also need this change in the core library.

https://github.com/PaulStoffregen/cores/commit/8ad7ce6b5453b10d67a82f11717d8772cdd1b687

The begin function expects a size in bytes, for the amount of the program memory you wish to use. It must be at least 65536 and smaller than the actual unused program space.

All interrupts are disabled during writing and erasing, so use of this storage does come with pretty substantial impact on interrupt latency for other libraries.

Uploading new code by Teensy Loader completely wipes the unused program space, erasing all stored files. But the filesystem persists across reboots and power cycling.
 
Downloaded the PCB Flash Abuse code - not updated yet. Will give it a try.

The QSPI on Two T_4.1's saw no errors in File Count with large number of Files add/extend, delete - let them run some time the first with larger files of 10K's, the second smaller 100's bytes.

Code:
Loop Count: 538 (#fileCycle=[B]45640[/B]), Bytes read 1050341820, written 328962690, #Files=156

Loop Count: 141 (#fileCycle=[B]20172[/B]), Bytes read 5247726, written 1625990, #Files=264
 
Error not found???:
Code:
LFSintegrity.ino:217: undefined reference to `LittleFS_Program::format()'

All seems well running on two T_4.1's with 4MB alloc - NO ERRORS.

<note>: that was first upload and it worked - now uploaded minor edits - Upload time is hideous with full Flash clearing? Is it working ? ? ? ?

Ran some part of 20 minutes on one:
Code:
 10 dirs with 26 files of Size 4277 Bytes
 Total 286 files of Size 49350 Bytes
Bytes Used: 794624, Bytes Total:4194304

	 Loop Count: 56 (#fileCycle=6764), Bytes read 1587833, written 546320, #Files=286

And the other:
Code:
 10 dirs with 24 files of Size 4321 Bytes
 Total 264 files of Size 42377 Bytes
Bytes Used: 638976, Bytes Total:4194304

	 Loop Count: 57 (#fileCycle=7290), Bytes read 1723998, written 586275, #Files=264

Shortened some SPEW lines and Added the LittleFS_Program to LFSintegrity - updated code posted above in p#280
 
Not using IDE but CMDLine IDE build from Editor with TyCommander integrated.

Seemed like the LFS_Program code uploaded and stayed in bootloader?

Got HEX name and opened TeensyLoader and pushed Upload and Reset and it was fine.

Closed TeensyLoader and opened TyComm and gave it the HEX and it worked okay?

De-Integrated TyCommander and Sublime IDE build then uses TeensyLoader and that worked fine to both units.

Did Compile/Upload from Sublime with TyComm - Program RED LED burns bright a short time then goes DIM RED - in bootloader. TyComm Reset does not bring it online?
Is there any new work for the uploader like TyCommander to do?
Repower Teensy and it does not appear?
TyComm reports this from Bootloader to failed upload::
Code:
Received removal notification for device 'HID\VID_16C0&PID_0478\7&3671B3D1&0&0000'
Received removal notification for device 'USB\VID_16C0&PID_0478\000BAFBE'
Remove device 'USB\VID_16C0&PID_0478\000BAFBE'
Received arrival notification for device 'USB\VID_16C0&PID_0478\000BAFBE'
Examining device node 'USB\VID_16C0&PID_0478\000BAFBE'
Device 'USB\VID_16C0&PID_0478\000BAFBE' has no 'PortName' registry property
Received arrival notification for device 'HID\VID_16C0&PID_0478\7&3671B3D1&0&0000'
Examining device node 'HID\VID_16C0&PID_0478\7&3671B3D1&0&0000'
Found port number of 'USB\VID_16C0&PID_0478\000BAFBE': 2
Found port number of 'USB\VID_045B&PID_0209\6&2DA2EF95&0&1': 1
Found port number of 'USB\VID_045B&PID_0209\5&38BA1A52&0&6': 6
Found controller ID for 'USB\ROOT_HUB30\4&362721B3&0&0': 1
Add HID device 'USB\VID_16C0&PID_0478\000BAFBE' on iface 0
  - USB VID/PID = 16c0:0478, USB location = usb-1-6-1-2
  - USB manufacturer = (none), product = (none), S/N = 000BAFBE
  - HID usage page = 0xff9c, HID usage = 0x25
Identified 'Teensy 4.1' with usage value 0x25
[upload@7658860-Teensy] Uploading to board '7658860-Teensy' (Teensy 4.1)
[upload@7658860-Teensy] Firmware: LFSintegrity.ino.TEENSY41.hex
[upload@7658860-Teensy] Flash usage: 72 kiB (0.9%)
[upload@7658860-Teensy] Sending reset command
Received removal notification for device 'HID\VID_16C0&PID_0478\7&3671B3D1&0&0000'
Received removal notification for device 'USB\VID_16C0&PID_0478\000BAFBE'
Remove device 'USB\VID_16C0&PID_0478\000BAFBE'
Received arrival notification for device 'USB\VID_16C0&PID_048B\7658860'
Examining device node 'USB\VID_16C0&PID_048B\7658860'
Device 'USB\VID_16C0&PID_048B\7658860' has no 'PortName' registry property
Received arrival notification for device 'USB\VID_16C0&PID_048B&MI_00\7&3A060C4A&0&0000'
Examining device node 'USB\VID_16C0&PID_048B&MI_00\7&3A060C4A&0&0000'
Found port number of 'USB\VID_16C0&PID_048B\7658860': 2
Found port number of 'USB\VID_045B&PID_0209\6&2DA2EF95&0&1': 1
Found port number of 'USB\VID_045B&PID_0209\5&38BA1A52&0&6': 6
Found controller ID for 'USB\ROOT_HUB30\4&362721B3&0&0': 1
Add serial device 'USB\VID_16C0&PID_048B\7658860' on iface 0
  - USB VID/PID = 16c0:048b, USB location = usb-1-6-1-2
  - USB manufacturer = Teensyduino, product = Dual Serial, S/N = 7658860
Identified 'Teensy 4.1' with bcdDevice value 0x280
Received arrival notification for device 'USB\VID_16C0&PID_048B&MI_02\7&3A060C4A&0&0002'
Examining device node 'USB\VID_16C0&PID_048B&MI_02\7&3A060C4A&0&0002'
Found port number of 'USB\VID_16C0&PID_048B\7658860': 2
Found port number of 'USB\VID_045B&PID_0209\6&2DA2EF95&0&1': 1
Found port number of 'USB\VID_045B&PID_0209\5&38BA1A52&0&6': 6
Found controller ID for 'USB\ROOT_HUB30\4&362721B3&0&0': 1
Add serial device 'USB\VID_16C0&PID_048B\7658860' on iface 2
  - USB VID/PID = 16c0:048b, USB location = usb-1-6-1-2
  - USB manufacturer = Teensyduino, product = Dual Serial, S/N = 7658860
Identified 'Teensy 4.1' with bcdDevice value 0x280

Asked for 5 iterations on both and they came to the same point - no errors including the DIR filecount and data checks:
Code:
 10 dirs with 10 files of Size 1135 Bytes
 Total 174 files of Size 21011 Bytes
Bytes Used: 172032, Bytes Total:4194304

	 Loop Count: 11 (#fileCycle=734), Bytes read 116977, written 59321, #Files=174

9 more iterations - again both good to the same point:
Code:
 10 dirs with 26 files of Size 3178 Bytes
 Total 286 files of Size 48458 Bytes
Bytes Used: 770048, Bytes Total:4194304

	 Loop Count: 21 (#fileCycle=2024), Bytes read 364886, written 163878, #Files=286
 
My understanding is that LFS stays in Flash (similar to EEPROM) and is not erased by bootloader.
If you put LFS in RAM, it will not be persistent

No, the space reserved on PROGMEM FLASH is WIPED on any upload

Paul p#281:
Code:
Uploading new code by Teensy Loader completely wipes the unused program space, erasing all stored files. But the filesystem persists across reboots and power cycling.

That programming and flash erase changed timing or something in the way TyCommander performs its upload/restart.
 
No, the space reserved on PROGMEM FLASH is WIPED on any upload

Paul p#281:
Code:
Uploading new code by Teensy Loader completely wipes the unused program space, erasing all stored files. But the filesystem persists across reboots and power cycling.

OK, my conclusion was wrong, but still cannot see that LFS "stayed in bootloader".
 
Did TEST_RAM with current sketch ( opps need to post tested code in p#280) and that worked the same for both T_4.1's with smaller files and DIR file validation. I should make that a toggle option to only do when a DIR is done as it slows it down.

OK, my conclusion was wrong, but still cannot see that LFS "stayed in bootloader".

Would/will be interesting if others see the same. It seemed like it worked once - so maybe it is on a timing edge koromix can adjust - but I don't think he has a T_4.1

It did here - tried a couple times and TyComm failed leaving the T_4.1 in bootloader red mode. Then went back to using T_Loader and all was well, luckily koromix added a disconnect command (-delegate) allowing it to drop SerMon connect and not interfere as T_Loader does that. That allows TyComm as SerMon for the TSET CMDLine build from editor without punching buttons or taking its Serial link offline manually to upload when not 'integrated'.

Doesn't seem like that Flash for Disk is a perfect option for lots of writes - though a shipped product could get 'config/authorization info' written there that would persist until recoded, and even obfuscated somewhat (with chip info MAC/Ser# to work on that MCU only) in case the flash were pulled off and read.
 
I brought format() into the LittleFS base class, so now you can call format() for any media type. Those #ifdef checks for which media supports format() should no longer be needed.
 
I brought format() into the LittleFS base class, so now you can call format() for any media type. Those #ifdef checks for which media supports format() should no longer be needed.

Very cool. Going to work on full chip reinit that @defrag mentioned in an earlier post. But first back to LitteleFS_Program.

Downloaded and updated LittleFS and the eeprom.c file in the core. I modified my test sketch that creates 12 files and 3 sub-directories and added in your printDirectory test sketch. Looks like its working but its overwhelming the serial monitor I think. It runs for a while the spew stops, if I close and reopen the SerMon it starts off spewing the directory listing again. Here is a run I mad with 2 close open on the SerMon:
Code:
run 3635, errors 0
printDirectory
--------------
PRINTOUTPUT1.txt		628
PRINTOUTPUT2.txt		630
file1.txt		32
file10.txt		16
file2.txt		32
file20.txt		16
file3.txt		32
file30.txt		16
structuredData / 
	logger.txt		480
test1 / 
	file1.txt		16
test2 / 
	file2.txt		16
test3 / 
	file3.txt		16

run

--------------------------  Close/Open SerMon
run 37534, errors 0
printDirectory
--------------
PRINTOUTPUT1.txt		62
---------------------

------------------------------  Close/Open SerMon
run 49248, errors 0
printDirectory
--------------
PRINTOUTPUT1.txt		628
PRINTOUTPUT2.txt		630
file1.txt		32
file10.txt		16
file2.txt		32
file20.txt		16
file3.txt		32
file30.txt		16
structuredData / 
	logger.txt		48
Now loading up a new sketch that just does a printDirectory in the setup:
Code:
LittleFS Test
started
printDirectory
--------------

Disk Usuage:
Bytes Used: 8192, Bytes Total:3997696
So looks like your statement that the File System is preserved works and no files are present.
 
Yeah, the program memory filesystem (probably) gives extremely fast read speed.

So far I've not done tried benchmarks. Maybe Mantiou will join in and bring his benchmarking magic?
 
@Paul - @defragster
Added this function to initialize QSPIFlash and seems to work - extracted from SPIFFs lib which is all from FrankB:
Code:
void LittleFS_QSPIFlash::initialize() {
  if(lfs_unmount(&lfs) < 0) {
	  Serial.println("Can not unmount device");
  } else {
	  mounted = false;
	  flexspi2_ip_command(10, 0x00800000);

	  Serial.println("Erasing... (may take some time)");
	  uint32_t t = millis();
	  FLEXSPI2_LUT60 = LUT0(CMD_SDR, PINS1, 0x60); //Chip erase
	  flexspi2_ip_command(15, 0x00800000);

	  while (wait(500000)) { //waitflash 500 milliseconds
		Serial.print(".");
	  }

	  t = millis() - t;
	  Serial.printf("\nChip erased in %d seconds.\n", t / 1000);
		Serial.println("attempting to mount existing media");
		if (lfs_mount(&lfs, &config) < 0) {
			Serial.println("couldn't mount media, attemping to format");
			if (lfs_format(&lfs, &config) < 0) {
				Serial.println("format failed :(");
				return false;
			}
			Serial.println("attempting to mount freshly formatted media");
			if (lfs_mount(&lfs, &config) < 0) {
				Serial.println("mount after format failed :(");
				return false;
			}
		}
		mounted = true;
		Serial.println("success");  
  }
}
Not pretty but if yo want to test :) just remember you have to mod LittleFS.h as well:
Code:
#if defined(__IMXRT1062__)
class LittleFS_QSPIFlash : public LittleFS
{
public:
	LittleFS_QSPIFlash() { }
	bool begin();
	void initialize();
private:
 
Yeah, the program memory filesystem (probably) gives extremely fast read speed.

So far I've not done tried benchmarks. Maybe Mantiou will join in and bring his benchmarking magic?

Well, as you said @manitou is better at this than me but I gave it a initial shot. I modified the SDFat Bench sketch to run with LittleFS but there were a couple of things:
1. there is no file.rewind function
2. there is no file.truncate function
3. Tried to use file.seek(0) for both but would get write failed errors on the second test. Also had to use FILE_WRITE when opening the file. SDFat used file.open("bench.dat", O_RDWR | O_CREAT | O_TRUNC).
4. wound up resorting to removing the benchmark file at start of test. Then it worked for QSP but not for program

For QSPI the sketch gave me:
Code:
started
FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
116.75,36867,809,4385
115.91,38200,808,4417

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
21737.74,129,14,23
21643.64,129,14,23

Done
and for Program
Code:
LittleFS Test
started
FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
518.26,3430,783,987
write failed
write failed
write failed
write failed
before it died on the second write test but I am using a 6MB buffer and a 5MB file. So that may be a problem

If you want to play this is what i have to start:
Code:
#include "Streaming.h"
#define cout Serial

#include <LittleFS.h>

//LittleFS_QSPIFlash myfs;
LittleFS_Program myfs;

File file, file1;


// Set SKIP_FIRST_LATENCY true if the first read/write to the SD can
// be avoid by writing a file header or reading the first record.
const bool SKIP_FIRST_LATENCY = true;

// Size of read/write.
const size_t BUF_SIZE = 512;

// File size in MB where MB = 1,000,000 bytes.
const uint32_t FILE_SIZE_MB = 5;

// Write pass count.
const uint8_t WRITE_COUNT = 2;

// Read pass count.
const uint8_t READ_COUNT = 2;
//==============================================================================
// End of configuration constants.
//------------------------------------------------------------------------------
// File size in bytes.
const uint32_t FILE_SIZE = 1000000UL*FILE_SIZE_MB;

// Insure 4-byte alignment.
uint32_t buf32[(BUF_SIZE + 3)/4];
uint8_t* buf = (uint8_t*)buf32;

void setup() {
  //pinMode(13, OUTPUT);
  pinMode(10, INPUT_PULLUP);
  //digitalWrite(13, HIGH);
  while (!Serial) ; // wait
  Serial.println("LittleFS Test"); delay(5);
  if (!myfs.begin(4000000)) {
  //if(!myfs.begin()){
    Serial.println("Serial.println starting spidisk");
    while (1) ;
  }
  Serial.println("started");

  float s;
  uint32_t t;
  uint32_t maxLatency;
  uint32_t minLatency;
  uint32_t totalLatency;
  bool skipLatency;
  myfs.remove("bench.dat");
  //for(uint8_t cnt=0; cnt < 10; cnt++) {
    // open or create file - truncate existing file.
    file = myfs.open("bench.dat", FILE_WRITE);

    // fill buf with known data
    if (BUF_SIZE > 1) {
    for (size_t i = 0; i < (BUF_SIZE - 2); i++) {
      buf[i] = 'A' + (i % 26);
    }
    buf[BUF_SIZE-2] = '\r';
    }
    buf[BUF_SIZE-1] = '\n';

    cout << F("FILE_SIZE_MB = ") << FILE_SIZE_MB << endl;
    cout << F("BUF_SIZE = ") << BUF_SIZE << F(" bytes\n");
    cout << F("Starting write test, please wait.") << endl << endl;

    // do write test
    uint32_t n = FILE_SIZE/BUF_SIZE;
    cout <<F("write speed and latency") << endl;
    cout << F("speed,max,min,avg") << endl;
    cout << F("KB/Sec,usec,usec,usec") << endl;
    for (uint8_t nTest = 0; nTest < WRITE_COUNT; nTest++) {
    file.seek(0);

    maxLatency = 0;
    minLatency = 9999999;
    totalLatency = 0;
    skipLatency = SKIP_FIRST_LATENCY;
    t = millis();
    for (uint32_t i = 0; i < n; i++) {
      uint32_t m = micros();
      if (file.write(buf, BUF_SIZE) != BUF_SIZE) {
      Serial.println("write failed");
      }
      m = micros() - m;
      totalLatency += m;
      if (skipLatency) {
      // Wait until first write to SD, not just a copy to the cache.
      skipLatency = file.position() < 512;
      } else {
      if (maxLatency < m) {
        maxLatency = m;
      }
      if (minLatency > m) {
        minLatency = m;
      }
      }
    }

    t = millis() - t;
    s = file.size();
    cout << s/t <<',' << maxLatency << ',' << minLatency;
    cout << ',' << totalLatency/n << endl;
    }
    cout << endl << F("Starting read test, please wait.") << endl;
    cout << endl <<F("read speed and latency") << endl;
    cout << F("speed,max,min,avg") << endl;
    cout << F("KB/Sec,usec,usec,usec") << endl;

    // do read test
    for (uint8_t nTest = 0; nTest < READ_COUNT; nTest++) {
      file.seek(0);
      maxLatency = 0;
      minLatency = 9999999;
      totalLatency = 0;
      skipLatency = SKIP_FIRST_LATENCY;
      t = millis();
      for (uint32_t i = 0; i < n; i++) {
        buf[BUF_SIZE-1] = 0;
        uint32_t m = micros();
        int32_t nr = file.read(buf, BUF_SIZE);
        if (nr != BUF_SIZE) {
          Serial.println("read failed");
        }
        m = micros() - m;
        totalLatency += m;
        if (buf[BUF_SIZE-1] != '\n') {
          Serial.println("data check Serial.println");
        }
        if (skipLatency) {
        skipLatency = false;
        } else {
        if (maxLatency < m) {
          maxLatency = m;
        }
        if (minLatency > m) {
          minLatency = m;
        }
      }
     }
    
    s = file.size();
    
    
    t = millis() - t;
    cout << s/t <<',' << maxLatency << ',' << minLatency;
    cout << ',' << totalLatency/n << endl;
    }
    cout << endl << F("Done") << endl;
    file.close();
  //}
}

void loop() {}
 
For SPI flash on my breakout:
Code:
LittleFS Test
flash begin
Flash ID: EF 70 18
Flash size is 16.00 Mbyte
attempting to mount existing media
couldn't mount media, attemping to format
attempting to mount freshly formatted media
success
started
FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
106.90,42979,1146,4789
113.86,107688,1146,4496

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2076.28,1388,154,246
2075.42,1387,154,246

Done

UPDATE: to get the benchmark working with Program had to reduce the filesize to 1MB and the buffer to 3mb, then no read or write errors would appear.
Code:
started
FILE_SIZE_MB = 1
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
520.80,2785,784,982
518.64,2785,784,987

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
47616.00,210,8,10
47616.00,41,8,10

Done
 
Last edited:
Good Morning all,

Lots happens when one sleeps at night ;)

I think I have everything synced back up again. I thought I would try the mtp-test_combined using the QSPI. I know I am late to the party :D

I built it and ran it on my newly soldered up T4.1 with Flash and PSRAM... It showed up on my window machine. Yesterday it looked like the program formatted it... an on WIndows machine in
Teensy\WinBonD I see one file mtpindex.dat

So I wondered what would happen if I tried to cut and paste a couple of files into here. In this case two larger bmp files with the T4 and T4.1 card like stuff on it.

I heard a beep and nothing. If I then click back up on the Teensy in the tree view it shows WinBonD gone

If I hit the program button on T4.1 it reprograms the windows explorer window disappears, if I bring it back up the first file it was trying to copy shows up in index with 0 bytes:

screenshot.jpg

Is this suppose to work?

I included one of the two files, which should look reasonably familiar.

EDIT: @mjs513 tried running your benchmark program on it, and it runs for awhile and then I get a bunch of read/write errors:
Code:
LittleFS Test
started
FILE_SIZE_MB = 5
BUF_SIZE = 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
write failed
write failed
write failed
.... (lots)
write failed
write failed
write failed
281.28,0,9999999,1449

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
read failed
data check Serial.println
read failed
... <lots>
data check Serial.println
read failed
data check Serial.println
39817.48,95,0,9

Done
So maybe I need to re-clean the pads on bottom and dry and maybe reformat it again?
 

Attachments

  • T4.1-Cardlike.jpg
    T4.1-Cardlike.jpg
    214.9 KB · Views: 47
Last edited:
Good Morning all,

Lots happens when one sleeps at night ;)

I think I have everything synced back up again. I thought I would try the mtp-test_combined using the QSPI. I know I am late to the party :D

I built it and ran it on my newly soldered up T4.1 with Flash and PSRAM... It showed up on my window machine. Yesterday it looked like the program formatted it... an on WIndows machine in
Teensy\WinBonD I see one file mtpindex.dat

So I wondered what would happen if I tried to cut and paste a couple of files into here. In this case two larger bmp files with the T4 and T4.1 card like stuff on it.

I heard a beep and nothing. If I then click back up on the Teensy in the tree view it shows WinBonD gone

If I hit the program button on T4.1 it reprograms the windows explorer window disappears, if I bring it back up the first file it was trying to copy shows up in index with 0 bytes:

...

Is this suppose to work?

I included one of the two files, which should look reasonably familiar.
Good Morning Kurt

Know the feeling everything seems to happen when I am sleeping as well :)

Yes it is suppose to work. I just copied and pasted your card image to QSPI as a test with out a problem:
Capture.PNG

Let me repush everything back up to github just in case.

Just pushed it but didn;t look like anything but the example sketch?

EDIT: Try turning off and on again. Just reran it here without errors. Did you remember to change the configuration from Program to QSPI with the defines and the begin?
 
@KurtE
Think we are cross-posting now with these edits?

Check the benchmark sketch is configure for QSPI not Program. The sketch as posted is configured for Program.

The results you are getting is what I am seeing when testing LittleFS_Program so that may be the issue
 
@mjs513 @PaulStoffregen - Wondering if I should try to reformat the QSPI disk, to make sure it is some known state again...

Not sure how to do that yet... Looks like the low level functions are there but I don't see how to gain access yet as things like config I believe are protected...

If not available, there maybe should be a way to call the format if the mount completes, but the data is corrupt. or you simply want to wipe things clean.
 
I will also try it with the original T4.1 beta that had memory on it...

Here is a picture of the old and the new...
screenshot.jpg
The original one is on the bottom of picture. I think they look like the same chips...

EDIT: I tried running MTP disk stuff on the Original T4.1 beta board and was able to copy files there.
So something wrong with other one or formatting...

EDIT2: tried running simple sketch to try to do reformat.
Code:
#include <LittleFS.h>
#include <IntervalTimer.h>
#include <SPI.h>

//#define TEST_RAM
//#define TEST_SPI
#define TEST_QSPI
#ifdef TEST_RAM
LittleFS_RAM myfs;
char buf[200000];
#elif defined(TEST_SPI)
LittleFS_SPIFlash myfs;
#else
LittleFS_QSPIFlash myfs;
#endif

void setup() {
  //  pinMode(13, OUTPUT);
  //  digitalWrite(13, HIGH);
  SPI.begin();
  while (!Serial) ; // wait
  Serial.println("LittleFS Test"); delay(5);

#ifdef TEST_RAM
  if (!myfs.begin(buf, sizeof(buf))) {
#elif defined(TEST_SPI)
  if (!myfs.begin(6)) {
#else
  if (!myfs.begin()) {
#endif
    Serial.println("Error starting LittleFS Disk");
    while (1) ;
  }
  
  Serial.println("started");

  // See if we can format it. 
  Serial.println("*** Enter anything to continue to format ***");
  while (!Serial.available()) ;

  Serial.println("Starting Format");
  if (myfs.format()) {
    Serial.println("Format completed");  
  } else {
    Serial.println("<<<<< Format Failed >>>>");
  }
}
void loop() {
}
Still fails to copy file to it. So again may need to verify I have the right chip...
 
Last edited:
Back
Top