LittleFS port to Teensy/SPIFlash

[defragster]

BTW @KurtE, mjs513, allcon: koromix posted fixed version of TyCommander to work on new TD1.54 USB Serial type change new release done: [URL="https://github.com/Koromix/tytools[/URL]

Big File write on PSRAM fails - more than doubles room for SLACK space and still failing with : -28, // No space left on device
[CODE]
28 dirs with 0 files of Size 0 Bytes
Total 0 files of Size 0 Bytes
Bytes Used: 18944, Bytes Total:16777216
[ 0.06 M]( 0.03 elap) Awaiting input 0123456789RdchkFqvplmuxBbZ+-? loops left 0 >
[ 0.06 M]( 0.00 elap) Awaiting input 0123456789RdchkFqvplmuxBbZ+-? loops left 0 >
Start Big write of 16655872 Bytes............................................. .....
Big write ERR# -28 0xFFFFFFE4

Big write took 20.33 Sec for 16236000 Bytes : file3.size()=2054847098
Big write KBytes per second 798.59 [/CODE]

For Flash about 40K from reported free space worked, now up to this and still fails : uint32_t xx, toWrite = (myfs.totalSize()) - myfs.usedSize() - 102400; // allow for slack space

>> And it takes a about 10 of the reported 20 seconds to report that after the DOTS stop.

This is with 16MB PSRAM - was getting same failure when it was sized 8MB.
Also ~doubled the 102400 to 202400 and it fails still on 8MB and 16MB of RAM.

QUESTION: Is there some alternate limit or problem with LARGE files on :: LittleFS_RAM myfs;

I cut the write size in half with :

Then both work for 4MB and 8MB on 8MB and 16MB PSRAM:
>> EXCEPT on BOTH look at the SAME bad reported :: [B][COLOR="#FF0000"]file3.size()=2054847098[/COLOR][/B]
Use 1 PSRAMs with 8MB

Start Big write of 4083632 Bytes............
Big write took  1.35 Sec for 4080000 Bytes : file3.size()=2054847098
	Big write KBytes per second 3027.19

both PSRAMs with 16MB

 28 dirs with 0 files of Size 0 Bytes
 Total 0 files of Size 0 Bytes
Bytes Used: 18944, Bytes Total:16777216
[  0.06 M](  0.03 elap) Awaiting input 0123456789RdchkFqvplmuxBbZ+-? loops left 0 >
[  0.06 M](  0.00 elap) Awaiting input 0123456789RdchkFqvplmuxBbZ+-? loops left 0 >
[  0.06 M](  0.00 elap) Awaiting input 0123456789RdchkFqvplmuxBbZ+-? loops left 0 >
Start Big write of 8277936 Bytes.........................
Big write took  3.45 Sec for 8276000 Bytes : file3.size()=2054847098
	Big write KBytes per second 2400.48

SKETCH run as is with HALFCUT to see ERROR in file3.size(). RUn again after commenting to see the failed file create:
See p#422 :: LittleFS-port-to-Teensy-SPIFlash

Output from those two runs as follows:

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 24 2020 20:11:09
LittleFS Test : File Integrity

Start Big write of 8287152 Bytes.........................
Big write took  3.38 Sec for 8284000 Bytes : file3.size()=2054847098
	Big write KBytes per second 2449.67 

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 24 2020 20:11:36
LittleFS Test : File Integrity

Start Big write of 16574304 Bytes..................................................
Big write ERR# -28 0xFFFFFFE4 

Big write took 19.98 Sec for 16256000 Bytes : file3.size()=2054847098
	Big write KBytes per second 813.52

<< This is turning into an ADVENTURE >>

<EDIT> See post following - in editing code to post ... #404?

For Proof I ran same sketch selected for QSPI - it is MUCH SLOWER - then the pared down sketch has not FORMAT - but gave the result for NOT HALFSIZE - showing ERROR in the file3.size.
So I ran the LFSintegrity sketch to do a quick format and the T_4.1 went OFFLINE ???
Apparently the FULL disk from running it twice without deleting the file left the MEDIA BROKEN on boot?
LFSintegrity had a FORMAT on boot commented out - but before or after BEGIN the T_4.1 BETA unit #38 won't run LittleFS, even with TWO 15s Restores.

SO the above on RAM will help investigate.
If changed to QSPI as written running TWICE has LittleFS bricked the T_4.1 ...

... gotta run ... too much adventure time used ...

[defragster]

<EDIT> See post following - in editing code to post ...

@Paul - post #401 didn't end where it started ... in the ruin should be steps to run a RAM based BigFile LittleFS that if halfsize completes with wrong file size. If run NOT halfsize it fails to find room for a file expected to fit - and oddly shows the same wrong file.size().

I thought the filesize() on QSPI and SPI was right - at least it was on the sizes used before ( larger ) and with those size and room for the slack space create it would finish.

It was the paired down sketch - no .remove() or even quick format to clear the 'disk' that when run twice it failed the second time where the first was NOT halfsize the second WAS halfsize - it did work first run - but before the output was captured it then got NO SPACE -28 on the 2nd run so I restarted.
> I also stripped out the PrintDirectory and did not add printed info on total and used space that would good feedback - because this started of so simple. And I have LFS type name strings that should have been printed so the output is self documenting ...

Then I remembered - opps - it was FLASH and restart would not free space like on just completed RAM version - it tried to run halfsize again and then it was never seen again and after being offline some short time TyComm dumps the window and I didn't

I can send you the board - T_4.1 Beta #38, or you can try to create your own to investigate - assuming I didn't actually do something else in the process.

However if the RAM size and filesize reporting is fixed for all - maybe it won't have that issue? It is odd it can fill RAM in under 10 seconds then it goes into some deep loop (garbage collection?) for 10 seconds before reporting the -28.

BTW: this was getting a RAM TEST to see what the prior noted lookahead change might do timewise ... without extreme I/O overhead in the way


TyCommander does write GUI prints to a file - this is what I found that should match what I did - missing identifying info have to read bottom up to fins start of QSPI - it looks like the final runs are not indicated:

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 24 2020 20:07:34
LittleFS Test : File Integrity

Start Big write of 8287152 Bytes.........................
Big write took  3.38 Sec for 8284000 Bytes : file3.size()=2054847098
	Big write KBytes per second 2449.73 

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 24 2020 20:08:05
LittleFS Test : File Integrity

Start Big write of 16574304 Bytes..................................................
Big write ERR# -28 0xFFFFFFE4 

Big write took 19.98 Sec for 16256000 Bytes : file3.size()=2054847098
	Big write KBytes per second 813.53 

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 24 2020 20:11:09
LittleFS Test : File Integrity

Start Big write of 8287152 Bytes.........................
Big write took  3.38 Sec for 8284000 Bytes : file3.size()=2054847098
	Big write KBytes per second 2449.85 

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 24 2020 20:11:09    // THIS IS A HALFSIZE RUN - SPEED suggests it was RAM BASED
LittleFS Test : File Integrity

Start Big write of 8287152 Bytes.........................
Big write took  3.38 Sec for 8284000 Bytes : file3.size()=2054847098
	Big write KBytes per second 2449.67 

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 24 2020 20:11:36    // NEW COMPILE TIME - should be QSPI NOT HALFSIZE - RUNS OUT OF SPACE
LittleFS Test : File Integrity

Start Big write of 16574304 Bytes..................................................
Big write ERR# -28 0xFFFFFFE4 

Big write took 19.98 Sec for 16256000 Bytes : file3.size()=2054847098
	Big write KBytes per second 813.52

[WMXZ]

Side Note: with the FS.h file and the like, especially with MTP and the like.

Is there a standard way for other parts of your program like MTP to know when a file is created and/or changes size... i.e. something like the File Change notify you can get in windows and the like.

That is suppose we have an FS or two (or more) showing up on a PC and I have open directory explorer windows open to show the contents. If the user does some form of manipulation on the PC, then MTP would know about it and either rely on the PC notify it of changes or deduce it and try to update all effected files...

But suppose instead you have a folder window on the PC and the Teensy sketch does something like create a new log file and/or append to an existing one... How would MTP (or other sub-systems) have a clue if anything changed in the FS...

Probably as a test, could maybe add an MTP function/method, that says something like mtpd.fsChanged(???) where you maybe pass in which FS object, and some indication of what may have changed...

On the PC side of MTP, you are limited with what you can do "directly" with the file, because file is not on PC but on Teensy. Consider the File Explorer a window into Teensy.

What can you do on PC: create a directory, delete and rename files, down and upload files, move and copy files, for the moment that is all. If you open say a text file with notepad, for example, the file will be first copied to PC and opened read only. Editing file is not enabled. Maybe it can be done, asthere is a MTP extension called begin/end_edit_object, but this is not activated.

Otherwise it is the PC that is in control on what Teensy allows to do, and that leads to 2nd scenario, where Teensy does something, how can MTP initiator know? he cannot. That is where events come into play. Teensy (MTP-responder) should only respond to PC, but be can send asynchronous events tell PC that something has changed. But AFAIK, a PC (MTP-initiator) can ignore all events. So either we know what PC does or we have to write own MTP initiator.

Would it be possible to use USB-Host to write a hybrid USB sniffer that passes USB packets from external device to PC, and observe USB traffic (especially events) to from PC. And simply try different commercial products (cameras, phones etc) to see what they are doing?

[defragster]

ARGHH - in editing code to post ... I set it back to RAM with 16 MB request on a board with QSPI Flash and only 8MB PSRAM - that is why it was dying on start ... opps ...

As noted the code should have printed the FS TYPE in use ... that is now corrected.

Here is output for 4 successive runs as self documented

First two are QSPI - half and full size of big file - the problem is the file3.size()=0

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 25 2020 00:26:19
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes.........................
Big write took 64.35 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 128.67 
Bytes Used: 8192, Bytes Total:16777216

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 25 2020 00:28:05
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 16566624 Bytes...................................................
Big write took 131.26 Sec for 16564000 Bytes : file3.size()=0
	Big write KBytes per second 126.19 
Bytes Used: 8192, Bytes Total:16777216

Same code switched to RAM_DISK Half size works but file3.size()=0 is wrong full BigFile - that worked with those calcs on QSPI Fails on RAM_DISK:

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 25 2020 00:32:47
Started RAM_DISK

Bytes Used: 512, Bytes Total:8388608


Start Big write of 4092848 Bytes............
Big write took  1.33 Sec for 4092000 Bytes : file3.size()=0
	Big write KBytes per second 3072.26 
Bytes Used: 512, Bytes Total:8388608

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 25 2020 00:33:21
Started RAM_DISK

Bytes Used: 512, Bytes Total:8388608


Start Big write of 8185696 Bytes.........................
Big write ERR# -28 0xFFFFFFE4 

Big write took  6.01 Sec for 8128000 Bytes : file3.size()=0
	Big write KBytes per second 1351.87 
Bytes Used: 512, Bytes Total:8388608

// Odd the FAIL case now took 6 not 20 seconds?
// Also diff is that the .size()'s are all zero not the sizes reported before
// Code posted below just shows FS type at start and the disk space info as reported
// As shown in code - the file is .close()'d before printing the disk space.

Code to repro on 16 MB QSPI or 8 MB PSRAM (with #define edits):

#include <LittleFS.h>

#define HALFCUT  // Comment this to see failed LARGE FILE
// HALFCUT defined will show invalid file size

#define TEST_RAM
//#define TEST_SPI
//#define TEST_QSPI
//#define TEST_PROG

// Set for SPI usage
const int FlashChipSelect = 6; // digital pin for flash chip CS pin

#ifdef TEST_RAM
LittleFS_RAM myfs;
// RUNTIME :: extern "C" uint8_t external_psram_size;
EXTMEM char buf[8 * 1024 * 1024];	// USE DMAMEM for more memory than ITCM allows - or remove
//DMAMEM char buf[490000];	// USE DMAMEM for more memory than ITCM allows - or remove
char szDiskMem[] = "RAM_DISK";
#elif defined(TEST_SPI)
//const int FlashChipSelect = 21; // Arduino 101 built-in SPI Flash
#define FORMATSPI
//#define FORMATSPI2
LittleFS_SPIFlash myfs;
char szDiskMem[] = "SPI_DISK";
#elif defined(TEST_PROG)
LittleFS_Program myfs;
char szDiskMem[] = "PRO_DISK";
#else // TEST_QSPI
LittleFS_QSPIFlash myfs;
char szDiskMem[] = "QSPI_DISK";
#endif

File file3;

void setup() {
	while (!Serial) ; // wait
	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);

#ifdef TEST_RAM
	if (!myfs.begin(buf, sizeof(buf))) {
#elif defined(TEST_SPI)
#ifdef FORMATSPI
	if (!myfs.begin( FlashChipSelect )) {
#elif defined(FORMATSPI2)
	pinMode(FlashChipSelect, OUTPUT);
	digitalWriteFast(FlashChipSelect, LOW);
	SPI2.setMOSI(50);
	SPI2.setMISO(54);
	SPI2.setSCK(49);
	SPI2.begin();
	if (!myfs.begin(51, SPI2)) {
#endif
#elif defined(TEST_PROG)
	if (!myfs.begin(1024 * 1024 * 4)) {
#else
	if (!myfs.begin()) {
#endif
		Serial.printf("Error starting %s\n", szDiskMem);
	}
	else
		Serial.printf("Started %s\n", szDiskMem);

	bigFile();
	Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());  // ADDED THIS LINE

	myfs.quickFormat(); // quick format on exit for next run
	Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());
}



void loop() {
	// put your main code here, to run repeatedly:

}

void bigFile(  ) {
	char myFile[] = "/bigfile.txt";

	// FILL DISK
	lfs_ssize_t resW = 1;
	char someData[4000];
	//uint32_t xx, toWrite = (myfs.totalSize()) - myfs.usedSize() - 40960; // allow for slack space :: WORKS on FLASH?
	uint32_t xx, toWrite = (myfs.totalSize()) - myfs.usedSize() - 202400; // allow for slack space
	Serial.printf("\nBytes Used: %llu, Bytes Total:%llu\n\n", myfs.usedSize(), myfs.totalSize());
#ifdef HALFCUT
	toWrite /= 2; // cutting to this works on LittleFS_RAM myfs - except reported file3.size()=2054847098
#endif
	xx = toWrite;
	memset( someData, 'z', 4000 );
	Serial.printf( "\nStart Big write of %u Bytes", xx);
	uint32_t timeMe = micros();
	file3 = myfs.open(myFile, FILE_WRITE);
	int hh = 0;
	while ( toWrite > 4000 && resW > 0 ) {
		resW = file3.write( someData , 4000 );
		hh++;
		if ( !(hh % 80) ) Serial.print('.');
		toWrite -= 4000;
	}
	xx -= toWrite;
	file3.close();
	timeMe = micros() - timeMe;
	file3 = myfs.open(myFile, FILE_WRITE);
	if ( resW < 0 ) {
		Serial.printf( "\nBig write ERR# %i 0x%X \n", resW, resW );
	}
	Serial.printf( "\nBig write took %5.2f Sec for %u Bytes : file3.size()=%u", timeMe / 1000000.0, xx, file3.size() );
	Serial.printf( "\n\tBig write KBytes per second %5.2f \n", xx / (timeMe / 1000.0) );
	file3.close();
}

[mjs513]

Morning All
Tried porting the W25N01G QSPI over to SPI and kind of works - sort of. Yes I put one of them on a breakout board. But running into a strange problem. So here goes me trying to explain.

1. Did a Device Erase - works no problem - I can read all 0xFF's.
2. In setup after initialization did 2 data reads: 1 starting at address 0 and the other starting at address 305000 (quite arbitrary) and then printed out some data and they all read 0xFF. So far so good.
3. At address 0 I wrote 32 constant values of 4 and used a char array(variation on beefy from @defragster) at address 305000. That seemed to work.
4. Read both back and the data read back from the associated address was correct. So far ok
5. Powered off and powered on to see if the data was still there. Sure enough the data was retained by the flash.

Now the problem:
6. Changed the data that I was writing (buf[i] = 4 say to buf[i] = 5) and reran the sketch. Results looked good. Old data still there and shows new data was written, supposedly.
7. Restarted sketch to see if the data on the chip was there wasn't. Read on the data showed some other number usually 0, as shown below:

EF
AA
21
Status of reg 0xa0: 
(HEX: ) 0x00, (Binary: )0

0
Status of reg 0xb0: 
(HEX: ) 0x18, (Binary: )11000

Erasing Device
DONE
Read after Erase
Reading Data beefy
ReadData:
	Target Page: 148
	Column Address: 1896
0x30[0], 0x31[1], 0x32[2], 0x33[3], 0x34[4], 0x35[5], 0x36[6], 0x37[7], 0x38[8], 0x39[9], 0x41[A], 0x61[a], 0x42[B], 0x62[b], 0x43[C], 0x63[c], 0x44[D], 0x64[d], 0x45[E], 0x65[e], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ], 
Reading test
Reading Data
ReadData: [The following data should be showing 0x05']
	Target Page: 0
	Column Address: 0
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
Reading Data
ReadData:  [Read back of address 0 after write to flash]
	Target Page: 0
	Column Address: 0
0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 0x05, 
Loading beefy
ReadData:
	Target Page: 148
	Column Address: 1896
0x30[0], 0x31[1], 0x32[2], 0x33[3], 0x34[4], 0x35[5], 0x36[6], 0x37[7], 0x38[8], 0x39[9], 0x41[A], 0x61[a], 0x42[B], 0x62[b], 0x43[C], 0x63[c], 0x44[D], 0x64[d], 0x45[E], 0x65[e], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ], 0x00[ ],

I really can't figure out whats wrong. If the data is not 0xff it doesn't write (have to reread the spec on this one again) At this point I am stuck. Will keep looking but if anyone wants to give it a try I am attaching the sketch - note not complete so keep to the basics for now.

[KurtE]

@mjs513 Sound like fun. Was wondering if you are using a breakout board for the memory that you made or are there some you can order?

[mjs513]

@mjs513 Sound like fun. Was wondering if you are using a breakout board for the memory that you made or are there some you can order?

@KurtE - didn't see any breakout boards for it but doesn't mean they are not out there. I made mine own using one of thest: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1. Actually thought I posted this on the NAND thread but guess I am loosing my mind I know nothing new.

[defragster]

Ahhh Beefy ... nice it had a unique name SublimeText found it.

I need to put the github fresh copy of LittleFX to confirm that issue I noted on file.size() from local copy corruption ...

[KurtE]

@KurtE - didn't see any breakout boards for it but doesn't mean they are not out there. I made mine own using one of thest: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1. Actually thought I posted this on the NAND thread but guess I am loosing my mind I know nothing new.

Actually it was probably me being lazy and figured easier to ask than search

Here is a quick and dirty layout that is similar to an Adafruit design for FSRAM...



What I sometimes don't understand is when does the chip know to use the D2/D3 pins... My guess is only by the actual commands/requests sent to it?

I went ahead and added PU resistor to HOLD/RESET/D3 pin as I think at least the one chip we are using says can be floating or tied high. So far I did not do anything with the WP pin, not sure if it should be optionally tied high or low? May add both just in case

[Frank B]

When WP-E=0, the device is in the Software Protection mode that only SR-1 can be protected. The /WPpin functions as adataI/O pin for the Quad SPI operations, as well as an active low input pin for the Write Protection function for SR-1. Refer to section 7.1.3 for detail information.When WP-E=1, the device is in the Hardware Protection mode that /WP becomes a dedicated active low input pin for the Write Protection of the entire device. If /WP is tied to GND, all “Write/Program/Erase” functions are disabled. The entire device (including all registers, memory array and OTPpages) will become read-only. Quad SPI read operationsare also disabled when WP-E is set to 1

The chips switch to QSPI via a software-command (register-write)

[KurtE]

Thanks Frank,

That is what I thought I was reading...

So Just for the heck of it added option PU or PD on both those two signals... Then If I screwed up, can wire it one way or the other... Or not populate and go to the exported pins.



Besides I did not change size of board so would cost the same.

[mjs513]

Thanks Frank,

That is what I thought I was reading...

So Just for the heck of it added option PU or PD on both those two signals... Then If I screwed up, can wire it one way or the other... Or not populate and go to the exported pins.



Besides I did not change size of board so would cost the same.

Very cool. Care to share your Gerbers Although think I would have edited the pad length on the pins to make them longer for things like the W25N01G chip Right now not using any pull ups on any of the lines - not sure if I need too. Any suggestions.

Oh - played some more with the SPI version and still can not find the problem. It should theoretically work - how many times have you heard that one!

[Frank B]

For SPI, I used to connect wp and reset with vcc, no pullups, direct connection. But maybe that's not needed. You say it works for you without that, so why not...

[KurtE]

I ordered from OSHPark... (6 for $4) So should maybe have an extra.

Not sure if you need PU or not...

The zip file is the one I uploaded to OSHPark, plus added directory with the Diptrace schematic and layout files

[mjs513]

I ordered from OSHPark... (6 for $4) So should maybe have an extra.

Not sure if you need PU or not...

The zip file is the one I uploaded to OSHPark, plus added directory with the Diptrace schematic and layout files

Thanks Kurt. Been looking at some flash boards with pullups but it all over the place 1 has 10ks on all pins, another on CS, WP and hold. Not electronics guy so .... I will leave it at that

[KurtE]

Note: I think these pads may be long enough already... Could be wrong, but...

Note the measurements are in mills. And I believe that the WinBond other one is about 315mil? Maybe wrong.

Once I have one of these will be fun to try out some memory chips and see if it works with my castellated T4.1 board where I bring out the bottom memory chip pins to external pads.

[defragster]

Getting the 64MB Flash from ProtoSupplies - Ken noted the center open solder pad - making sure it didn't get shorted during assembly - so as long as not too close in the middle - and soldered square - and the traces all masked like the T_4.1 we have done.

He said they arrived so will have them next week - the short breakable M&F headers not there yet so will wait to get a resupply of those that @mjs513 shared before.

[mjs513]

Note: I think these pads may be long enough already... Could be wrong, but...

Note the measurements are in mills. And I believe that the WinBond other one is about 315mil? Maybe wrong.

Once I have one of these will be fun to try out some memory chips and see if it works with my castellated T4.1 board where I bring out the bottom memory chip pins to external pads.

Just checked and you are write based on the envelop drawing - 315 mils.

[KurtE]

@defragster and all

Yesterday I soldered up another T4.1 with PSRAM and FLEX , to get an idea of speed if the one I was using was somehow different.

So tried your recent test, editd for QSPI (left the half defined):

With the new board:

C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 25 2020 17:11:43
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes.........................
Big write took 100.51 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 82.38 
Bytes Used: 8192, Bytes Total:16777216

My previous soldered up board:

C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 25 2020 17:11:43
Started QSPI_DISK

Bytes Used: 1093632, Bytes Total:16777216


Start Big write of 7740592 Bytes........................
Big write took 96.52 Sec for 7740000 Bytes : file3.size()=0
	Big write KBytes per second 80.19 
Bytes Used: 8192, Bytes Total:16777216

And The beta board:

C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 25 2020 17:25:21
Started QSPI_DISK

Bytes Used: 1150976, Bytes Total:16777216


Start Big write of 7711920 Bytes........................
Big write took 58.51 Sec for 7708000 Bytes : file3.size()=0
	Big write KBytes per second 131.74 
Bytes Used: 8192, Bytes Total:16777216

So I wonder why the Beta board is about 1.5 times faster then the two I soldered up. The recent board chip came from PJRC, the 2nd one came from Digikey, and not sure where Paul got them for the beta boards?

Edit: Same program ran on all three boards, I simply turned each one on and hit the program button.

[defragster]

@defragster and all

Yesterday I soldered up another T4.1 with PSRAM and FLEX , to get an idea of speed if the one I was using was somehow different.

So tried your recent test, editd for QSPI (left the half defined):

...

Big write took 100.51 Sec for 8280000 Bytes : file3.size()=0
Big write took 96.52 Sec for 7740000 Bytes : file3.size()=0
Big write took 58.51 Sec for 7708000 Bytes : file3.size()=0

...

So I wonder why the Beta board is about 1.5 times faster then the two I soldered up. The recent board chip came from PJRC, the 2nd one came from Digikey, and not sure where Paul got them for the beta boards?

Edit: Same program ran on all three boards, I simply turned each one on and hit the program button.

Interesting to see you got the ZERO file.size() repro

Depending on what else was on the disk - it may have had more cleanup?

But the one was probably empty - and the file writes single entry to root - and that isn't a lot of work as far as I saw for a single entry.

Though you can see altered Bytes written suggest there was other stuff on the disk already.

I have been taking a break with other distractions - in the middle of one now - and have not posted the latest LFSintegrity to github as noted ...

It seems I did add quick format at the end of that BigTest though? So running again on all three will shows them EMPTY and start with the same space.

Yes I did - but the last lines should be altered to peek at the disk used space after the bigFile() write - adding the BOLD line I'll do above:

	bigFile();
	Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());

	myfs.quickFormat(); // quick format on exit for next run
	Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());

[defragster]

Got the current github LittleFS - knew my gated changes only applied to Q/SPI :: and RAM showed the same .size()==0 but with that and edit above here is output on one T_4.1 with RAM then QSPI:

Started RAM_DISK

Bytes Used: 512, Bytes Total:8388608


Start Big write of 4092848 Bytes............
Big write took  1.32 Sec for 4092000 Bytes : file3.size()=0
	Big write KBytes per second 3095.85 
Bytes Used: 4224512, Bytes Total:8388608
Bytes Used: 512, Bytes Total:8388608

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 25 2020 19:22:23
Started QSPI_DISK

Bytes Used: 3592192, Bytes Total:16777216


Start Big write of 6491312 Bytes....................
Big write took 53.19 Sec for 6488000 Bytes : file3.size()=0
	Big write KBytes per second 121.98 
Bytes Used: 10096640, Bytes Total:16777216
Bytes Used: 8192, Bytes Total:16777216

With the added file print before format it shows Overall Disk space has dropped from the added bigfile.txt.
The the quick format shows the space is clean ... except for root directory holder.

@KurtE - those are slow times - unit above is production unit with 16M Flash soldered here.
Below are two runs on the Beta unit I thought I broke - middle lines between runs deleted:

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 25 2020 19:22:23
Started QSPI_DISK
Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes.........................
Big write took 66.14 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 125.19 

Start Big write of 8283312 Bytes.........................
Big write took 67.14 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 123.32 


Bytes Used: 8306688, Bytes Total:16777216
Bytes Used: 8192, Bytes Total:16777216

[defragster]

@Paul if you look at this - things have changed - mostly the details printed in samples sketch - current copy below, AND the print format strings used ????
>> Just seeing the LittleFS functions return uint64_t's - opps - prtinf( ... %llu ... ) {okay those annoying warnings would have helped here }

With %llu the file size is showing correctly on the HALFCUT - Problem solved of .size()=0 is now showing properly

As far as -28 : The added print data indicates the source of that problem - though used size on empty drive 512 .vs. 8192 is a big clue
>> RAM allocation units create more overhead/slack space in the disk structures causing written data to consume MUCH MORE space that on FLASH MEDIA for a Large file:
- because of the smaller :: config.block_size = 256; versus config.block_size = 4096; The Metadata and storage blocks will cascade 'wastefully'
>> When using Large PSRAM of 8 or 16 MB the block sizing should match the values used for larger Flash ???
Space loss is about 600KB with current RAM params versus FLASH this works :: uint32_t toWrite = (myfs.totalSize()) - myfs.usedSize() - 640000; // allow for slack space
RAM : Bytes Used: 16657152, Bytes Total:16777216 .vs. QSPI : Bytes Used: 16171008, Bytes Total:16777216

#include <LittleFS.h>

#define HALFCUT  // Comment this to see failed LARGE FILE
// HALFCUT defined will show invalid file size

#define TEST_RAM
//#define TEST_SPI
//#define TEST_QSPI
//#define TEST_PROG

// Set for SPI usage
const int FlashChipSelect = 6; // digital pin for flash chip CS pin

#ifdef TEST_RAM
LittleFS_RAM myfs;
// RUNTIME :: extern "C" uint8_t external_psram_size;
EXTMEM char buf[8 * 1024 * 1024];	// USE DMAMEM for more memory than ITCM allows - or remove
//DMAMEM char buf[490000];	// USE DMAMEM for more memory than ITCM allows - or remove
char szDiskMem[] = "RAM_DISK";
#elif defined(TEST_SPI)
//const int FlashChipSelect = 21; // Arduino 101 built-in SPI Flash
#define FORMATSPI
//#define FORMATSPI2
LittleFS_SPIFlash myfs;
char szDiskMem[] = "SPI_DISK";
#elif defined(TEST_PROG)
LittleFS_Program myfs;
char szDiskMem[] = "PRO_DISK";
#else // TEST_QSPI
LittleFS_QSPIFlash myfs;
char szDiskMem[] = "QSPI_DISK";
#endif

File file3;

void setup() {
	while (!Serial) ; // wait
	Serial.println("\n" __FILE__ " " __DATE__ " " __TIME__);

#ifdef TEST_RAM
	if (!myfs.begin(buf, sizeof(buf))) {
#elif defined(TEST_SPI)
#ifdef FORMATSPI
	if (!myfs.begin( FlashChipSelect )) {
#elif defined(FORMATSPI2)
	pinMode(FlashChipSelect, OUTPUT);
	digitalWriteFast(FlashChipSelect, LOW);
	SPI2.setMOSI(50);
	SPI2.setMISO(54);
	SPI2.setSCK(49);
	SPI2.begin();
	if (!myfs.begin(51, SPI2)) {
#endif
#elif defined(TEST_PROG)
	if (!myfs.begin(1024 * 1024 * 4)) {
#else
	if (!myfs.begin()) {
#endif
		Serial.printf("Error starting %s\n", szDiskMem);
	}
	else
		Serial.printf("Started %s\n", szDiskMem);

	bigFile();
	Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());

	myfs.quickFormat(); // quick format on exit for next run
	Serial.printf("\t\tmyfs.quickFormat() Completed\n");
	Serial.printf("Bytes Used: %llu, Bytes Total:%llu\n", myfs.usedSize(), myfs.totalSize());
}



void loop() {
	// put your main code here, to run repeatedly:

}

void bigFile(  ) {
	char myFile[] = "/bigfile.txt";

	// FILL DISK
	lfs_ssize_t resW = 1;
	char someData[4000];
	//uint32_t xx, toWrite = (myfs.totalSize()) - myfs.usedSize() - 40960; // allow for slack space :: WORKS on FLASH?
	uint32_t xx;
	uint32_t toWrite = (myfs.totalSize()) - myfs.usedSize() - 202400; // allow for slack space
	Serial.printf("\nBytes Used: %llu, Bytes Total:%llu\n\n", myfs.usedSize(), myfs.totalSize());
#ifdef HALFCUT
	toWrite /= 2; // cutting to this works on LittleFS_RAM myfs - except reported file3.size()=2054847098 OR now 0
#endif
	xx = toWrite;
	memset( someData, 'z', 4000 );
	Serial.printf( "\nStart Big write of %lu Bytes\n", (toWrite - toWrite % 4000));
	uint32_t timeMe = micros();
	file3 = myfs.open(myFile, FILE_WRITE);
	int hh = 0;
	while ( toWrite > 4000 && resW > 0 ) {
		resW = file3.write( someData , 4000 );
		hh++;
		if ( !(hh % 80) ) {
			Serial.print('.');
			Serial.printf("  Bytes To Write:%llu\tBytes Used: %llu, totalSize()=%llu {diff=%llu)\n", (toWrite - toWrite % 4000), myfs.usedSize(), myfs.totalSize(), myfs.totalSize() - myfs.usedSize() );
		}
		toWrite -= 4000;
	}
	xx -= toWrite;
	file3.close();
	timeMe = micros() - timeMe;
	file3 = myfs.open(myFile, FILE_WRITE);
	if ( resW < 0 ) {
		Serial.printf( "\nBig write ERR# %i 0x%X \n", resW, resW );
	}
	Serial.printf( "\nBig write took %5.2f Sec for %llu Bytes : file3.size()=%llu", timeMe / 1000000.0, xx, file3.size() );
	Serial.printf( "\n\tBig write KBytes per second %5.2f \n", xx / (timeMe / 1000.0) );
	file3.close();
}

Here is output run first as NOT_DEFINED HALFCUT - then as HALFCUT defined:

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 26 2020 03:19:10
Started RAM_DISK

Bytes Used: 512, Bytes Total:8388608


Start Big write of 8184000 Bytes
.  Bytes To Write:7868000	Bytes Used: 331008, totalSize()=8388608 {diff=8057600)
.  Bytes To Write:7548000	Bytes Used: 661248, totalSize()=8388608 {diff=7727360)
.  Bytes To Write:7228000	Bytes Used: 991488, totalSize()=8388608 {diff=7397120)
.  Bytes To Write:6908000	Bytes Used: 1321984, totalSize()=8388608 {diff=7066624)
.  Bytes To Write:6588000	Bytes Used: 1652224, totalSize()=8388608 {diff=6736384)
.  Bytes To Write:6268000	Bytes Used: 1982464, totalSize()=8388608 {diff=6406144)
.  Bytes To Write:5948000	Bytes Used: 2312960, totalSize()=8388608 {diff=6075648)
.  Bytes To Write:5628000	Bytes Used: 2643200, totalSize()=8388608 {diff=5745408)
.  Bytes To Write:5308000	Bytes Used: 2973440, totalSize()=8388608 {diff=5415168)
.  Bytes To Write:4988000	Bytes Used: 3303936, totalSize()=8388608 {diff=5084672)
.  Bytes To Write:4668000	Bytes Used: 3634176, totalSize()=8388608 {diff=4754432)
.  Bytes To Write:4348000	Bytes Used: 3964416, totalSize()=8388608 {diff=4424192)
.  Bytes To Write:4028000	Bytes Used: 4294912, totalSize()=8388608 {diff=4093696)
.  Bytes To Write:3708000	Bytes Used: 4625152, totalSize()=8388608 {diff=3763456)
.  Bytes To Write:3388000	Bytes Used: 4955392, totalSize()=8388608 {diff=3433216)
.  Bytes To Write:3068000	Bytes Used: 5285888, totalSize()=8388608 {diff=3102720)
.  Bytes To Write:2748000	Bytes Used: 5616128, totalSize()=8388608 {diff=2772480)
.  Bytes To Write:2428000	Bytes Used: 5946368, totalSize()=8388608 {diff=2442240)
.  Bytes To Write:2108000	Bytes Used: 6276608, totalSize()=8388608 {diff=2112000)
.  Bytes To Write:1788000	Bytes Used: 6607104, totalSize()=8388608 {diff=1781504)
.  Bytes To Write:1468000	Bytes Used: 6937344, totalSize()=8388608 {diff=1451264)
.  Bytes To Write:1148000	Bytes Used: 7267584, totalSize()=8388608 {diff=1121024)
.  Bytes To Write:828000	Bytes Used: 7598080, totalSize()=8388608 {diff=790528)
.  Bytes To Write:508000	Bytes Used: 7928320, totalSize()=8388608 {diff=460288)
.  Bytes To Write:188000	Bytes Used: 8258560, totalSize()=8388608 {diff=130048) // NOT ENOUGH SPACE diff too small

Big write ERR# -28 0xFFFFFFE4 

Big write took  7.07 Sec for 8128000 Bytes : file3.size()=0
	Big write KBytes per second 1149.72 
Bytes Used: 512, Bytes Total:8388608
		myfs.quickFormat() Completed
Bytes Used: 512, Bytes Total:8388608

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 26 2020 03:21:44
Started RAM_DISK

Bytes Used: 512, Bytes Total:8388608


Start Big write of 4092000 Bytes
.  Bytes To Write:3776000	Bytes Used: 331008, totalSize()=8388608 {diff=8057600)
.  Bytes To Write:3456000	Bytes Used: 661248, totalSize()=8388608 {diff=7727360)
.  Bytes To Write:3136000	Bytes Used: 991488, totalSize()=8388608 {diff=7397120)
.  Bytes To Write:2816000	Bytes Used: 1321984, totalSize()=8388608 {diff=7066624)
.  Bytes To Write:2496000	Bytes Used: 1652224, totalSize()=8388608 {diff=6736384)
.  Bytes To Write:2176000	Bytes Used: 1982464, totalSize()=8388608 {diff=6406144)
.  Bytes To Write:1856000	Bytes Used: 2312960, totalSize()=8388608 {diff=6075648)
.  Bytes To Write:1536000	Bytes Used: 2643200, totalSize()=8388608 {diff=5745408)
.  Bytes To Write:1216000	Bytes Used: 2973440, totalSize()=8388608 {diff=5415168)
.  Bytes To Write:896000	Bytes Used: 3303936, totalSize()=8388608 {diff=5084672)
.  Bytes To Write:576000	Bytes Used: 3634176, totalSize()=8388608 {diff=4754432)
.  Bytes To Write:256000	Bytes Used: 3964416, totalSize()=8388608 {diff=4424192)

Big write took  1.58 Sec for 4092000 Bytes : file3.size()=4092000
	Big write KBytes per second 2588.71 
Bytes Used: 4224512, Bytes Total:8388608
		myfs.quickFormat() Completed
Bytes Used: 512, Bytes Total:8388608

Running the same code on QSPI FLASH with same calc you can see it has MUCH more space FREE after writing the file
Yes this is an Oranges to Mandarins as FLASH if 16 MB and tested PSRAM is 8 MB ::

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 26 2020 03:26:28
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 16564000 Bytes
.  Bytes To Write:16248000	Bytes Used: 331776, totalSize()=16777216 {diff=16445440)
.  Bytes To Write:15928000	Bytes Used: 651264, totalSize()=16777216 {diff=16125952)
.  Bytes To Write:15608000	Bytes Used: 970752, totalSize()=16777216 {diff=15806464)
.  Bytes To Write:15288000	Bytes Used: 1294336, totalSize()=16777216 {diff=15482880)
.  Bytes To Write:14968000	Bytes Used: 1613824, totalSize()=16777216 {diff=15163392)
.  Bytes To Write:14648000	Bytes Used: 1933312, totalSize()=16777216 {diff=14843904)
.  Bytes To Write:14328000	Bytes Used: 2252800, totalSize()=16777216 {diff=14524416)
.  Bytes To Write:14008000	Bytes Used: 2576384, totalSize()=16777216 {diff=14200832)
.  Bytes To Write:13688000	Bytes Used: 2895872, totalSize()=16777216 {diff=13881344)
.  Bytes To Write:13368000	Bytes Used: 3215360, totalSize()=16777216 {diff=13561856)
.  Bytes To Write:13048000	Bytes Used: 3538944, totalSize()=16777216 {diff=13238272)
.  Bytes To Write:12728000	Bytes Used: 3858432, totalSize()=16777216 {diff=12918784)
.  Bytes To Write:12408000	Bytes Used: 4177920, totalSize()=16777216 {diff=12599296)
.  Bytes To Write:12088000	Bytes Used: 4497408, totalSize()=16777216 {diff=12279808)
.  Bytes To Write:11768000	Bytes Used: 4820992, totalSize()=16777216 {diff=11956224)
.  Bytes To Write:11448000	Bytes Used: 5140480, totalSize()=16777216 {diff=11636736)
.  Bytes To Write:11128000	Bytes Used: 5459968, totalSize()=16777216 {diff=11317248)
.  Bytes To Write:10808000	Bytes Used: 5779456, totalSize()=16777216 {diff=10997760)
.  Bytes To Write:10488000	Bytes Used: 6103040, totalSize()=16777216 {diff=10674176)
.  Bytes To Write:10168000	Bytes Used: 6422528, totalSize()=16777216 {diff=10354688)
.  Bytes To Write:9848000	Bytes Used: 6742016, totalSize()=16777216 {diff=10035200)
.  Bytes To Write:9528000	Bytes Used: 7065600, totalSize()=16777216 {diff=9711616)
.  Bytes To Write:9208000	Bytes Used: 7385088, totalSize()=16777216 {diff=9392128)
.  Bytes To Write:8888000	Bytes Used: 7704576, totalSize()=16777216 {diff=9072640)
.  Bytes To Write:8568000	Bytes Used: 8024064, totalSize()=16777216 {diff=8753152)
.  Bytes To Write:8248000	Bytes Used: 8347648, totalSize()=16777216 {diff=8429568)
.  Bytes To Write:7928000	Bytes Used: 8667136, totalSize()=16777216 {diff=8110080)
.  Bytes To Write:7608000	Bytes Used: 8986624, totalSize()=16777216 {diff=7790592)
.  Bytes To Write:7288000	Bytes Used: 9310208, totalSize()=16777216 {diff=7467008)
.  Bytes To Write:6968000	Bytes Used: 9629696, totalSize()=16777216 {diff=7147520)
.  Bytes To Write:6648000	Bytes Used: 9949184, totalSize()=16777216 {diff=6828032)
.  Bytes To Write:6328000	Bytes Used: 10268672, totalSize()=16777216 {diff=6508544)
.  Bytes To Write:6008000	Bytes Used: 10592256, totalSize()=16777216 {diff=6184960)
.  Bytes To Write:5688000	Bytes Used: 10911744, totalSize()=16777216 {diff=5865472)
.  Bytes To Write:5368000	Bytes Used: 11231232, totalSize()=16777216 {diff=5545984)
.  Bytes To Write:5048000	Bytes Used: 11550720, totalSize()=16777216 {diff=5226496)
.  Bytes To Write:4728000	Bytes Used: 11874304, totalSize()=16777216 {diff=4902912)
.  Bytes To Write:4408000	Bytes Used: 12193792, totalSize()=16777216 {diff=4583424)
.  Bytes To Write:4088000	Bytes Used: 12513280, totalSize()=16777216 {diff=4263936)
.  Bytes To Write:3768000	Bytes Used: 12836864, totalSize()=16777216 {diff=3940352)
.  Bytes To Write:3448000	Bytes Used: 13156352, totalSize()=16777216 {diff=3620864)
.  Bytes To Write:3128000	Bytes Used: 13475840, totalSize()=16777216 {diff=3301376)
.  Bytes To Write:2808000	Bytes Used: 13795328, totalSize()=16777216 {diff=2981888)
.  Bytes To Write:2488000	Bytes Used: 14118912, totalSize()=16777216 {diff=2658304)
.  Bytes To Write:2168000	Bytes Used: 14438400, totalSize()=16777216 {diff=2338816)
.  Bytes To Write:1848000	Bytes Used: 14757888, totalSize()=16777216 {diff=2019328)
.  Bytes To Write:1528000	Bytes Used: 15081472, totalSize()=16777216 {diff=1695744)
.  Bytes To Write:1208000	Bytes Used: 15400960, totalSize()=16777216 {diff=1376256)
.  Bytes To Write:888000	Bytes Used: 15720448, totalSize()=16777216 {diff=1056768)
.  Bytes To Write:568000	Bytes Used: 16039936, totalSize()=16777216 {diff=737280)
.  Bytes To Write:248000	Bytes Used: 16363520, totalSize()=16777216 {diff=413696)

Big write took 135.58 Sec for 16564000 Bytes : file3.size()=16564000
	Big write KBytes per second 122.17 
Bytes Used: 16605184, Bytes Total:16777216
		myfs.quickFormat() Completed
Bytes Used: 8192, Bytes Total:16777216

So for 16 MB Oranges to Oranges here is a 16 MB PSRAM with code edited 8 to 16 in the EXTMEM buf[] ALLOC:

T:\tCode\littleFS\RAM_BadBigFile\RAM_BadBigFile.ino Nov 26 2020 03:31:35
Started RAM_DISK

Bytes Used: 512, Bytes Total:16777216


Start Big write of 16572000 Bytes
.  Bytes To Write:16256000	Bytes Used: 331008, totalSize()=16777216 {diff=16446208)
.  Bytes To Write:15936000	Bytes Used: 661248, totalSize()=16777216 {diff=16115968)
.  Bytes To Write:15616000	Bytes Used: 991488, totalSize()=16777216 {diff=15785728)
.  Bytes To Write:15296000	Bytes Used: 1321984, totalSize()=16777216 {diff=15455232)
.  Bytes To Write:14976000	Bytes Used: 1652224, totalSize()=16777216 {diff=15124992)
.  Bytes To Write:14656000	Bytes Used: 1982464, totalSize()=16777216 {diff=14794752)
.  Bytes To Write:14336000	Bytes Used: 2312960, totalSize()=16777216 {diff=14464256)
.  Bytes To Write:14016000	Bytes Used: 2643200, totalSize()=16777216 {diff=14134016)
.  Bytes To Write:13696000	Bytes Used: 2973440, totalSize()=16777216 {diff=13803776)
.  Bytes To Write:13376000	Bytes Used: 3303936, totalSize()=16777216 {diff=13473280)
.  Bytes To Write:13056000	Bytes Used: 3634176, totalSize()=16777216 {diff=13143040)
.  Bytes To Write:12736000	Bytes Used: 3964416, totalSize()=16777216 {diff=12812800)
.  Bytes To Write:12416000	Bytes Used: 4294912, totalSize()=16777216 {diff=12482304)
.  Bytes To Write:12096000	Bytes Used: 4625152, totalSize()=16777216 {diff=12152064)
.  Bytes To Write:11776000	Bytes Used: 4955392, totalSize()=16777216 {diff=11821824)
.  Bytes To Write:11456000	Bytes Used: 5285888, totalSize()=16777216 {diff=11491328)
.  Bytes To Write:11136000	Bytes Used: 5616128, totalSize()=16777216 {diff=11161088)
.  Bytes To Write:10816000	Bytes Used: 5946368, totalSize()=16777216 {diff=10830848)
.  Bytes To Write:10496000	Bytes Used: 6276608, totalSize()=16777216 {diff=10500608)
.  Bytes To Write:10176000	Bytes Used: 6607104, totalSize()=16777216 {diff=10170112)
.  Bytes To Write:9856000	Bytes Used: 6937344, totalSize()=16777216 {diff=9839872)
.  Bytes To Write:9536000	Bytes Used: 7267584, totalSize()=16777216 {diff=9509632)
.  Bytes To Write:9216000	Bytes Used: 7598080, totalSize()=16777216 {diff=9179136)
.  Bytes To Write:8896000	Bytes Used: 7928320, totalSize()=16777216 {diff=8848896)
.  Bytes To Write:8576000	Bytes Used: 8258560, totalSize()=16777216 {diff=8518656)
.  Bytes To Write:8256000	Bytes Used: 8589056, totalSize()=16777216 {diff=8188160)
.  Bytes To Write:7936000	Bytes Used: 8919296, totalSize()=16777216 {diff=7857920)
.  Bytes To Write:7616000	Bytes Used: 9249536, totalSize()=16777216 {diff=7527680)
.  Bytes To Write:7296000	Bytes Used: 9580032, totalSize()=16777216 {diff=7197184)
.  Bytes To Write:6976000	Bytes Used: 9910272, totalSize()=16777216 {diff=6866944)
.  Bytes To Write:6656000	Bytes Used: 10240512, totalSize()=16777216 {diff=6536704)
.  Bytes To Write:6336000	Bytes Used: 10571008, totalSize()=16777216 {diff=6206208)
.  Bytes To Write:6016000	Bytes Used: 10901248, totalSize()=16777216 {diff=5875968)
.  Bytes To Write:5696000	Bytes Used: 11231488, totalSize()=16777216 {diff=5545728)
.  Bytes To Write:5376000	Bytes Used: 11561984, totalSize()=16777216 {diff=5215232)
.  Bytes To Write:5056000	Bytes Used: 11892224, totalSize()=16777216 {diff=4884992)
.  Bytes To Write:4736000	Bytes Used: 12222464, totalSize()=16777216 {diff=4554752)
.  Bytes To Write:4416000	Bytes Used: 12552960, totalSize()=16777216 {diff=4224256)
.  Bytes To Write:4096000	Bytes Used: 12883200, totalSize()=16777216 {diff=3894016)
.  Bytes To Write:3776000	Bytes Used: 13213440, totalSize()=16777216 {diff=3563776)
.  Bytes To Write:3456000	Bytes Used: 13543936, totalSize()=16777216 {diff=3233280)
.  Bytes To Write:3136000	Bytes Used: 13874176, totalSize()=16777216 {diff=2903040)
.  Bytes To Write:2816000	Bytes Used: 14204416, totalSize()=16777216 {diff=2572800)
.  Bytes To Write:2496000	Bytes Used: 14534656, totalSize()=16777216 {diff=2242560)
.  Bytes To Write:2176000	Bytes Used: 14865152, totalSize()=16777216 {diff=1912064)
.  Bytes To Write:1856000	Bytes Used: 15195392, totalSize()=16777216 {diff=1581824)
.  Bytes To Write:1536000	Bytes Used: 15525632, totalSize()=16777216 {diff=1251584)
.  Bytes To Write:1216000	Bytes Used: 15856128, totalSize()=16777216 {diff=921088)
.  Bytes To Write:896000	Bytes Used: 16186368, totalSize()=16777216 {diff=590848)
.  Bytes To Write:576000	Bytes Used: 16516608, totalSize()=16777216 {diff=260608)

Big write ERR# -28 0xFFFFFFE4 

Big write took 24.36 Sec for 16256000 Bytes : file3.size()=0
	Big write KBytes per second 667.23 
Bytes Used: 512, Bytes Total:16777216
		myfs.quickFormat() Completed
Bytes Used: 512, Bytes Total:16777216

[defragster]

opps for the big interruption to find something that was obvious ... Large RAM needs larger blocks ... and not knowing the file sys functions returned 64 bit values

back to the point of that test for LittleFS_RAM on 16GB PSRAM::

		config.block_count = size / 256; // 16777216/256 == 65536
		//config.lookahead_size = 64;  // was 64
		config.lookahead_size = (config.block_count / 8 + 7) & 0xFFFFFFF8;   // 8192

With 64 WAS :: 
Big write took 13.45 Sec for 16136000 Bytes : file3.size()=16136000
With 128X LARGER of 8192 ::
Big write took  8.10 Sec for 16136000 Bytes : file3.size()=16136000

NOTE:: This also explains the LONG time for the FAIL out of space timeout of 24 to fail when it works in 13 seconds.
Here is the result of the FAIL case when the allowance for the SLACK space is reduced and the bigfile() write fails - using larger lookahead

Big write ERR# -28 0xFFFFFFE4 

Big write took  8.22 Sec for 16256000 Bytes : file3.size()=0

On QSPI change to lookahead makes no difference - the major time is spent formatting blocks:

Big write took 132.00 Sec for 16368000 Bytes : file3.size()=16368000
	Big write KBytes per second 124.00 
.versus.
Big write took 132.03 Sec for 16368000 Bytes : file3.size()=16368000
	Big write KBytes per second 123.97

The surprise answer is the 0.03 longer one was with 512.vs.256 :: config.lookahead_size = (config.block_count / 8 + 7) & 0xFFFFFFF8; and the first used config.lookahead_size = info->progsize;

I ran again on both to confirm and this time the 'AS WRITTEN= info->progsize;' was 0.17 sec faster

[defragster]

Back to the real issue in #422 (tiny alloc blocks in BIG PSRAM) doing this for LittleFS_RAM() { } :: bool begin(()
> it runs maybe 3-5X faster - but I get LFSintegrity file verify failure so something is mismatched

		if ( size > 1024*1024 ) {
			config.read_size = 64;
			config.prog_size = 64;
			config.block_size = 4096;
			config.block_count = size / 4096;
			config.block_cycles = 900;
			config.cache_size = 256;
			//config.lookahead_size = 64;  // was 64
			config.lookahead_size = 512; // (config.block_count / 8 + 7) & 0xFFFFFFF8;
		}
		else {
			config.read_size = 64;
			config.prog_size = 64;
			config.block_size = 256;
			config.block_count = size / 256;
			config.block_cycles = 50;
			config.cache_size = 64;
			config.lookahead_size = 64;  // was 64
			//config.lookahead_size = (config.block_count / 8 + 7) & 0xFFFFFFF8;
		}

Makes 16 MB PSRAM do this:

Big write took  2.48 Sec for 16728000 Bytes : file3.size()=16728000
	Big write KBytes per second 6758.09 
Bytes Used: 16777216, Bytes Total:16777216

TWO THINGS:
> The Speed 2.48 versus 13 seconds for a ~16 MB file
> AND that is a full size bigfile now fitting 16728000 bytes versus 16572000
-> using :: uint32_t toWrite = (myfs.totalSize()) - myfs.usedSize() - 40960; // allow for slack space :: WORKS on FLASH?


Notes on:: config.block_cycles
> For RAM this should be MAX as it won't wear out
> For Flash - since it may always store DIR data in the same spot given the same program behavior - it should tend to be smaller perhaps or it may always hit the same spot and then stay there re-formatting for a large number of cycles.

I added explicit timing for LLformat and bigfile() - but my quick failed elapsed timer on command line never got addressed - if I get that working the goal was timing between user input on a number of iterations and I could test the Files in directory that may be more like standard logging.

[KurtE]

I think I have everything synced up. And built again. I also edited the example sketch that at end allows me to quickformat or lowlevelFormat or none...

Running on my Beta Faster board...

C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 25 2020 17:25:21
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes......................
C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 26 2020 06:54:59
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes.........................
Big write took 62.44 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 132.62 
Bytes Used: 8192, Bytes Total:16777216

C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 26 2020 06:59:44
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes.........................
Big write took 60.74 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 136.32 
Bytes Used: 8306688, Bytes Total:16777216

: q-quickformat, l-lowlevel format, else no format
Bytes Used: 8192, Bytes Total:16777216

C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 26 2020 06:59:44
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes.........................
Big write took 64.65 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 128.08 
Bytes Used: 8306688, Bytes Total:16777216

: q-quickformat, l-lowlevel format, else no format
.........................................................................................................................
Bytes Used: 8192, Bytes Total:16777216

C:\Users\kurte\Documents\Arduino\bar\bar.ino Nov 26 2020 06:59:44
Started QSPI_DISK

Bytes Used: 8192, Bytes Total:16777216


Start Big write of 8283312 Bytes.........................
Big write took 62.15 Sec for 8280000 Bytes : file3.size()=0
	Big write KBytes per second 133.23 
Bytes Used: 8306688, Bytes Total:16777216

: q-quickformat, l-lowlevel format, else no format

So it stays at about the 120-130+kb per second. Now to try again on New chip:
First with the older one I did, even after low level format about 78kb...
Dito for the new one...

Just to make sure same memory on each board, I update the littlefs to print out data about the identification of the chip...


Which you can see the details from screenshot of three TyCommander windows, where all three chips were programmed at the same time

It is almost like maybe the original chip had something run on it that converted it to work in Quad mode and the newer boards are still running ins single or dual pin SPI? Not sure if that
makes sense?