Teensyduino File System Integration, including MTP and MSC

Just a quick update, I tossed the N record cache code, figured it did not give enough payback for the overhead...

So started a different approach of grouping groups of items into blocks of records, which I read in and write out 2K chunks.
I also try to reduce the memory usage by only storing the actual characters needed to hold the name...

I limited the number of records to < 64 and keep in offset to each possible record in the start of the buffer...

I now have some of it starting to work. I have it that in theory you could stay with the non block code for low memory or choose how many 2k blocks to keep in memory at once... Right now I choose 4 (except t3.2)...
Still a few code places to fix:
I encode the Block and record into the object_id. So not all ids will be in use. That is I have 6 bits allocated for the record within block. I also reserve some spare space in each block to hopefully have room if file names get renamed...
...

So if I get up to lets say record 50 and I don't think I will have room to safely store 51, it will choose to then go to next block record 0...

Anyway did some rudimentary testing, and the interesting thing is I browsed through maybe 50+ files:

Code:
*** Storage Record Read/Write counts ***

    Writes: 69 to FS:0
    Reads: 909 from FS:0
Record Info Blocks (weight, block_index dirty):
  0: 678 0 1
  Count:54 NF: 1700
00000000 - C1 F3 CB 61 C1 F3 CB 61  02 00 16 00 00 00 02 03  : ...a...a ........
00000010 - 31 31 31 00 C0 F3 CB 61  C0 F3 CB 61 02 00 00 00  : 111....a ...a....
00000020 - 10 00 02 01 32 32 32 00  C1 F3 CB 61 C1 F3 CB 61  : ....222. ...a...a
00000030 - 02 00 00 00 11 00 02 01  33 33 33 00 7F 0A CC 61  : ........ 333....a
00000040 - 7F 0A CC 61 02 00 00 00  12 00 02 01 54 65 73 74  : ...a.... ....Test
00000050 - 44 69 72 00 C1 F3 CB 61  C1 F3 CB 61 10 00 00 01  : Dir....a ...a....
00000060 - 00 00 02 00 61 61 61 2E  74 78 74 00 C1 F3 CB 61  : ....aaa. txt....a
00000070 - C1 F3 CB 61 10 00 00 02  14 00 02 00 62 62 62 2E  : ...a.... ....bbb.
00000080 - 74 78 74 00 C1 F3 CB 61  C1 F3 CB 61 10 00 00 03  : txt....a ...a....
00000090 - 15 00 02 00 63 63 63 2E  74 78 74 00 98 1A AB 4B  : ....ccc. txt....K
000000A0 - 27 EB CE 61 03 00 C7 78  00 00 03 00 44 53 43 30  : '..a...x ....DSC0
000000B0 - 33 33 35 35 2E 4A 50 47  00 00 00 00 4B 7E C5 61  : 3355.JPG ....K~.a
000000C0 - 4B 7E C5 61 00 00 00 00  00 00 00 00 6D 74 70 69  : K~.a.... ....mtpi
000000D0 - 6E 64 65 78 2E 64 61 74  00 00 00 00 A0 C5 BE 61  : ndex.dat .......a
000000E0 - A0 C5 BE 61 05 00 24 00  00 00 05 03 32 32 32 00  : ...a..$. ....222.
000000F0 - A4 C5 BE 61 A4 C5 BE 61  05 00 27 00 19 00 05 03  : ...a...a ..'.....
00000100 - 33 33 33 00 00 00 00 00  72 6D C5 61 05 00 00 00  : 333..... rm.a....
00000110 - 1A 00 05 00 6D 74 70 69  6E 64 65 78 2E 64 61 74  : ....mtpi ndex.dat
00000120 - 00 00 00 00 18 7E C5 61  18 7E C5 61 05 00 21 00  : .....~.a .~.a..!.
00000130 - 1B 00 05 03 31 31 31 00  64 B4 C6 61 64 B4 C6 61  : ....111. d..ad..a
00000140 - 05 00 00 00 1C 00 05 01  41 46 6F 6C 64 65 72 00  : ........ AFolder.
00000150 - 58 70 DE 61 58 70 DE 61  05 00 59 00 1D 00 05 03  : Xp.aXp.a ..Y.....
00000160 - 50 69 63 74 75 72 65 73  00 00 00 00 5A C6 BC 61  : Pictures ....Z..a
00000170 - 18 7E C5 61 1C 00 00 01  00 00 05 00 61 61 61 2E  : .~.a.... ....aaa.
00000180 - 74 78 74 00 5A C6 BC 61  1A 7E C5 61 1C 00 00 02  : txt.Z..a .~.a....
00000190 - 1F 00 05 00 62 62 62 2E  74 78 74 00 5A C6 BC 61  : ....bbb. txt.Z..a
000001A0 - 1A 7E C5 61 1C 00 00 03  20 00 05 00 63 63 63 2E  : .~.a....  ...ccc.
000001B0 - 74 78 74 00 5A C6 BC 61  A0 C5 BE 61 19 00 00 01  : txt.Z..a ...a....
000001C0 - 00 00 05 00 61 61 61 2E  74 78 74 00 5A C6 BC 61  : ....aaa. txt.Z..a
000001D0 - A2 C5 BE 61 19 00 00 02  22 00 05 00 62 62 62 2E  : ...a.... "...bbb.
000001E0 - 74 78 74 00 5A C6 BC 61  A2 C5 BE 61 19 00 00 03  : txt.Z..a ...a....
000001F0 - 23 00 05 00 63 63 63 2E  74 78 74 00 5A C6 BC 61  : #...ccc. txt.Z..a
00000200 - A4 C5 BE 61 1A 00 00 01  00 00 05 00 61 61 61 2E  : ...a.... ....aaa.
00000210 - 74 78 74 00 5A C6 BC 61  A6 C5 BE 61 1A 00 00 02  : txt.Z..a ...a....
00000220 - 25 00 05 00 62 62 62 2E  74 78 74 00 5A C6 BC 61  : %...bbb. txt.Z..a
00000230 - A8 C5 BE 61 1A 00 00 03  26 00 05 00 63 63 63 2E  : ...a.... &...ccc.
00000240 - 74 78 74 00 52 4A 4C 61  80 70 DE 61 1E 00 02 F5  : txt.RJLa .p.a....
00000250 - 00 00 05 00 54 34 2E 31  2D 6D 75 78 2E 6A 70 67  : ....T4.1 -mux.jpg
00000260 - 00 00 00 00 02 4A 4C 61  84 70 DE 61 1E 00 09 01  : .....JLa .p.a....
00000270 - 28 00 05 00 54 34 2E 31  2D 43 61 72 64 6C 69 6B  : (...T4.1 -Cardlik
00000280 - 65 2E 6A 70 67 00 00 00  4E 34 FF 5F 88 70 DE 61  : e.jpg... N4._.p.a
00000290 - 1E 00 1E 32 29 00 05 00  54 34 2D 43 61 72 64 6C  : ...2)... T4-Cardl
000002A0 - 69 6B 65 2E 6A 70 67 00  D4 FB 7A 4E E6 70 DE 61  : ike.jpg. ..zN.p.a
000002B0 - 1E 00 C3 32 2A 00 05 00  41 70 6F 64 2D 70 72 65  : ...2*... Apod-pre
000002C0 - 76 69 65 77 2E 6A 70 67  00 00 00 00 EC 55 4F 55  : view.jpg .....UOU
000002D0 - E6 70 DE 61 1E 00 C2 32  2B 00 05 00 43 61 6D 65  : .p.a...2 +...Came
000002E0 - 72 61 2D 6F 6E 2D 62 72  61 63 6B 65 74 2E 6A 70  : ra-on-br acket.jp
000002F0 - 67 00 00 00 D6 F8 26 4F  E6 70 DE 61 1E 00 72 32  : g.....&O .p.a..r2
00000300 - 2C 00 05 00 42 41 50 2D  42 42 32 2E 6A 70 67 00  : ,...BAP- BB2.jpg.
00000310 - AA ED A2 52 E6 70 DE 61  1E 00 12 32 2D 00 05 00  : ...R.p.a ...2-...
00000320 - 4C 41 2D 50 6F 77 65 72  4F 4B 2E 6A 70 67 00 00  : LA-Power OK.jpg..
00000330 - 9E 4A 52 4E E6 70 DE 61  1E 00 8C 31 2E 00 05 00  : .JRN.p.a ...1....
00000340 - 41 52 2D 44 49 59 2D 34  2E 6A 70 67 00 00 00 00  : AR-DIY-4 .jpg....
00000350 - 62 CA 88 56 E6 70 DE 61  1E 00 E4 2D 2F 00 05 00  : b..V.p.a ...-/...
00000360 - 48 52 4F 53 31 2D 4E 65  6F 70 69 78 65 6C 2D 43  : HROS1-Ne opixel-C
00000370 - 68 65 73 74 2D 66 72 6F  6E 74 2D 2E 6A 70 67 00  : hest-fro nt-.jpg.
00000380 - D4 DA A9 4B E6 70 DE 61  1E 00 AE 2D 30 00 05 00  : ...K.p.a ...-0...
00000390 - 70 68 6F 65 6E 69 78 2D  72 6F 74 61 74 65 2D 52  : phoenix- rotate-R
000003A0 - 4A 6F 79 2D 4C 52 2E 6A  70 67 00 00 80 5E 26 5A  : Joy-LR.j pg...^&Z
000003B0 - E6 70 DE 61 1E 00 D5 2B  31 00 05 00 42 53 53 31  : .p.a...+ 1...BSS1
000003C0 - 33 38 43 54 2D 61 6E 64  2D 70 75 6C 6C 75 70 2E  : 38CT-and -pullup.
000003D0 - 6A 70 67 00 F2 D5 A2 4D  E6 70 DE 61 1E 00 21 29  : jpg....M .p.a..!)
000003E0 - 32 00 05 00 52 43 2D 69  6E 70 75 74 2D 74 69 6D  : 2...RC-i nput-tim
000003F0 - 69 6E 67 73 2E 6A 70 67  00 00 00 00 00 74 2D 5D  : ings.jpg .....t-]
00000400 - E6 70 DE 61 1E 00 9E 27  33 00 05 00 4B 65 44 65  : .p.a...' 3...KeDe
00000410 - 69 54 65 73 74 2E 6A 70  67 00 00 00 AC 6E 44 51  : iTest.jp g....nDQ
00000420 - E6 70 DE 61 1E 00 1E 27  34 00 05 00 53 68 6F 72  : .p.a...' 4...Shor
00000430 - 74 65 6E 65 64 2D 75 73  62 2E 6A 70 67 00 00 00  : tened-us b.jpg...
00000440 - 38 3C 52 4E E6 70 DE 61  1E 00 11 27 35 00 05 00  : 8<RN.p.a ...'5...
00000450 - 41 52 2D 44 49 59 2D 32  2E 6A 70 67 00 00 00 00  : AR-DIY-2 .jpg....
00000460 - 40 DB A9 4B E6 70 DE 61  1E 00 CF 26 36 00 05 00  : @..K.p.a ...&6...
00000470 - 43 68 72 33 2D 72 6F 74  61 74 65 2D 52 4A 6F 79  : Chr3-rot ate-RJoy
00000480 - 2D 4C 52 2E 6A 70 67 00  EA A7 66 56 E6 70 DE 61  : -LR.jpg. ..fV.p.a
00000490 - 1E 00 E4 25 37 00 05 00  50 68 61 6E 74 6F 6D 58  : ...%7... PhantomX
000004A0 - 2D 51 75 61 64 2D 41 72  62 6F 74 69 78 2D 70 6C  : -Quad-Ar botix-pl
000004B0 - 61 74 65 2E 6A 70 67 00  B8 88 84 4F E6 70 DE 61  : ate.jpg. ...O.p.a
000004C0 - 1E 00 0A 25 38 00 05 00  44 69 70 54 72 61 63 65  : ...%8... DipTrace
000004D0 - 2D 53 63 68 65 6D 61 74  69 63 2D 2D 2D 50 53 32  : -Schemat ic---PS2
000004E0 - 2D 56 6F 2E 6A 70 67 00  D8 D1 48 4E E8 70 DE 61  : -Vo.jpg. ..HN.p.a
000004F0 - 1E 00 74 24 39 00 05 00  41 72 64 75 69 6E 6F 2D  : ..t$9... Arduino-
00000500 - 52 65 6D 6F 74 65 2D 62  6F 78 32 2E 6A 70 67 00  : Remote-b ox2.jpg.
00000510 - 4E 5F BA 4E E8 70 DE 61  1E 00 04 21 3A 00 05 00  : N_.N.p.a ...!:...
00000520 - 41 52 2D 44 49 59 2D 31  30 2E 6A 70 67 00 00 00  : AR-DIY-1 0.jpg...
00000530 - 30 76 94 52 E8 70 DE 61  1E 00 90 20 3B 00 05 00  : 0v.R.p.a ... ;...
00000540 - 54 41 2D 55 53 42 32 41  58 2D 31 2E 6A 70 67 00  : TA-USB2A X-1.jpg.
00000550 - 1A F4 2A 4D E8 70 DE 61  1E 00 19 20 3C 00 05 00  : ..*M.p.a ... <...
00000560 - 50 73 32 2D 77 69 74 68  2D 63 61 62 6C 65 2E 6A  : Ps2-with -cable.j
00000570 - 70 67 00 00 BA B4 DA 4C  E8 70 DE 61 1E 00 0F 20  : pg.....L .p.a... 
00000580 - 3D 00 05 00 62 65 6C 6B  69 6E 2D 70 6C 61 79 2D  : =...belk in-play-
00000590 - 73 65 72 69 61 6C 2D 32  2E 6A 70 67 00 00 00 00  : serial-2 .jpg....
000005A0 - 70 63 12 5B E8 70 DE 61  1E 00 4B 1F 3E 00 05 00  : pc.[.p.a ..K.>...
000005B0 - 49 4D 47 5F 30 33 33 36  2D 28 30 30 32 29 2E 6A  : IMG_0336 -(002).j
000005C0 - 70 67 00 00 C6 39 17 4E  E8 70 DE 61 1E 00 2F 1E  : pg...9.N .p.a../.
000005D0 - 3F 00 05 00 68 61 6E 64  2E 6A 70 67 00 00 00 00  : ?...hand .jpg....
000005E0 - 2C D1 B5 55 E8 70 DE 61  1E 00 DD 1D 40 00 05 00  : ,..U.p.a ....@...
000005F0 - 6F 64 72 6F 69 64 2D 78  75 33 2D 62 65 66 6F 72  : odroid-x u3-befor
00000600 - 65 2D 70 6F 77 65 72 6F  66 66 2E 6A 70 67 00 00  : e-powero ff.jpg..
00000610 - 28 C4 28 5D E8 70 DE 61  1E 00 42 1D 41 00 05 00  : (.(].p.a ..B.A...
00000620 - 50 57 4D 2D 73 6F 75 6E  64 2D 6F 75 74 2E 6A 70  : PWM-soun d-out.jp
00000630 - 67 00 00 00 36 6F 49 5E  E8 70 DE 61 1E 00 8A 1C  : g...6oI^ .p.a....
00000640 - 42 00 05 00 41 6E 6E 69  65 5F 68 65 61 64 31 31  : B...Anni e_head11
00000650 - 30 78 31 31 30 2E 6A 70  67 00 00 00 30 AE F6 54  : 0x110.jp g...0..T
00000660 - E8 70 DE 61 1E 00 54 1A  43 00 05 00 73 6F 75 6E  : .p.a..T. C...soun
00000670 - 64 2D 63 69 72 63 75 69  74 2E 6A 70 67 00 00 00  : d-circui t.jpg...
00000680 - 7A 58 2B 5D E8 70 DE 61  1E 00 24 1A 44 00 05 00  : zX+].p.a ..$.D...
00000690 - 49 4D 47 5F 30 37 39 36  2D 28 30 30 32 29 2E 6A  : IMG_0796 -(002).j
000006A0 - 70 67 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : pg...... ........
000006B0 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
...	 10 duplicate line(s) removed.
00000760 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
00000770 - 00 00 00 00 00 00 00 00  00 00 00 00 00           : ........ .....
0(0):0: 2 1 1 2 0 22 1640756161 1640756161 111
1(14):1: 2 1 0 2 16 0 1640756160 1640756160 222
2(28):2: 2 1 0 2 17 0 1640756161 1640756161 333
3(3c):3: 2 1 0 2 18 0 1640761983 1640761983 TestDir
4(54):4: 2 0 0 16 0 256 1640756161 1640756161 aaa.txt
5(6c):5: 2 0 0 16 20 512 1640756161 1640756161 bbb.txt
6(84):6: 2 0 0 16 21 768 1640756161 1640756161 ccc.txt
7(9c):7: 3 0 0 3 0 30919 1269504664 1640950567 DSC03355.JPG
8(bc):8: 0 0 0 0 0 0 1640332875 1640332875 mtpindex.dat
9(dc):9: 5 1 1 5 0 36 1639892384 1639892384 222
10(f0):10: 5 1 1 5 25 39 1639892388 1639892388 333
11(104):11: 5 0 0 5 26 0 0 1640328562 mtpindex.dat
12(124):12: 5 1 1 5 27 33 1640332824 1640332824 111
13(138):13: 5 1 0 5 28 0 1640412260 1640412260 AFolder
14(150):14: 5 1 1 5 29 89 1641967704 1641967704 Pictures
15(16c):15: 5 0 0 28 0 256 1639761498 1640332824 aaa.txt
16(184):16: 5 0 0 28 31 512 1639761498 1640332826 bbb.txt
17(19c):17: 5 0 0 28 32 768 1639761498 1640332826 ccc.txt
18(1b4):18: 5 0 0 25 0 256 1639761498 1639892384 aaa.txt
19(1cc):19: 5 0 0 25 34 512 1639761498 1639892386 bbb.txt
20(1e4):20: 5 0 0 25 35 768 1639761498 1639892386 ccc.txt
21(1fc):21: 5 0 0 26 0 256 1639761498 1639892388 aaa.txt
22(214):22: 5 0 0 26 37 512 1639761498 1639892390 bbb.txt
23(22c):23: 5 0 0 26 38 768 1639761498 1639892392 ccc.txt
24(244):24: 5 0 0 30 0 62722 1632389714 1641967744 T4.1-mux.jpg
25(264):25: 5 0 0 30 40 265 1632389634 1641967748 T4.1-Cardlike.jpg
26(288):26: 5 0 0 30 41 12830 1610560590 1641967752 T4-Cardlike.jpg
27(2a8):27: 5 0 0 30 42 12995 1316682708 1641967846 Apod-preview.jpg
28(2cc):28: 5 0 0 30 43 12994 1431262700 1641967846 Camera-on-bracket.jpg
29(2f4):29: 5 0 0 30 44 12914 1327954134 1641967846 BAP-BB2.jpg
30(310):30: 5 0 0 30 45 12818 1386409386 1641967846 LA-PowerOK.jpg
31(330):31: 5 0 0 30 46 12684 1314015902 1641967846 AR-DIY-4.jpg
32(350):32: 5 0 0 30 47 11748 1451805282 1641967846 HROS1-Neopixel-Chest-front-.jpg
33(380):33: 5 0 0 30 48 11694 1269422804 1641967846 phoenix-rotate-RJoy-LR.jpg
34(3ac):34: 5 0 0 30 49 11221 1512464000 1641967846 BSS138CT-and-pullup.jpg
35(3d4):35: 5 0 0 30 50 10529 1302517234 1641967846 RC-input-timings.jpg
36(3fc):36: 5 0 0 30 51 10142 1563259904 1641967846 KeDeiTest.jpg
37(41c):37: 5 0 0 30 52 10014 1363439276 1641967846 Shortened-usb.jpg
38(440):38: 5 0 0 30 53 10001 1314012216 1641967846 AR-DIY-2.jpg
39(460):39: 5 0 0 30 54 9935 1269422912 1641967846 Chr3-rotate-RJoy-LR.jpg
40(488):40: 5 0 0 30 55 9700 1449568234 1641967846 PhantomX-Quad-Arbotix-plate.jpg
41(4b8):41: 5 0 0 30 56 9482 1334085816 1641967846 DipTrace-Schematic---PS2-Vo.jpg
42(4e8):42: 5 0 0 30 57 9332 1313395160 1641967848 Arduino-Remote-box2.jpg
43(510):43: 5 0 0 30 58 8452 1320836942 1641967848 AR-DIY-10.jpg
44(530):44: 5 0 0 30 59 8336 1385461296 1641967848 TA-USB2AX-1.jpg
45(550):45: 5 0 0 30 60 8217 1294660634 1641967848 Ps2-with-cable.jpg
46(574):46: 5 0 0 30 61 8207 1289401530 1641967848 belkin-play-serial-2.jpg
47(5a0):47: 5 0 0 30 62 8011 1527931760 1641967848 IMG_0336-(002).jpg
48(5c4):48: 5 0 0 30 63 7727 1310144966 1641967848 hand.jpg
49(5e0):49: 5 0 0 30 64 7645 1437978924 1641967848 odroid-xu3-before-poweroff.jpg
50(610):50: 5 0 0 30 65 7490 1562952744 1641967848 PWM-sound-out.jpg
51(634):51: 5 0 0 30 66 7306 1581870902 1641967848 Annie_head110x110.jpg
52(65c):52: 5 0 0 30 67 6740 1425452592 1641967848 sound-circuit.jpg
53(680):53: 5 0 0 30 68 6692 1563121786 1641967848 IMG_0796-(002).jpg
-----------------------------------------------
  1: -275 1 1
  Count:10 NF: 356
00000000 - 3E 6E D6 53 E8 70 DE 61  1E 00 5C 19 45 00 05 00  : >n.S.p.a ..\.E...
00000010 - 41 72 64 75 69 6E 6F 2D  48 65 61 64 65 72 2D 48  : Arduino- Header-H
00000020 - 65 69 67 68 74 73 2E 6A  70 67 00 00 A8 63 12 5B  : eights.j pg...c.[
00000030 - E8 70 DE 61 1E 00 E1 16  50 00 05 00 49 4D 47 5F  : .p.a.... P...IMG_
00000040 - 30 33 33 37 2D 28 30 30  32 29 2E 6A 70 67 00 00  : 0337-(00 2).jpg..
00000050 - 4E A7 B7 5E E8 70 DE 61  1E 00 51 16 51 00 05 00  : N..^.p.a ..Q.Q...
00000060 - 74 65 65 6E 73 79 34 30  5F 62 61 63 6B 5F 73 6D  : teensy40 _back_sm
00000070 - 64 2E 6A 70 67 00 00 00  FA 19 C8 4B E8 70 DE 61  : d.jpg... ...K.p.a
00000080 - 1E 00 FD 15 52 00 05 00  4D 69 6E 6F 72 2D 4E 69  : ....R... Minor-Ni
00000090 - 74 2D 6F 6E 2D 73 74 61  72 74 75 70 2E 6A 70 67  : t-on-sta rtup.jpg
000000A0 - 00 00 00 00 FA C4 44 5F  98 73 DE 61 1E 00 90 0A  : ......D_ .s.a....
000000B0 - 53 00 05 00 74 65 65 6E  73 79 34 31 5F 42 50 2E  : S...teen sy41_BP.
000000C0 - 6A 70 67 00 BA 73 DE 61  BA 73 DE 61 1E 00 00 00  : jpg..s.a .s.a....
000000D0 - 54 00 05 01 31 30 31 4D  53 44 43 46 00 00 00 00  : T...101M SDCF....
000000E0 - F2 EF 93 55 CA 73 DE 61  1E 00 28 06 55 00 05 00  : ...U.s.a ..(.U...
000000F0 - 58 55 33 44 2D 42 41 44  2D 54 4F 2E 6A 70 67 00  : XU3D-BAD -TO.jpg.
00000100 - 3C BF D6 50 02 74 DE 61  1E 00 97 F7 56 00 05 00  : <..P.t.a ....V...
00000110 - 41 6E 74 31 2E 6A 70 67  00 00 00 00 22 E4 CF 5E  : Ant1.jpg ...."..^
00000120 - 08 74 DE 61 1E 00 8F FC  57 00 05 00 49 4D 47 5F  : .t.a.... W...IMG_
00000130 - 31 31 32 37 2E 6A 70 67  00 00 00 00 CC 36 BA 56  : 1127.jpg .....6.V
00000140 - 0C 74 DE 61 1E 00 08 FC  58 00 05 00 4C 41 2D 58  : .t.a.... X...LA-X
00000150 - 75 33 49 2D 41 58 42 55  53 2D 41 50 2E 6A 70 67  : u3I-AXBU S-AP.jpg
00000160 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
...	 95 duplicate line(s) removed.
00000760 - 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  : ........ ........
00000770 - 00 00 00 00 00 00 00 00  00 00 00 00 00           : ........ .....
0(0):0: 5 0 0 30 69 6492 1406561854 1641967848 Arduino-Header-Heights.jpg
1(2c):1: 5 0 0 30 80 5857 1527931816 1641967848 IMG_0337-(002).jpg
2(50):2: 5 0 0 30 81 5713 1589094222 1641967848 teensy40_back_smd.jpg
3(78):3: 5 0 0 30 82 5629 1271405050 1641967848 Minor-Nit-on-startup.jpg
4(a4):4: 5 0 0 30 83 2704 1598342394 1641968536 teensy41_BP.jpg
5(c4):5: 5 1 0 30 84 0 1641968570 1641968570 101MSDCF
6(e0):6: 5 0 0 30 85 1576 1435758578 1641968586 XU3D-BAD-TO.jpg
7(100):7: 5 0 0 30 86 63383 1356250940 1641968642 Ant1.jpg
8(11c):8: 5 0 0 30 87 64655 1590682658 1641968648 IMG_1127.jpg
9(13c):9: 5 0 0 30 88 64520 1455044300 1641968652 LA-Xu3I-AXBUS-AP.jpg
-----------------------------------------------
  2: 0 65535 0
  3: 0 65535 0

I added more debug output here to look at the index/records...


And the interesting thing is these 50 plus files fit into the first 2 2k buffers and in fact it only used about 350 bytes into 2nd buffer...
And There were no writes or reads to the underlying file

Again still more things to fix and test... Up in the cache_storage_records branch...
 
I just checked Mouser and Digikey, both are out of stock on the W25N02KVZEIR NAND. So can not even get one so I can compare apples to apples. But in lieu of the N02 I used a M02 NAND 2Gbit Flash and reran the SPI integrity sketch - all tests passed.

...
EDTI: Tried to copy the ManyF/333 directory to the N02 chip but started in again with files not being written as described before. However, this time when I copied the 73Mbyte music directory to the N02 it worked no issues and was able to play wav and mp3 files directly off the chip without issue.

So probably going to have to reduce the clock speed down to 45Mhz - now the question is why the delta between the T4.1 and TMM.

... Morning full of chores into afternoon - and another ahead - woodstove not even started yet ...

Will look to test on T_MM on return.

Test: Current main T_MTP, T_MM or T_4.1 with PJRC 2GB NAND on CS==4 {// Tested NOR 64MB on #5, #6 : NAND 1Gb on #3, 2GB on #4}, other details prior post?
> the 2G chip on that was a pre-release sample? Marking here seems as: 25M02GV
 
Just a quick update, I tossed the N record cache code, figured it did not give enough payback for the overhead...

So started a different approach of grouping groups of items into blocks of records, which I read in and write out 2K chunks.
I also try to reduce the memory usage by only storing the actual characters needed to hold the name...

...

Again still more things to fix and test... Up in the cache_storage_records branch...

Bummer the work didn't seem to have good payoff - I only got a quick glance at some git updates and never enough to follow.

Better luck with the next effort. Hopefully I'll get to look into it.

Just home after 2 hour visit, in time for supper - need to set up to test the spi flash a bit to see if what Mike is seeing is just his chip or something odd... then add the Verify file parsing.
 
Using the Integrity SPI.INO - I cannot get TEST_SPI or TEST_SPI_NAND chips to show anything but "Error starting SPI_?????" for CS of 3,4,5,, or 6 with the PJRC 4X SPI breakout and T_4.1 ???

Was there a breaking CORES change post td1.56? Or some other update not in place? Little_FS seems current ...
 
Using the Integrity SPI.INO - I cannot get TEST_SPI or TEST_SPI_NAND chips to show anything but "Error starting SPI_?????" for CS of 3,4,5,, or 6 with the PJRC 4X SPI breakout and T_4.1 ???

Was there a breaking CORES change post td1.56? Or some other update not in place? Little_FS seems current ...

Strange - I have been using it to test with without any problems.
 
Bummer the work didn't seem to have good payoff - I only got a quick glance at some git updates and never enough to follow.

Better luck with the next effort. Hopefully I'll get to look into it.

Thanks, I am not sad to abandon the simple cache of lets say 8 items. Yes it did help, but I still not like using all of the space, and have that space so sparsely used. That is for an 8 record buffer 8*280 > 2K bytes.
With the packing into blocks I may instead of holding 8 records, I may be holding something like 50 and right now with 4 blocks on a T4.x that means I might have 200 in memory and in probably a high percentage case that may be all of the records that will be used in majority of runs...

There are lots of tweaks that could happen here like, could start off with none of the blocks allocated. Currently fixed memory (DMAMEM), but could be array of pointers, could malloc or extemem alloc them when needed up to a defined maximum... Could shrink/free the list of allocated ones if sketch desires it... example if no mtp operation has happened for 10 minutes, maybe sketch could say released some/all buffers....

Time for some more playing ;)
 
Good Morning all
Got a bit tired of remembering what chips I have soldered onto memory boards or what flash I have on QSPI so as a prelude to incorporating a new function to get the part number we put together a little sketch to get Flash and FRAM manufacturers part numbers:


Which gives the following output for one of my breakout boards.

Code:
flash begin

------------------------------------
Look at Pin: 3
NOR Flash ID: 1F 84 01
NOR Flash size is 0.50 Mbyte
NOR Part Number: AT25SF041

------------------------------------
Look at Pin: 4
NOR Flash ID: EF 40 17
NOR Flash size is 8.00 Mbyte
NOR Part Number: W25Q64JV*Q/W25Q64FV

------------------------------------
Look at Pin: 5
NOR Flash ID: EF 80 19
NOR Flash size is 32.00 Mbyte
NOR Part Number: W25Q256JW*M

------------------------------------
Look at Pin: 6

------------------------------------
Look at Pin: 6
Not found in list !!!!

------------------------------------
Look at Pin: 7
NOR Flash ID: EF 80 19
NOR Flash size is 32.00 Mbyte
NOR Part Number: W25Q256JW*M

I do have a branch of a modified version of LittleFS so you can call getPN() directly from a sketch: https://github.com/mjs513/LittleFS/tree/PartNumber

Simple example would be:
Code:
#include <LittleFS.h>
LittleFS_SPIFlash myfs;

uint8_t chipSelect = 3;

void setup() {
  Serial.begin(9600);
  delay(5000);
  if(!myfs.begin(chipSelect)) {
    Serial.println("SPI FLASH Failed to start");
  } else {
    const char *pn = myfs.getPN();
    Serial.printf("Device on pin %d is a %s\n", chipSelect, pn);
  }
  
}

void loop() {

}
 

Attachments

  • LFS_PN_Test.zip
    2.2 KB · Views: 24
Good Morning all
Got a bit tired of remembering what chips I have soldered onto memory boards or what flash I have on QSPI so as a prelude to incorporating a new function to get the part number we put together a little sketch to get Flash and FRAM manufacturers part numbers:
Me too, especially when I have a few different versions soldered up....

Thanks
 
Me too, especially when I have a few different versions soldered up....

Thanks

Thanks we have been talking about this for a while now so figured it was about time. I just cleaned the sketch up a bit and corrected a minor error when getting FRAM pn's so attaching the updated sketch:

Code:
flash begin

------------------------------------
Look at Pin: 3
FRAM Flash ID: AE 83 09
FRAM size is 0.12 Mbyte
FRAM Part Number: MR45V100A

------------------------------------
Look at Pin: 4
FRAM Flash ID: 03 2E C2
FRAM size is 1.00 Mbyte
FRAM Part Number: CY15B108QN

------------------------------------
Look at Pin: 5
NAND Flash ID: EF BB 21
NAND Flash size is 253.00 Mbyte
NAND Flash ID: EF BB 21
NAND Part Number: W25M02

------------------------------------
Look at Pin: 6
Not found in list !!!!

------------------------------------
Look at Pin: 7
Not found in list !!!!
 

Attachments

  • LFS_PN_Test.zip
    2.3 KB · Views: 22
Good tool - Good work Mike! I don't even have the right 4X board in place I see :(

Found PJRC board and SPI.ino is WORKING ... will T_MM test next.

Code:
[B]Here is edited  [ATTACH]27206._xfImport[/ATTACH][/B]

Did a quick EDIT to detect when PCB has an active PCB PullUp on CS pin. This will differentiate 3,4,5,6 boards from 3,4,5,7 boards.
Code:
flash begin
	No PCB Pullup on Pin #6

------------------------------------
Look at Pin: 3
NOR Flash ID: EF 40 18
NOR Flash size is 16.00 Mbyte
NOR Part Number: W25Q128JV*Q/W25Q128FV

------------------------------------
Look at Pin: 4
NOR Flash ID: EF 70 20
NOR Flash size is 64.00 Mbyte
NOR Part Number: W25Q512JV*M (DTR)

------------------------------------
Look at Pin: 5
Not found in list !!!!

------------------------------------
Look at Pin: 6
NO PCB PULLUP: Not found in list !!!!

------------------------------------
Look at Pin: 7
FRAM Flash ID: AE 83 09
FRAM size is 0.12 Mbyte
FRAM Part Number: MR45V100A

Found the PJRC board:
Code:
flash begin
	No PCB Pullup on Pin #7

------------------------------------
Look at Pin: 3
NAND Flash ID: EF AA 21
NAND Flash size is 125.50 Mbyte
NAND Flash ID: EF AA 21
NAND Part Number: W25N01GVZEIG

------------------------------------
Look at Pin: 4
NAND Flash ID: EF AA 22
NAND Flash size is 253.00 Mbyte
NAND Flash ID: EF AA 22
NAND Part Number: W25N02KVZEIR

------------------------------------
Look at Pin: 5
NOR Flash ID: EF 70 20
NOR Flash size is 64.00 Mbyte
NOR Part Number: W25Q512JV*M (DTR)

------------------------------------
Look at Pin: 6
NOR Flash ID: EF 40 20
NOR Flash size is 64.00 Mbyte
NOR Part Number: W25Q512JV*Q

------------------------------------
Look at Pin: 7
NO PCB PULLUP: Not found in list !!!!

This is a one chip FRAM on 3,4,5,7 PCB with Audio board:
Code:
------------------------------------
Look at Pin: 3
FRAM Flash ID: AE 83 09
FRAM size is 0.12 Mbyte
FRAM Part Number: MR45V100A

------------------------------------
Look at Pin: 4
Not found in list !!!!

------------------------------------
Look at Pin: 5
Not found in list !!!!

------------------------------------
Look at Pin: 6
NOR Flash ID: EF 40 18
NOR Flash size is 16.00 Mbyte
NOR Part Number: W25Q128JV*Q/W25Q128FV

------------------------------------
Look at Pin: 7
Not found in list !!!!
 
Yep having some fun...

Mike, Paul, ... I did a PR back to your PartNumber branch with some edits to make it a virtual function to LittleFS added it to the wrapper classes, plus moved all strings to Flash.

I also hacked up a mtp test sketch with some real hacks... But fun..

Code:
const char *getFSPN(uint32_t ii) {
  FS* pfs = MTP.storage()->getStoreFS(ii);
  // total set of hacks...
  if (pfs == (FS *)&lfsram) return lfsram.getPN();
  if (pfs == (FS *)&lfsProg) return lfsProg.getPN();
  #ifdef ARDUINO_TEENSY41
  if (pfs == (FS *)&lfsqspi) return lfsqspi.getPN();
  #endif
  for (uint8_t i = 0; i < CLFSSPIPINS; i++) {
    if (pfs == (FS *)&lfsspi[i]) return lfsspi[i].getPN();
  }
  return "";
}

void loop() {
  if (DBGSerial.available()) {
    uint8_t command = DBGSerial.read();
    int ch = DBGSerial.read();

    switch (command) {
    case '1': {
      // first dump list of storages:
      uint32_t fsCount = MTP.storage()->getFSCount();
      DBGSerial.printf("\nDump Storage list(%u)\n", fsCount);
      for (uint32_t ii = 0; ii < fsCount; ii++) {
        DBGSerial.printf("store:%u storage:%x name:%s fs:%x pn:", ii,
                         MTP.Store2Storage(ii), MTP.storage()->getStoreName(ii),
                         (uint32_t)MTP.storage()->getStoreFS(ii));
        Serial.flush();        
        DBGSerial.println(getFSPN(ii));
      }
      DBGSerial.println("\nDump Index List");
      MTP.storage()->dumpIndexList();
    } break;

And test run output:
Code:
Dump Storage list(7)
store:0 storage:10001 name:Program fs:200059f0 pn:PROGRAM
store:1 storage:20001 name:RAM fs:20005abc pn:DMAMEM
store:2 storage:30001 name:Flash_3 fs:20005b88 pn:W25Q128JV*Q/W25Q128FV
store:3 storage:40001 name:NAND_4 fs:20005e50 pn:W25N01GVZEIG
store:4 storage:50001 name:Flash_5 fs:20006118 pn:W25Q512JV*M (DTR)
store:5 storage:60001 name:SD_Builtin fs:20006cbc pn:
store:6 storage:70001 name:SD_SPI fs:2000718c pn:

Dump Index List

I think it would be great to get the littlefs stuff merged back to main ;)
 
I just checked Mouser and Digikey, both are out of stock on the W25N02KVZEIR NAND. So can not even get one so I can compare apples to apples.

I have 2 of those W25N02KVZEIR chips left. You're welcome to have them both. Just let me know how you want them... soldered to boards or loose in a bag?
 
I have 2 of those W25N02KVZEIR chips left. You're welcome to have them both. Just let me know how you want them... soldered to boards or loose in a bag?

Thanks Paul - while my soldering has gotten better feel better if its not too much trouble if you can solder if you have an extra board. If not I have an extra board I can use.
 
I made a couple commits to fix issues that come up with the Android File Transfer app on MacOS. Files can now be copied from Mac to Teensy.
 
Mike/All: I put 4XSPI on PJRC_MM - with SPI.ino and it came up:
<EDIT>: Sanity check unzipped 1.8.16 and installed TD 1.56 to fresh folder - Get the SAME behavior on that 2Gbit SPI chip on T_MM+pjrcBreakout with integrity SPI.INO CS==4 - notes at bottom
>> Second note same board and code on T_4.1 does not at any time lose access to the 2Gbit NAND: Program, Power Off/On restart, repeat
>> Then put that 4XSPI onto prior programmed T_MM and: Error starting SPI_NAND

Code:
LittleFS Test : File Integrity
printDirectory SPI_NAND
--------------
	>>>	>>>>> No Dir
 Total 0 files of Size 0 Bytes
Bytes Used: 265289728, Bytes Total:265289728


	WARNING: DISK FULL >>>>>  Bytes Used: 265289728, Bytes Total:265289728

Same sketch compiled T_4.1 with same board then moved comes up:
Code:
C:\T_Drive\arduino-1.8.19\hardware\teensy\avr\libraries\LittleFS\examples\Test_Integrity\SPI\SPI.ino Jan 13 2022 18:14:28
LittleFS Test : File Integrity
printDirectory SPI_NAND
--------------
DIR	1_dir / 
	FILE	D_file.txt		3560
	FILE	E_file.txt		4072

 0 dirs with 2 files of Size 7632 Bytes
DIR	2_dir / 
	FILE	A_file.txt		7144
	FILE	B_file.txt		2536

 0 dirs with 2 files of Size 9680 Bytes
DIR	3_dir / 
	FILE	C_file.txt		3048
	FILE	D_file.txt		3560

 0 dirs with 2 files of Size 6608 Bytes
DIR	4_dir / 
	FILE	A_file.txt		14288
	FILE	B_file.txt		5072
	FILE	C_file.txt		6096
	FILE	D_file.txt		7120
	FILE	E_file.txt		8144

 0 dirs with 5 files of Size 40720 Bytes
DIR	5_dir / 
	FILE	B_file.txt		2536
	FILE	C_file.txt		3048

 0 dirs with 2 files of Size 5584 Bytes
FILE	B_file.txt		2536
FILE	C_file.txt		3048

 5 dirs with 2 files of Size 5584 Bytes
 Total 15 files of Size 75808 Bytes
Bytes Used: 3538944, Bytes Total:265289728

Put 4XSPI back on PJRC_MM_breakout and this time:
Code:
LittleFS Test : File Integrity
Error starting SPI_NAND
Only Diff is First was after programming, this second was fresh power up.

Then back on T_4.1 fresh power up:
Code:
LittleFS Test : File Integrity
printDirectory SPI_NAND
--------------

 0 dirs with 0 files of Size 0 Bytes
 Total 0 files of Size 0 Bytes
Bytes Used: 262144, Bytes Total:265289728

On IDE 1.8.16 and TD 1.56 to assure it wasn't some other CORES or other changes breaking this it acted the same and these installed LIBS - nothing custom:
Code:
Using library LittleFS at version 1.0.0 in folder: C:\T_Drive\arduino-1.8.16\hardware\teensy\avr\libraries\LittleFS 
Using library SPI at version 1.0 in folder: C:\T_Drive\arduino-1.8.16\hardware\teensy\avr\libraries\SPI 
Using library SdFat at version 2.1.0 in folder: C:\T_Drive\arduino-1.8.16\hardware\teensy\avr\libraries\SdFat
 
Last edited:
Yep having some fun...

Mike, Paul, ... I did a PR back to your PartNumber branch with some edits to make it a virtual function to LittleFS added it to the wrapper classes, plus moved all strings to Flash.

I also hacked up a mtp test sketch with some real hacks... But fun..

Code:
const char *getFSPN(uint32_t ii) {
  FS* pfs = MTP.storage()->getStoreFS(ii);
  // total set of hacks...
  if (pfs == (FS *)&lfsram) return lfsram.getPN();
  if (pfs == (FS *)&lfsProg) return lfsProg.getPN();
  #ifdef ARDUINO_TEENSY41
  if (pfs == (FS *)&lfsqspi) return lfsqspi.getPN();
  #endif
  for (uint8_t i = 0; i < CLFSSPIPINS; i++) {
    if (pfs == (FS *)&lfsspi[i]) return lfsspi[i].getPN();
  }
  return "";
}

void loop() {
  if (DBGSerial.available()) {
    uint8_t command = DBGSerial.read();
    int ch = DBGSerial.read();

    switch (command) {
    case '1': {
      // first dump list of storages:
      uint32_t fsCount = MTP.storage()->getFSCount();
      DBGSerial.printf("\nDump Storage list(%u)\n", fsCount);
      for (uint32_t ii = 0; ii < fsCount; ii++) {
        DBGSerial.printf("store:%u storage:%x name:%s fs:%x pn:", ii,
                         MTP.Store2Storage(ii), MTP.storage()->getStoreName(ii),
                         (uint32_t)MTP.storage()->getStoreFS(ii));
        Serial.flush();        
        DBGSerial.println(getFSPN(ii));
      }
      DBGSerial.println("\nDump Index List");
      MTP.storage()->dumpIndexList();
    } break;

And test run output:
Code:
Dump Storage list(7)
store:0 storage:10001 name:Program fs:200059f0 pn:PROGRAM
store:1 storage:20001 name:RAM fs:20005abc pn:DMAMEM
store:2 storage:30001 name:Flash_3 fs:20005b88 pn:W25Q128JV*Q/W25Q128FV
store:3 storage:40001 name:NAND_4 fs:20005e50 pn:W25N01GVZEIG
store:4 storage:50001 name:Flash_5 fs:20006118 pn:W25Q512JV*M (DTR)
store:5 storage:60001 name:SD_Builtin fs:20006cbc pn:
store:6 storage:70001 name:SD_SPI fs:2000718c pn:

Dump Index List

I think it would be great to get the littlefs stuff merged back to main ;)

All
Just hacked up my wacky MTP_TEST_INTEGRITY and with the changes to LFS and @KurtE's function above which I hacked up further to cover FRAM's as well:
Code:
Dump Storage list(8)
store:0 storage:10001 name:sdio fs:2000bf40 pn:
store:1 storage:20001 name:RAM0 fs:2000bd48 pn:EXTMEM
store:2 storage:30001 name:RAM1 fs:2000be14 pn:EXTMEM
store:3 storage:40001 name:PROGM fs:2000bba4 pn:PROGRAM
store:4 storage:50001 name:QSPI fs:2000bc70 pn:W25Q128JV*M (DTR)
store:5 storage:60001 name:Adesto fs:2000c410 pn:AT25SF041
store:6 storage:70001 name:Q64 fs:2000c4ec pn:W25Q64JV*Q/W25Q64FV
store:7 storage:80001 name:Q256JWIM fs:2000c5c8 pn:W25Q256JW*M
The dump is from a second memory board that I soldered up.
 
defragster said:
Mike/All: I put 4XSPI on PJRC_MM - with SPI.ino and it came up:
Code:
LittleFS Test : File Integrity
printDirectory SPI_NAND
--------------
>>> >>>>> No Dir
Total 0 files of Size 0 Bytes
Bytes Used: 265289728, Bytes Total:265289728


WARNING: DISK FULL >>>>> Bytes Used: 265289728, Bytes Total:265289728
Same sketch compiled T_4.1 with same board then moved comes up:
Tim try changing the SPI clock from 55Mhz to 45Mhz in littleFS_NAND.cpp - think that change works for the TMM but don't understand why have to slow the speed down for the TMM?
 
Tim try changing the SPI clock from 55Mhz to 45Mhz in littleFS_NAND.cpp - think that change works for the TMM but don't understand why have to slow the speed down for the TMM?

Oppps - was editing old post without refresh : THIS WORKS

Code:
//#define SPICONFIG_NAND   SPISettings(55000000, MSBFIRST, SPI_MODE0)
#define SPICONFIG_NAND   SPISettings(45000000, MSBFIRST, SPI_MODE0)

At least on the PJRC_MM_breakout - hand wired 3" to SFun ATP and it didn't work there (matching your results Mike) if it was tested right
 
Oppps - was editing old post without refresh : THIS WORKS

Code:
//#define SPICONFIG_NAND   SPISettings(55000000, MSBFIRST, SPI_MODE0)
#define SPICONFIG_NAND   SPISettings(45000000, MSBFIRST, SPI_MODE0)

At least on the PJRC_MM_breakout - hand wired 3" to SFun ATP and it didn't work there (matching your results Mike) if it was tested right

Thanks for checking Tim but we may have to reduce the 45Mhz to 30Mhz. Have to run some more tests

EDIT: If you reduce the SPI Clock further to 30Mhz
1. Still seeing failure to copy some of the ManyF files to the Win2G so no help there
2. Using a MM ATP Carrier recognizes the chips if you hardwire the memory board to the carrier.
Code:
Dump Storage list(5)
store:0 storage:10001 name:sdio fs:2000beb4 pn:
store:1 storage:20001 name:PROGM fs:2000bd88 pn:PROGRAM
store:2 storage:30001 name:sflash5 fs:2000c384 pn:W25Q512JV*M (DTR)
store:3 storage:40001 name:WINBOND1G fs:2000b93c pn:W25N01GVZEIG
store:4 storage:50001 name:WINBOND2G fs:2000ba30 pn:W25N02KVZEIR
Note: I didn;t connect sflash6 that is why its not in the list. Think with the longer wires the faster clock is an issue in the case of hardwiring to the carrier board. Flash is already at 30Mhz
 
Last edited:
Thanks for checking Tim but we may have to reduce the 45Mhz to 30Mhz. Have to run some more tests

EDIT: If you reduce the SPI Clock further to 30Mhz
1. Still seeing failure to copy some of the ManyF files to the Win2G so no help there
2. Using a MM ATP Carrier recognizes the chips if you hardwire the memory board to the carrier.
... Note: I didn;t connect sflash6 that is why its not in the list. Think with the longer wires the faster clock is an issue in the case of hardwiring to the carrier board. Flash is already at 30Mhz

Glad that is pinpointed - though odd and not ideal.

I wonder if Paul could put those two 2Gbit chips onto TWO PCB's? And on at least one not put pins on? Then you could cut custom wires to solder on that PCB after pushing wires into the proper ATP headers to minimize their length to min needed for the needed SPI wires?

If that doesn't work on T_MM then the interference may come from the PCB layout on the ATP or SFun MM PCB? It wouldn't likely work for anything but the ATP at that point without taking those wires off. Clearly 3" Dupont MF jumpers are too long ... first 45 MHz test was using them on ATP before going back to PJRC_MM PCB.

Not sure if that has any chance of help. If the problem is resistance or capacitance from extra wire? Maybe the SPI connect pins should be WW/30ga to PCB and a male pin set into the header? Where the female headers need 22ga otherwise?

<edit> MakeFiles didn't get any time for fix of All dir scanning. Edit in recursive scanning wrongly put in sep function as written that jumps over dirs it should not - but made first parsing seem easier.
... busy day with some outside time on a nice day filling and emptying pick three times so far with tree branches - more tomorrow ...
 
Last edited:
Glad that is pinpointed - though odd and not ideal.

Not really odd at all.

screenshot.jpg

30 MHz is the maximum rated SPI frequency. Anything over 30 MHz is overclocking, which often works, but should not be expected 100% reliable!

Even at 30 MHz, you can see in these specs a 10ns setup time is required. If the SPI memory chip changes its output on the falling edge and Teensy receives the data on the rising edge, that's only 16.7ns. So to meet this requirement, the memory chip needs to have its data valid with only 6.7ns delay from the falling clock edge.

If you look at the W25N02KVZEIR datasheet, on page 62 you'll see the "Clock Low to Output Valid" specification is 7ns maximum.

screenshot2.jpg

So even at 30 MHz, strictly speaking we're violating the timing requirements by 0.3ns. These specs are very conservative, as we can see the chip often does work at 45 MHz, but the specs are meant to cover a wide temperature range, the entire rated power supply range, and variation in silicon process and maybe other factors. We can probably "get away with" only a tiny 0.3ns violation of the spec, allowing 16.7ns when we should give it 17ns.

But the SPI clock definitely should not be configured faster than 30 MHz. Using 45 or 55 MHz is so far beyond the specification it should be expected to cause data loss. 30 MHz isn't odd.

If we talk of what would be "ideal", reducing the clock to 29.4 MHz would allow the SPI half cycle to fully meet the Winbond's worst case 7ns data valid spec and NXP's worst case 10ns setup time specification.
 
Last edited:
Not really odd at all.

....

30 MHz is the maximum rated SPI frequency. Anything over 30 MHz is overclocking, which often works, but should not be expected 100% reliable!

Even at 30 MHz, you can see in these specs a 10ns setup time is required. If the SPI memory chip changes its output on the falling edge and Teensy receives the data on the rising edge, that's only 16.7ns. So to meet this requirement, the memory chip needs to have its data valid with only 6.7ns delay from the falling clock edge.

If you look at the W25N02KVZEIR datasheet, on page 62 you'll see the "Clock Low to Output Valid" specification is 7ns maximum.

...

So even at 30 MHz, strictly speaking we're violating the timing requirements by 0.3ns. These specs are very conservative, as we can see the chip often does work at 45 MHz, but the specs are meant to cover a wide temperature range, the entire rated power supply range, and variation in silicon process and maybe other factors. We can probably "get away with" only a tiny 0.3ns violation of the spec, allowing 16.7ns when we should give it 17ns.

But the SPI clock definitely should not be configured faster than 30 MHz. Using 45 or 55 MHz is so far beyond the specification it should be expected to cause data loss. 30 MHz isn't odd.

If we talk of what would be "ideal", reducing the clock to 29.4 MHz would allow the SPI half cycle to fully meet the Winbond's worst case 7ns data valid spec and NXP's worst case 10ns setup time specification.

Thank you for the detailed explanation on the NAND timings. It really helps for the future. To be honest forget all about the max value for SPI for the IMXRT1062. We have been running SPI at all different speeds above 30Mhz, especially with displays that it never even registered.

It is interesting though that the N01 and M02 never seemed to have an issue with running at 55Mhz except on the TMM and when wired to the ATP carrier board. I have wired them up in the past on a breadboard and even on wired test board I put together without any issue. Just goes to show you that it can be deceiving.

Did check the other NANDs out of curiosity for their "Clock Low to Output Valid" max values:
Code:
W25N01 -  7ns
W25M02 -  8ns

I did put in a PR (https://github.com/PaulStoffregen/LittleFS/pull/48) to change the clock to 30Mhz as well as the changes to get the flash part number.
 
I did put in a PR (https://github.com/PaulStoffregen/LittleFS/pull/48) to change the clock to 30Mhz as well as the changes to get the flash part number.

:D

I have been playing more with storage and buffering into blocks (2k).. There is still a couple of things to finish, but appears to be working pretty well.
That is I played with setup with several storages on T4.1 and I looked at several of the storage units, plus I copied some files, ended up with about
300 files.

Reads/writes:
Code:
167779 RESP:2001(RSP:OK)l: 12 T:b6e
*** Storage Record Read/Write counts ***

    Writes: 49 to FS:7
    Reads: 3998 from FS:22
Record Info Blocks (index	block_index	last hit	dirty):
The write count is wrong, I logically moved the append code into the append method and did not increment the count...

If you wish to play, my cache_storage_records branch is updated.

Note: I temporarily added usage of my memory hex dump library to dump some of these blocks... It uses some of the more recent added stuff in that library. In specific
an optional argument that allows you to indicate what the location numbers should start at. Defaults to memory location, but in this case I prefer 0...

Now back to filling in some holes, like:
rename a file, with new name length > old name length...
Could run into issue if, I did not leave enough slop space in records for this... Worst case, I tell MTP to hit the panic button, and simply tell host to reset...
At which case we dump the entire index and it rebuilds again with again... Rebuilding again is on demand...
 
Back
Top