PaulStoffregen
Well-known member
Duff's Snooze library was added to 1.21.
Look in File > Examples > Snooze > deepsleep > pushbutton_pullup_deepSleep
Teensyduino 1.21 Released
- Thread starter Paul
- Start date
- Status
Duff's Snooze library was added to 1.21.
Look in File > Examples > Snooze > deepsleep > pushbutton_pullup_deepSleep
Duff's Snooze library was added to 1.21.
Look in File > Examples > Snooze > deepsleep > pushbutton_pullup_deepSleep
Cool Thanks!
defragster
Senior Member+
... watching and saw that m#24 m#25 discussion . . . I wasn't sure if it was the same or better under 1.20 in the end.
zachtos: I'd suppose the content of the other files shouldn't matter so if you could take even 1 sample file and programmatically make the 100('s) of needed unique file named versions it would be easy to reproduce that from a batch/script file. Doing that and seeing the change in perf between 1.20 and 1.2x going forward could then be reproduced from the script and the one source file and no wait for snail mail.
Using the 'code' below in a DOS box as a '.bat' batch file with one file "_.wav" I get 285 copies from a clean directory.
If you could program around this in a sample sketch Paul could repro it with a single copy of "_.wav" on a formatted card.
Notes:
* This assume all files in a single flat directory
* If you have subdirectories I can make update to do that, this isn't elegant but it works
* It will create a clean FAT table with no file fragments on a clean 'disk'
* If this doesn't allow you to repro then the issue may be in the file management over FILE or DIR fragments.
* If you edit the 'rem' off line #2 you end up with 545 copies.
* Start with fewer files by removing any number of the values in one or more sets of parenthesis.
* You can get 10 unique wav files by starting with #.wav and putting 'rem' on the first line.
zachtos: I'd suppose the content of the other files shouldn't matter so if you could take even 1 sample file and programmatically make the 100('s) of needed unique file named versions it would be easy to reproduce that from a batch/script file. Doing that and seeing the change in perf between 1.20 and 1.2x going forward could then be reproduced from the script and the one source file and no wait for snail mail.
Using the 'code' below in a DOS box as a '.bat' batch file with one file "_.wav" I get 285 copies from a clean directory.
If you could program around this in a sample sketch Paul could repro it with a single copy of "_.wav" on a formatted card.
Code:
FOR %%i IN ( 1 2 3 4 5 6 7 8 9 0) DO copy _.wav %%i*.*
rem FOR %%i IN ( A B C D E F G H I J) DO copy *.wav %%i*.*
FOR %%i IN (a b c d e f g h i j k l m n o p q r z t u v w x y z) DO copy ?.wav ?%%i*.*Notes:
* This assume all files in a single flat directory
* If you have subdirectories I can make update to do that, this isn't elegant but it works
* It will create a clean FAT table with no file fragments on a clean 'disk'
* If this doesn't allow you to repro then the issue may be in the file management over FILE or DIR fragments.
* If you edit the 'rem' off line #2 you end up with 545 copies.
* Start with fewer files by removing any number of the values in one or more sets of parenthesis.
* You can get 10 unique wav files by starting with #.wav and putting 'rem' on the first line.
not very helpful, i know: but anecdotally, re play_sd_wav i think i can confirm there seems to be some deterioration in performance.
there were too many changes recently to say much more. fwiw: i still have 1.0.6 installed with 1.21beta-2, in which case performance is ok. using the same sketch with 1.6.1 and official 1.21 (120MHz, compiled with -O2), there's occasional hick-ups.
OK, I have compiled a test and simple code that proves the bug. I will share the files you can put on an SD card to replicate test:
test setup:
-Teensy 3.1 and audio shield, using amplifier line out left channel to amplifier then to speaker
-one copy of teensyduino 1.2 and arduino 1.06
-one copy of teensyduino 1.21 and arduino 1.61
-class 4 sandisk 4gb SD card (class 10 version made no difference for me)
-copy directory into root of SD card (large number files = 800 raw files)
-run playback interval set at 350ms (adjust volume variable as needed)
-faster you playback files, the worse the squeek/pop between files should get
-also different file numbers will have worse squeek/pop (how far into directory?)
-v1.06 should play a countdown verbally '1, 2, 3, 4'
-v1.61 should try to play same countdown, but skips many numbers randomly '1,3,4... 2,3,1, .. 3,4... 1,2,3...'
*I know bug got worse after version 1.61 arduino without my code changing at all. I can't play multi sounds at all anymore reliably!
*there is a 'squeek/pop' bug that I KNOW is somehow related to the number of files on SD card, it gets worse the lower you set playbackInterval
** if you try the same thing w/ only 4 files in the directory (one, two, three, four), the squeek bug goes away (may need to change file names on card).
Link to files (27mb rar file)
Please note, I had a similar bug with the old arduino wave shield regarding squeeks/pops with large file numbers. I ended up solving it by using play files by index (example in old WAVE_HC google code libarary). Basically you indexed file names into an array before starting the program, then playback files by the file number/address on card instead. This would be very valuable to us, and I imagine it could be as easy as playByIndex() or something in the audio library, and an index setup command.
Last edited:
PaulStoffregen
Well-known member
Ok, I've put this back on my list of stuff to investigate.
At this moment, I'm wrapping up the SerialFlash library. Then I'm going to look into the Lidar crash. And then a couple documentation tasks. Then probably this... so probably look for it sometime next week.
At this moment, I'm wrapping up the SerialFlash library. Then I'm going to look into the Lidar crash. And then a couple documentation tasks. Then probably this... so probably look for it sometime next week.
PaulStoffregen
Well-known member
Yes. I made more improvements to the serial monitor, which are in the 1.6.1 (and 1.6.2 beta) patches. If you use the serial monitor, especially on Windows, those improvements are very worthwhile. That stuff won't ever be backported to the older versions.
PaulStoffregen
Well-known member
The new library manager in 1.6.2 is going to be pretty nice (already it's getting filled with a lot of libs). If you want only stable stuff, wait for 1.6.3 and the final Teensyduino 1.22 release. 1.6.2 is still a bit rough around the edges, though Teensy isn't depending on the new package system they built. If you're willing to try beta stuff, 1.6.2 should work ok, and you can see where things are going....
Let me know when you start investigating this problem. We are stuck at v1.06 until then. I also spent over 40 hours in the last month trying to solve the high file number audio squeak bug, and would be willing to pay for help at this point.
PaulStoffregen
Well-known member
I actually started looking into it last night, but just looking at the code. The SD library files are exactly the same, so whatever happened must be something subtle. I wanted to do a quick code review before releasing 1.22.
This and the Lidar issues are near the top of my list to investigate after 1.22 is released (which I'm doing right now...)
Are you using Visual Micro or some other non-Arduino way of building your code? I'm planning to test using only the Arduino IDE.
This and the Lidar issues are near the top of my list to investigate after 1.22 is released (which I'm doing right now...)
Are you using Visual Micro or some other non-Arduino way of building your code? I'm planning to test using only the Arduino IDE.
I am strictly using teensyduino and arduino. Easy design with the power of an ARM controller is very appealing for a small company like us. As you well know, I wear many hats as the lone engineer.
defragster
Senior Member+
zachtos: I was following a thread that lead to a link that ended here: https://forum.pjrc.com/threads/16758-Teensy-3-MicroSD-guide - from when SDfat got started and folks setting up and benchmarking.
I haven't set up any of this or looked at the samples - but I was wondering if any of the benchmarks referenced might show a difference from standard library examples from the 1.0.6 and 1.6.3 IDE's. Of course they may work perfectly - and you might need to mod them to do multi-file access to demo your situation.
If that gives a difference Paul can use that feedback - if not then it would be interference from another part of your project which would again be good feedback.
Also what happens if you use the 1.0.6 installation with TeensyDuino 1.22? That would point to either the changed libs getting to 1.22 or the IDE as the source.
I haven't set up any of this or looked at the samples - but I was wondering if any of the benchmarks referenced might show a difference from standard library examples from the 1.0.6 and 1.6.3 IDE's. Of course they may work perfectly - and you might need to mod them to do multi-file access to demo your situation.
If that gives a difference Paul can use that feedback - if not then it would be interference from another part of your project which would again be good feedback.
Also what happens if you use the 1.0.6 installation with TeensyDuino 1.22? That would point to either the changed libs getting to 1.22 or the IDE as the source.
PaulStoffregen
Well-known member
I am going to get to the bottom of this, and in general really look into SD's efficiency, likely sometime in the coming week.
(this weekend was mostly burned up by a much-needed Linux reinstall on my computer, which I've been putting off for months due to Teensy-LC and then the rapid release of 4 Arduino versions)
The SD code also looks really slow for Teensy-LC, and there seems to be quite a bit of opportunity to improve it for Teensy 3.1 too. Before Arduino 1.6.x, I've always been really cautious about doing too much in these libraries, since I do very little testing on non-Teensy boards. Now that we have 1.6.x with the ability to isolate changes to only Teensy, it's time for me to really put some serious optimization into the normal SD library!
(this weekend was mostly burned up by a much-needed Linux reinstall on my computer, which I've been putting off for months due to Teensy-LC and then the rapid release of 4 Arduino versions)
The SD code also looks really slow for Teensy-LC, and there seems to be quite a bit of opportunity to improve it for Teensy 3.1 too. Before Arduino 1.6.x, I've always been really cautious about doing too much in these libraries, since I do very little testing on non-Teensy boards. Now that we have 1.6.x with the ability to isolate changes to only Teensy, it's time for me to really put some serious optimization into the normal SD library!
PaulStoffregen
Well-known member
Oh, great, now I'm seeing this after copying all the files to a Sandisk Ultra 16 GB.
The only other SD cards I have here are unlabeled speed "Sandisk 8GB". I suspect they're counterfeits, not genuine Sandisk.
I'm going to try with these 2 cards. If neither reproduces the problem, I'm not sure what to do next....
defragster
Senior Member+
Is there only one way to format the cards? Different format type, and the fact that yours went from empty to full with a single linear copy means no fragments and a clean directory list could make a difference.
Do the file lend themselves to a modification of the standard benchmark test to find named list or wild card of all the files and read them with a time per file and overall logged. With a modified benchmark sketch just on file read zachtos could perform that as a baseline. His comparative results would point out the next direction/step.
I assumed I'd see 'benchmark' as a sample - looks like you could combine listfile and dumpfile examples easily enough without editing up a file list. Though you could optionally read the a counting number file between the other files.
Do the file lend themselves to a modification of the standard benchmark test to find named list or wild card of all the files and read them with a time per file and overall logged. With a modified benchmark sketch just on file read zachtos could perform that as a baseline. His comparative results would point out the next direction/step.
I assumed I'd see 'benchmark' as a sample - looks like you could combine listfile and dumpfile examples easily enough without editing up a file list. Though you could optionally read the a counting number file between the other files.
defragster
Senior Member+
This compiles and will hopefully blink the light as each file reads without error. I didn't run as I have not set up for SD and never used this code before.
View attachment SDfileListDump.ino
Prints summary totals when done. To get individual per file data uncomment line below, it adds some delay() and the USB I/O will intermix with SD access:
Anxious to hear how close it is to working for one of you to post and share to the other to compare.
I should have counted bytes as I read them in DumpFile() to make sure the file size matches the read size.
Doing PER_FILE without delay()'s comparison would show USB impact if any.
View attachment SDfileListDump.ino
Prints summary totals when done. To get individual per file data uncomment line below, it adds some delay() and the USB I/O will intermix with SD access:
Anxious to hear how close it is to working for one of you to post and share to the other to compare.
I should have counted bytes as I read them in DumpFile() to make sure the file size matches the read size.
Doing PER_FILE without delay()'s comparison would show USB impact if any.
PaulStoffregen
Well-known member
I've got the test running now with my 8GB card. I'm able to reproduce the problems, squeeks on 1.0.6 TD 1.20 and skipped numbers and pops on 1.6.3 TD 1.22.
This is looking like a very subtle race condition somewhere, including managing access to the SD card and SPI bus between the main program and audio interrupt. I've found a few ways to affect the problem, but it's going to take some serious digging into the code to figure out exactly where it's going wrong.
I'm on it. This is looking like it's going to be a really tough one. Will post updates here as I learn more....
This is looking like a very subtle race condition somewhere, including managing access to the SD card and SPI bus between the main program and audio interrupt. I've found a few ways to affect the problem, but it's going to take some serious digging into the code to figure out exactly where it's going wrong.
I'm on it. This is looking like it's going to be a really tough one. Will post updates here as I learn more....
Thanks, I appreciate this greatly. Our product will be very professional grade after stamping out this final squeak bug. If not, play files by index may be a work around (open all files at start and store in index array for opening later). I anxiously await a solution.
PaulStoffregen
Well-known member
Just a quick update... I'm actively working on this. It's turning out to be a combination of several subtle interrupt safety issues and also some places disabling interrupts too long.
The good news is looking like the SD and SPI libraries certainly have the raw performance needed. Latency to open a file near the end of the directory is about 50 ms with the card I'm testing.
At the moment I'm digging into parts of the SD library I've never touched before. There seems to be some static data structures somewhere buried in there, which also need proper management for shared access.
The good news is looking like the SD and SPI libraries certainly have the raw performance needed. Latency to open a file near the end of the directory is about 50 ms with the card I'm testing.
At the moment I'm digging into parts of the SD library I've never touched before. There seems to be some static data structures somewhere buried in there, which also need proper management for shared access.
PaulStoffregen
Well-known member
Yes. Well, it will be with the updates I've been developing today.
It turns out the other major obstacle is the single-sector cache in the SD library. That's where the noises and failure to open files are coming from. This afternoon I've been working with Bill's SdFat code in the utility directory, to add functions to properly track the usage time window for each cached sector. Once that's done, I should be able to expand the cache to multiple sectors, which will allow non-conflicting directory searches and playback, and it should also improve performance, since the single sector cache experiences a lot of cache misses.
For years I've resisted really digging into the SD library code. I just patched the low-level card access. Well, now I'm finally overhauling the entire SD library so it will truly perform properly for these much more demanding uses.
I hope to have a test version posted early next week.
It turns out the other major obstacle is the single-sector cache in the SD library. That's where the noises and failure to open files are coming from. This afternoon I've been working with Bill's SdFat code in the utility directory, to add functions to properly track the usage time window for each cached sector. Once that's done, I should be able to expand the cache to multiple sectors, which will allow non-conflicting directory searches and playback, and it should also improve performance, since the single sector cache experiences a lot of cache misses.
For years I've resisted really digging into the SD library code. I just patched the low-level card access. Well, now I'm finally overhauling the entire SD library so it will truly perform properly for these much more demanding uses.
I hope to have a test version posted early next week.
I anxiously await, while plowing ahead myself on other parts of the project. We just sent our second video to (-a popular American TV show-) for review (second year now). Hopefully we make it on this time! All thanks to the rapid prototyping powers of Teensy and Arduino combined.
Last edited:
- Status