Hello,
I need to implement some form of on-board database connected to either a Teensy 3.2 or 4.0.
Some preliminary information:
I have been playing with a teensy 3.2 connected to a standard SDHC formatted as Fat32 using the SD library. My thought was that if I am to have virtually immediate finds and updates that I should write one file per entry. This way my "key" is the filename. The issue I am having with this at the moment is that it is taking upwards of 20 minutes just to transfer the first 13000 entries and this kind of timing is unacceptable. 5 minutes to transfer the full 100,000 entries is my upper limit target. However, I do not know for certain why it is slow at this time.
My other idea I want to explore is using FRAM. I'm thinking I can implement a hash table directly into FRAM ensuring fast finds and updates. I am not certain of the time it will take to do the initial write, but since I can address the FRAM with more granularity instead of writing big blocks I think it would be improved.
Any thoughts on such a system? Does anyone have experience in dealing with >=100,000 database entries with Teensy?
I need to implement some form of on-board database connected to either a Teensy 3.2 or 4.0.
Some preliminary information:
- 100,000 entries transferred when initiated by user
- ~100 bytes per entry
- Finding and updating an entry needs to be virtually immediate
- Non-Volatile
I have been playing with a teensy 3.2 connected to a standard SDHC formatted as Fat32 using the SD library. My thought was that if I am to have virtually immediate finds and updates that I should write one file per entry. This way my "key" is the filename. The issue I am having with this at the moment is that it is taking upwards of 20 minutes just to transfer the first 13000 entries and this kind of timing is unacceptable. 5 minutes to transfer the full 100,000 entries is my upper limit target. However, I do not know for certain why it is slow at this time.
My other idea I want to explore is using FRAM. I'm thinking I can implement a hash table directly into FRAM ensuring fast finds and updates. I am not certain of the time it will take to do the initial write, but since I can address the FRAM with more granularity instead of writing big blocks I think it would be improved.
Any thoughts on such a system? Does anyone have experience in dealing with >=100,000 database entries with Teensy?