nieuwemaker
Member
A part of my Teensy 4.1 project involves a simple webserver using QNEthernet and SD to physically read files from the SD card. My website is completely served from the Teensy and is located on the SD card. The header of the html looks like this:
As the example shows, a lot of files are requested from the SD card and this is where the weird stuff begins. If I only request 1 file it loads perfectly well, but when all these files are requested some files load perfectly, others stop loading at a random amount of lines and even weirder, some code even gets mixed up between files. So CSS code is mixed through JS code for example.
To mitigate the problem I've added a delayMicroseconds(100) after every line that is read and that works! This looks like this:
This is complete Bonkers right? The result is that it takes 25 seconds to load the page for 300kb of data, which is unacceptable for my usecase. So what is going on? Teensy cannot multi-task right, so although the browser requests multiple files at the same time, Teensy handles them in a serial order right? And why would that delay even matter? Am I reading too fast from the SD card? Is that even possible using the sdFileHandle.available() that is used in every example?
To provide some more context I will add snippets from the rest of my code so you can see what I do:
Every 5 milliseconds this method is called from the main loop to check if a web-request is being done:
Then it goes to the servePage() method to handle the file that is requested. It uses the standard fileHandler to read the file and send the input directly to the client stream.
I use the SD.open() function to open a READ handle for the file like this:
Hopefully someone here understands what is happening here and can help me out. As said. It works, but is very slow and I hate it if things work without knowing why.
<link href="css/app.css" rel="stylesheet"> <!-- GENERIC GLOBAL CSS -->
<link href="css/sections/appContainer.css" rel="stylesheet"> <!-- BASE DOCUMENT CSS -->
<link href="css/sections/appHeader.css" rel="stylesheet"> <!-- All header markup CSS -->
<link href="css/sections/connectTab.css" rel="stylesheet"> <!-- Connect tab CSS -->
<link href="css/sections/playTab.css" rel="stylesheet"> <!-- Play tab CSS -->
<link href="css/sections/solveTab.css" rel="stylesheet"> <!-- Solve tab CSS -->
<!-- JAVASCRIPT -->
<script src="http://pixelperfect/js/ipaddress.js"></script> <!-- Controller generated content to prefill data fields -->
<script src="js/generic.js"></script> <!-- Generic functions and GLOBALS that are used throughout the APP-->
<script src="js/plugins/svg.js"></script> <!-- little SVG library helping me to draw -->
<!-- JAVASCRIPT :: model classes -->
<script src="js/models/PubSubBase.js"></script>
<script src="js/models/Clip.js"></script>
<script src="js/models/Channel.js"></script>
<script src="js/models/Bank.js"></script>
<script src="js/models/Project.js"></script>
<script src="js/models/Animation.js"></script>
<script src="js/models/Sequencer.js"></script>
<!-- JAVASCRIPT :: web components -->
<script type="module" src="js/components/toggle-group.js"></script>
<script type="module" src="js/components/vertical-slider.js"></script>
<script type="module" src="js/components/channel-header.js"></script>
<script type="module" src="js/components/clip-view.js"></script>
<script type="module" src="js/components/animation-select-modal.js"></script>
<script type="module" src="js/components/clip-min-view.js"></script>
<script type="module" src="js/components/channel-min-view.js"></script>
<script type="module" src="js/components/bank-min-view.js"></script>
<script type="module" src="js/components/channel-view.js"></script>
<script type="module" src="js/components/bank-select-view.js"></script>
<script type="module" src="js/components/bank-view.js"></script>
<script type="module" src="js/components/quick-clips-view.js"></script>
<script type="module" src="js/components/project-view.js"></script>
<script type="module" src="js/components/sequencer-view.js"></script>
<script type="module" src="js/components/performance-view.js"></script>
<script type="module" src="js/components/status-bar.js"></script>
<script type="module" src="js/components/color-select-view.js"></script>
<script type="module" src="js/components/default-video-view.js"></script>
<!-- JAVASCRIPT :: Manager classes -->
<script src="js/models/managers/LocalStorageManager.js"></script>
<script src="js/models/managers/AnimationManager.js"></script>
<script src="js/models/managers/ControllerManager.js"></script>
<script src="js/models/managers/ProjectManager.js"></script>
<script src="js/models/managers/MidiManager.js"></script>
<script src="js/main.js"></script> <!-- main() function to start APP -->
As the example shows, a lot of files are requested from the SD card and this is where the weird stuff begins. If I only request 1 file it loads perfectly well, but when all these files are requested some files load perfectly, others stop loading at a random amount of lines and even weirder, some code even gets mixed up between files. So CSS code is mixed through JS code for example.
To mitigate the problem I've added a delayMicroseconds(100) after every line that is read and that works! This looks like this:
while (sdFileHandle.available()) {
client.write(sdFileHandle.read()); // stream file per byte to http client
delayMicroseconds(100); // was experiment to see if files are loaded completely this way ...
}
This is complete Bonkers right? The result is that it takes 25 seconds to load the page for 300kb of data, which is unacceptable for my usecase. So what is going on? Teensy cannot multi-task right, so although the browser requests multiple files at the same time, Teensy handles them in a serial order right? And why would that delay even matter? Am I reading too fast from the SD card? Is that even possible using the sdFileHandle.available() that is used in every example?
To provide some more context I will add snippets from the rest of my code so you can see what I do:
Every 5 milliseconds this method is called from the main loop to check if a web-request is being done:
void WebserverController::HandleClient(){
this->client = this->server->available();
if(this->client && this->IsConnected() && this->client.available()){
// step 1: read what the client is requesting
// code here to do so
// step 2: if it is a request for a weburl
this->servePage(getUrl); // --> this handles the web url request
this->client.stop(); // saves memory to quit after each request
}
}
Then it goes to the servePage() method to handle the file that is requested. It uses the standard fileHandler to read the file and send the input directly to the client stream.
void WebserverController::servePage(String url){
if(url == "/"){ url = "/index.html"; } // redirect Root to index
String sdName = SD_FILE_PATH + url.substring(1); // get rid of the "/" in front
File sdFileHandle = this->sdController->OpenForRead(sdName);
if(sdFileHandle){ // The file exists and can be opened
String fileType = url.substring(url.indexOf(".")+1);
this->sendCorrectHeader(fileType.toLowerCase());
while (sdFileHandle.available()) {
client.write(sdFileHandle.read()); // stream file per byte to http client
delayMicroseconds(100); // was experim,ent to see if files are loaded completely this way ...
}
sdFileHandle.close();
client.closeOutput();
} else { // requested source not found. Let's send a friendly message
this->sendHTMLHeader(RESPONSE_NOT_FOUND,MIME_HTML);
this->client.printf("<html><body>requested HTML document <b>%s</b> does not exist</body></html>",sdName.c_str());
}
}
I use the SD.open() function to open a READ handle for the file like this:
File SDController::OpenForRead(String filename){
if(this->initialized){
return SD.open(filename.c_str(),FILE_READ);
}
return NULL;
}
Hopefully someone here understands what is happening here and can help me out. As said. It works, but is very slow and I hate it if things work without knowing why.