I2C, SPI, ESP webSockets controller for teensy

Status
Not open for further replies.
dont worry about the queueing :)

i like to test things and last test i did for payload queueing i did just over 5300 queues, we can for sure offer limits for stale by popping the front items to keep queues within limits, by assigning a limit of x queues, thats simple

my ESP callbacks have queueing support without limits, adding that protection is a simple 1 liner
 
No worry - but you don't have it done yet :)

No rush as noted - not sure if/when I'd even get it in use. Though the project that brought the idea about in adding more MATH is bogged down again after the next step. I didn't look at it yet, but the math may be taking a full 1 ms now and that generates a desire to post the data stream to be plotted for viewing - and that is a double negative with no positive side. So maybe it might be a way to evolve the math and allow a full view of the data at the same time to refine and monitor it.

TeensyThreads can break up the math so other things can work - but that introduces issues with data management to print it before it is updated - and still doesn't assure the system doesn't get overloaded where formatting the float data for output seems to take 0.5 ms. so it can stuff it on USB to get to a PC that doesn't seem to have issues resolving the data to nice graphic feedback.

Writing that made me wonder about going wireless - since these are GPS/Motion issues - if the data streamed out wirelessly ... rather than wired SPI ... the slave Teensy could stay static on the ground and feed the data to the PC. That might even be a better way to go. Perhaps a pair of Teensy+ESP8266 and a ... New project code named: teensquitto ???

<edit> BTW - in end use once the code is seen to be working - the need for over the top logging data will go away - so this is a development time issue - compounding the development . . .
 
for SPI the master controls the clock, so if master doesnt pulse then the slave doesnt shift out to master, thats why
 
if you want i could template a master slave simple controller just for your needs without the extra overhead pf the controller code, then we can go from there at maximum implementation of custom code

if you also want you could run master/slave on both ends with 2 spi ports giving full bidirectional support, if you have 2 spi pirts to spare, which t3.5/6 do :)
 
tonton81 said:
for SPI the master controls the clock, so if master doesnt pulse then the slave doesnt shift out to master, thats why
Click to expand...

Understood - the Master doesn't have to care if the Slave is there - that is the User's job to arrange. As noted it is Development use only so it can be ignored when it doesn't matter and can be fixed when it does.

Putting anything in the MISO stream would just be a way to know at the Master that the Slave was online - thought that was what you wondered. There is no need for back channel communication - this is just a broadcast. If it is reliable when properly configured for Dev usage then it will solve that need.

Next post: If you wanted to make a simple proof of concept version that would be enough to start with to see the utility. I only expect a single packet and message - and that could be overloaded to hang a second struct on if needed - or union it and not tell anyone that every third message was different based on the first element's value.

Once it is seen to work and have value it could be over engineered later:)

You can pick any critical parts of my post #19 steps that would get it off the ground and ignore the rest that would fit in with the code you have?

If you put the code on github I can play along and learn something and maybe even be helpful.
 
yeah ill prolly strip it down to bare SPI without all the port controls, just to get the basics understood, at the same time ill setup a simple payload function as a sample of how it will be implemented, thisll make it easier to understand, and perhaps implement the slave as a library exposing a callback function in the sketch that you’ll be able to add your own parsers without all the backend bloat visible

something like the slave being simpler for user:

#include .......
(constructor)
setup(){
callback enabler function;
}
loop(){}
callback(spi_buffer_packet_from_master,len);


this in return, you may not only be able to add your own spi parser in sketch, but, you may also add background parsers that wont goto user callback if you want a permanent code that runs without going through the callback. Sort of like an internal callback passthrough interface, if a hook doesnt exist for the given payload, it sends the data to the user callback instead, sorta like a filter

once that setup is completely cleaned up like above, as simple as it is, and your code is implemented, ill show you how the callback queuing support is done when its needed, putting it in the user interface will give you better control on what you want queued or not

ps: i never did any github uploading nor do i know how (yeah im a n00b at some stuff), and your all aware my documentation is lacking and relied only on forum posts
 
Last edited:
This looks promising as you - A: have a working starting point B:seem interested C: have a plan it looks like D: seem to think the plan will work

So by strip it down - you will be starting with the sources on this thread?

I think the callback is a good starting point. The slave will have work to do when it gets the packet, once it secures the packet in the callback, it can go to work on it and then busy itself with whatever until the next callback. If it gets involved in the SPI handling it could turn into a distraction or a disaster and turn the SPI code into a bunch of special purpose spaghetti without adding anything as it will just be waiting for data to consume.


Github gives me unending grief - but when it works it is very useful. I've used decent but pre-historic source control code that seemed obvious and good for big projects. Github offers a well developed simple GUI that somehow doesn't work as I've decided it should based on what I used to work with. Often the issue is clone versus fork and not having rights to make changes in other's sources - but trying to figure out how to do a Pull request from changes on my machine. More often the trouble is I make changes I want to undo and somehow get github out of sync with the world and have to delete the clone or fork and start over to get it to not remember what I tried to tell it to forget. But it clearly works when used as designed.

If you have a github account or make one let me know - I could start a project and initialize it and give you write access to it - or the other way around - and maybe it will work. On the other thread this code may get used in we are posting zip's across 3-4 folks in some fashion and it is extra work with no code tracking and piles of stuff on my computer. Maybe it will inspire me to RTFM and learn what it thinks I'm doing wrong so I can do it wrong and it will think it is right?
 
yeah im willing to work on it, im goinng to stip out all the unecessary i2c and controller code and start something new as you gabe me ideas, i might add plugin support so we can add additional functionality without overloading a single file, and ill add the other port accesses as a plugin later on
 
Last edited:
That sounds good and promising for this and future projects.

The other project has multiple files - they can be .INO or .CPP I learned - I had wrongly thought it was a single INO per sketch. Makes it easy to jump to specific code.

Is there any future in writing an ESP Slave to talk to Teensy that would do this - have that slave talk to a 2nd ESP - then have that ESP act as Master to a second Teensy? Of course that could be a variation of Teensquitto usage in some fashion with WiFi messaging? You were posting over 3000 message/sec using that in one test at 2Mbaud on Serial?
 
yes, its only at 2megabaud so i can use serial monitor, theres no baud at 3megabaud in serial monitor so i cant debug ESP until i lower it, but 3megabaud indeed works and pretty solid.

the thing with the ESP is the SPI slave code is not very stable, heck, the SPI master even has problems randomly shifting the bits, I gave up with that long ago when i did the winsock version, which, like i said, i should have started with mqtt, its way faster and persistant than winsock version. the other problem woth ESP, I2C is not hardware, its software I2C. When I dove into the uart area, i took a good week or so to decide on a protocol that supported dynamic buffers with crc checks to validate inter-mcu connections, its been solid since and never had a hickup, nor required modifications despite everything i throw at it.

regarding the ESP talking to teensy, with my uart protocol, its 2 way traffic, it can send commands, teensy can send commands, they can update each other, which is what the current project on ESP does, so as of this, its technically 3megabaud dual master. teensquitto will receive a multi-teensy plugin in future that would take advantage of the ESP functions on teensy, I havnt worked on that since the progress of the ESP library, but it is planned. If your referring to messaging between teensy’s ESP’s then yes:
teensy&ESP <—— wifi ——> mqtt server <—— wifi ——> teensy&ESP
thats what i plan on working on in future

so yeah currently both ESP & teensy can talk to each other currently, and both have the same uart protocol i wrote, the only custom function implemented is a lambda anonymous function binded to the ESP object for ESP.onDetect callback, otherwise I try to keep it as transparent as possible.
 
im going to redesign the spi controller, the command headers will be 16bit instead of current 8bit with 16bit transfers
i already started a barebone slave setup ready for data input and got a led toggling on the slave (after valid crc check) in a tight 20ms loop followed by changing it in the IDE to 0ms gives the led a nice steady glow while CRC validations are still flooding debugging console. later on i will strip down the master end to a bare minimum as well for a redesign as well

after the master strip down, ill tackle a handler setup for the user as well as a new constructor for the master
probably before the weekend hopefully you can start trying stuff. i will also try to get the slave end as a library as well so the sketch can be dedicated to the user, and what i might do later on is add/make sure
bi-directional support for cross-SPI implementation

teensy 3.5/3.6 teensy3.5/3.6
SPI2 ————————————> SPI0
SPI0 <———————————— SPI2

this will hopefully be a master-master design using the same library, and allow each one to control each other, at crazy speeds! :)

PS:
heres an uploaded video of the 20ms and 0ms tight loops with CRC verifications flooding the slave debug console
https://www.youtube.com/watch?v=sog8Z6UPT8o
The camera doesnt do justice for the speed of the led, as we all know, but it's super fast, with the console going nuts with flooded CRC validated packets

The SPI buffer on the new spi controller code will be set to 1024 by default, the data packets will be same as the ESP project
0xAAAA, 0xBBBB, 0xCCCC, 0xCCCC, 0xCCCC, 0xCCCC, etc....... -> 0xDDDD
AAAA will be a 16 bit header
BBBB is length of command data payload
CCCC will be data, depending on the transfer handling of the function, it will be filled with either 8 or 16 bits, and handled same on other end
DDDD will be the checksum

theres 2 point check for end of length and end of buffer, when either is reached, the buffer pointer is reset and all further SPI0_POPR are dropped until next transaction.
The idea here will be when a command is sent that requires a response (return variable per say), it will start pumping 0xFFFF packets to get the SCK line pumping while the data is ignored at slave incomming end, the slave can push the data in the register when it finally gets it and the data will flow back to the master for capture, at which point, the line is deasserted and continued. So this gives the slave enough time to finish it's processing depending on the function called to return data, while the master helps it push the data out.

oh yeah! I just had a new idea of how im going to implement an async callback for the master end when a user sends data to a slave and doesnt want to wait, the callback data will be queued at the slave, eventually when events() hit (just like the ESP), events will check for a callback queue size, if there are queues, we wont tie up the SPI port to drop from an ISR into a handler, instead my idea is as follows:

1) The MASTER sends data to the slave, saying "Hey, whats the time!?"; and deasserts the line and continues the loop();
2) The Slave has the data ready, so it queues it up into a vector queue and continues it's own loop();
3) Master makes it's trip over the events() in loop, and requests a queue size, slave immediately responds "I have 3 items in queue!"; The master dumps the 3 remote queues into it's own local queue buffer and deasserts the line, giving the slave's SPI ISR some rest while it continues it's own loop(); Right after the line deasserts, the handlers are called immediately and start processing. If a handler contains a function in the scope of this spi_controller library, further callbacks in the Master's queue are stopped dequeing pending function completion, then on next round trip in loop, if no functions are pending, it processes local handlers in queue and checks again if any others exist. This should allow callbacks that will not tie up the SPI bus giving both teensies more freedom
 
Last edited:
Eureka, I figured out an implementation!

1) Lets say the Master wants to send a request and immediately want the response (this isn't fire and forget test, without dropping the CS line, and during the same transaction)
^-- the catch is, the data transfer payload happens first (very similar to UART! )
2) Slave ISR fires, starts capturing up to the packet size specified in previous post, the processing takes over and validates the CRC first.
3) Now that the processing is open with the payload and lenght, and valid CRC, your function requests a response, so if a valid function was entered this happens:
3.1 Since master is waiting, it keeps pulsating the clock line so the slave can immediately respond when ready <---, what about sync you say?!? Hold up and see!
3.2 Slave sends the array (yeah! I actually said an array!) that has a VALID 16bit CRC, by sending total length and looping the transfer until the master is satisfied. <--- wait! you think im getting more crazy!? what about the sync!? Just wait now! :)
3.3 Master picks up the header in the stream after ignoring other bytes unknown. Out response header? 0xAA55. An array is built based on the size of the next uint16_t byte pulled in.
The 1st byte of buffer is set to the header 0xAA55 and the next byte of the buffer is set to the length, the remaining length is pulled in TO BE VERIFIED!
The Master captures the array and does a 16bit CRC check on it to match with the last byte of the array. 2 fun things happen:
If the CRC is valid, line is deasserted and the function exits! YAY SUCCESS!
And if the CRC fails? No problem! Perhaps you had a bad SPI connection giving a bad signal. Shame on you breadboard wires! ;)
^-- it continues pulsing until the next response comes in, which is the same one the slave is loop sending as the master toggles the clock!!!!
^--- verification again! crc passed! you can be assured your transaction is a success! :)

To put it in simple terms, Master sends a complete array to the slave, and THEN the slave sends a complete array to the master (VERY similar to UART!) :)

defragster! perhaps asynchronous callbacks will be EASY now that arrays can be sent back and forth! hahaha :)

welcome to array based SPI protocol T3.5 SLAVE & T3.6 HOST
https://www.youtube.com/watch?v=LW7PoVLKqII

Also! If you didnt catch in the video! noticed the host has no while loop?:) It pushed 0xFFFF *X* amount of times (in this case 100), which can captured the array up to 10 times before giving up, or just quit early if successful with valid crc! :)

Perhaps later on I could create a hardware uart thats baudless and communicates over *x* MHz SPI bus We'll call it Serial7? to talk between both teensies. So much potential

lets push it to 0ms toggling tight loop with end-to-end CRC verification and back!
https://www.youtube.com/watch?v=_1KTeCOZRBY

I did a millis() second interval check for a count, and it seems to be doing 1000 returned valid CRC responses, PER SECOND! :) (count 6006 -> 7007 == 1 sec, count 1899 -> 2900 == 1 sec! its pretty consistant! )
 
Last edited:
now since i figured out how to get back guarenteed responses, i have another idea to work on today, where when the master will send an array with CRC and will get acknowledgement within a certain timeframe, lets say 100ms, that the packet was accepted due to CRC validation, before it falls into the slave’s response.

Just like the SLAVE -> MASTER data protection of a looped array in stream, the MASTER will have an auto resend feature if the packet sent is not ACK’d within 100ms immediately after CRC validation, if the ACK occurs within the timeframe, the response stream will be started :)
 
Ok! Good news! I've tested the T3.6 Host on the T3.5 Slave

I wrote a test digitalWrite function with new code

Code: 
void loop() {
  static uint32_t _timer = millis();
  if ( millis() - _timer > 1250 ) {
     _timer = millis(); 
    static bool flip = 0;
    uint32_t _time = micros();
    teensy_gpio.digitalWrite(13, flip = !flip);
    Serial.println(micros() - _time);
  }
}

So the HOST (T3.6) basically runs the test sketch blinking the remote led.
Resend support is added, there is automatic resend of up to 3 times of the array from HOST -> SLAVE ( 3 resends at 100ms timeout until it gives up), it's a chain reaction, slave sends back the ACK, master sends back continue packet, and then master skips to the response pulsing, and it's the slave's turn to circular send the array as long as the master is pulsating to capture a valid array. All good so far! :)

So now I have implemented 2 way ACKnowledgements with automatic resends for master's output and circular buffer sending of slave's responses!

Now in the above code, I'm testing micros() at T3.5/6 default CPU speed
1MHz SPISettings == 243 micros
2MHz SPISettings == 141 micros
4MHz SPISettings == 136 micros
8MHz SPISettings == 62 micros
16MHz SPISettings == 53 micros
24MHz SPISettings == 52 micros

These were tested with full implementation, CRC checks for traffic both ways and data assurance for both ends, including the time it takes to digitalWriteFast the pin state

I want to also throw in here that there are NO delayMicroseconds in the test code between the SPI transfers. !! :)

I just redone the digitalRead function now, and tested the new loop()

Code: 
void loop() {
  static uint32_t _timer = millis();
  if ( millis() - _timer > 1250 ) {
    _timer = millis();
    uint32_t _time = micros();
    teensy_gpio.digitalWrite(13, !teensy_gpio.digitalRead(13));
    Serial.println(micros() - _time);
  }
}

at 4MHz, using the Read/write combined method, micros is at 158 :) thats 22uS difference than using a local bool :)
 
Last edited:
After disabling debug printing in the ISR (yeah yeah i know :) ), im getting 130uS with a read&write to the led pin of the slave at 4MHz

Testing hot plugging! I changed the retry count to 44 temporarily to demonstrate the same function that didnt get validated due to missing CS line! :D

Here's the video!

https://www.youtube.com/watch?v=Xpuz8SUhp8I

Serial monitor demonstrates a print statement when the resend command is automatically retransmitted due to a 100ms timeout from lack of ACK response due to hot plugging the CS line of teensy!

running this occasional loop in flooding mode of 20ms (just cause i want to at least see the led blink a bit), i do sometimes hit a resend:

Code: 
31
130
130
131
130
130
131
130
[COLOR="#FF0000"]RESENT PAYLOAD[/COLOR]
111068 [COLOR="#FF0000"]<-- micros time recovered with a single resend (111ms total time of transaction), successful transaction![/COLOR]
130 [COLOR="#FF0000"]<-- 0.13ms for next transaction, as you can see, the 100ms lost was the timeout before resend :) [/COLOR]
130
130
131
131
131

I think this is a good reason I implemented error checking :)

Update: I'm doing my SPI tests using teensquitto, both running same time! Heres a video:
https://www.youtube.com/watch?v=aPm2L1AM2LU

If you do catch the "resend payload", good thing it's error corrected internally :)
 
Last edited:
defragster, I'm going to start working on the handler, is there any features we should include regarding that?

I need a name of 2 things,

The name of the callback function to assign your callback (at SLAVE end)
Code: 
ex.:  Obj.[COLOR="#FF0000"]setCallback[/COLOR](myFunc);
.onReceive? .onCall? .etc?
Then you could do something like this:
Code: 
void myFunc(uint16_t *data, uint16_t len) {}

and most importantly, a standard sending function to submit your own array to the handler (from MASTER end);
Code: 
ex.:  Obj.[COLOR="#FF0000"]send[/COLOR](buf,sizeof(buf));
share()? transfer()? upload? etc?


Without tieing up the slave's ISR line, I will make the slave queue up a vector array to store the callback, and when the ISR exits, the callback will be dequeued from the loop() and pushed into your actual handler
I believe that once thats done, and your arrays are transfered into your callback, you may start processing them outside of the SPI ISR while each teensy maintains their own loop();
 
defragster, I'm going to start working on the handler, is there any features we should include regarding that?

I need a name of 2 things,

The name of the callback function to assign your callback (at SLAVE end)
Code: 
ex.:  Obj.[COLOR="#FF0000"]setCallback[/COLOR](myFunc);--> onReceive? setHandler? etc?
Then you could do something like this:
Code: 
void myFunc(uint16_t *data, uint16_t len) {}

and most importantly, a standard sending function to submit your own array to the handler (from MASTER end);
Code: 
ex.:  Obj.[COLOR="#FF0000"]send[/COLOR](buf,sizeof(buf));--> .share?.upload?.sync?.etc?


Without tieing up the slave's ISR line, I will make the slave queue up a vector array to store the callback, and when the ISR exits, the callback will be dequeued from the loop() and pushed into your actual handler
I believe that once thats done, and your arrays are transfered into your callback, you may start processing them outside of the SPI ISR while each teensy maintains their own loop();
 
i think adding a packet identifier to identify the array being sent is a good idea, to manage arrays being sent from different events that you want handled differently, something like:

void myFunc(uint16_t *data, uint16_t len, uint16_t packetID) {} // (slave handler side)

That way you could setup a switch statement to process arrays differently based on the packerID

Code: 
switch (packetID) {
  case: 0x1234: { 
//do something with *data array

break;
}
 case: 0x7321: { 
//do something else with *data array

break;
}



}

the beauty is during the transaction, the arrays are already transfered before the ISR exits, its just a matter of pushing them to a vector deque, and when the loop() on slave is reentered, the vector is popped to the user handler for processing without interrupting further SPI transactions :)
 
Today I started working on the slave library code and reduced all the tabs to a single one in the IDE, the ISR is also handled by the library as well.

this is what I have working now in the IDE

Code: 
#include <spi_controller_slave.h>

spi_controller_slave slave = spi_controller_slave("SLAVE", "STANDALONE");

void setup() {
  slave.begin();
}
void loop() {
  Serial.println(millis());
}

minimal spi slave setup: (only slave registers are configured, ready to use)
Code: 
#include <spi_controller_slave.h>

spi_controller_slave slave = spi_controller_slave("SLAVE", "STANDALONE");

void setup() {
}
void loop() {
}

pretty simple and works well using same videos above as a reference

I might combine the master & slave into a single library H/CPP file, should we target a new name for this project since it's a remake of the original spi_controller, but more advanced?

BTW: slave.begin() is not needed, it's only needed to initialize all teensy's interface ports to recommended state (which doesnt work from within constructor), the SPI0 ISR and port registers are set from within the constructor, and from the tests, it seems to be successful in setting the registers before even setup() is called. So if you plan on using the slave shared with other code, you can omit port defaults and either activate them remotely or use them locally as you wish with your own code

even while Serial.println(millis()); is running a tight loop on slave, the led keeps toggling same pace while serial monitor is being flooded while the master is also flooding the led toggling of the slave and running mqtt :)
 
Last edited:
Post #45 is my favorite post so far :) - I can fully understand it - and it was in my original 'spec' :cool: Will be good to see the code for full context and set it up when I have time.

Naming isn't always easy? - SPIMS_controller. Seems like it has some key 'packet transfer' feature not 'controller'? SPI_MStransfer?

Nice that it has hot recovery! CRC checking and RESEND! Seems like a good complete solution! Also it could be expanded to "SPI broadcast" where Master enabled multiple CS and all the slaves just stayed quiet on MISO? You could quickly queue data to a bunch of Teensy units doing a common task with a set of data on a 'common clock' .

Post #40 shows timing - must have been a minimal packet size? Wondering about timing of packets of ~60. How many 60 byte packets can be sent in 1 second at fastest rate? I suppose that be would about 100% CPU on both ends?
 
The timing was based on 5 bytes from master -> slave, processing the toggle, and 6 bytes return packet, with validated crc, (2x because its a read&write call); so the timing was both for the toggling as well as both transfers to read the led and write back flipped value, so 2 arrays being send, 2 arrays returned, quad verification :)

broadcast may be implemented with a different addition, however, would most likely be like a UDP implementation, which would be fire and forget, except the slave end will only take in CRC validated packets, if this is a wrong idea let me know ;)

The design for the master to slave capture must toggle the CS line for a new data capture, only the slave does circular buffer transmission endlessly until the master drops the line when the payload and CRC passes, all good.

i'd have to test a 60 byte payload to test the timing
 
I figured the packets were small given the small change in time at faster SPI speeds :)
Yes, the broadcast idea looks like you noted.

My use case was to minimize Master involvement - fire and forget is all I was expecting, you did the over engineering up front ;) With CRC the Slave can decide if it got a valid packet and ignore it if bad.
 
Before I did the pinMode function, I used it to create a random generated 100 byte array validated by CRC to slave, and return an array back, of same size and validation, without running remote functions as a test
Results: ?

4MHz 100 bytes -> slave & slave -> master 100 random bytes, CRC both ways, we're at 1270 micros


fire and forget array (100bytes) from master -> slave == 794 micros!

im getting 157micros for:
Code: 
teensy_gpio.digitalWrite(13, !teensy_gpio.digitalRead(13));
and 59 micros for pinMode:
Code: 
teensy_gpio.pinMode(13, flip = !flip);
 
Last edited:
Status
Not open for further replies.
Back
Top