PaulStoffregen
Well-known member
For quite some time I've been considering the need for an event callback API in Teensy's core library.
I want to collaborate with Arduino. Eventually I want to contribute this to all Arduino boards, or perhaps all 32 bit boards. But working with Arduino on even fairly simple features often goes very slowly and ends up stalled by many people disagreeing or bikeshedding. My goal is to move quickly and perhaps get a first experimental version into the 1.38 release. So instead of their developer list, I'm going to first compose my thoughts here and ask for your feedback. Contributing back to Arduino can come later...
In designing any API, you can take a top-down approach focusing on the features and functions provided to users, or a bottom-up method focusing on the implementation details. Some amount of bottom-up planning is needed to end up with an efficient implementation, but I'd prefer to start from the top where user needs come first. Over the last year I've talked with several people in person, where usually there's a strong tendency to immediately focus on the difficult low level details. Those things are important, but please for a moment let's think top-down...
Event processing is meant to provide at least 2 major features.
1: Timing sensitive functions can post events to be later processed by other functions, which typically aren't as timing sensitive. In particular, events provide a way for interrupt handlers to schedule non-interrupt functions to later run. Non-interrupt code can generate events too, but the interrupt to non-interrupt case is likely to be the most common usage.
2: A consistent API allows libraries that produce events to be connected to other libraries or user-written code that consumes events. Of course both sides have to agree on the meaning of the events to actually do something useful, but the general idea is some degree of "pluggable" modularity between separately developed code.
This ought to sound pretty similar to GUI programming, where controls or widgets generate events and most programming is done by attaching handler functions. Obviously I don't want to make this as large & complex as Microsoft WIN32. Perhaps events only need to be of a single no-data type? Or maybe some of the features of traditional GUI events would be valuable? I think it's at least worth considering.
Of course this needs to be kept fairly lightweight at runtime, which is where bottom-up design is needed. Posting an event needs to be quick. For example, the Wire library might offer a non-blocking endTransmission() function with posts an event when the transmission actually finishes. Long-term I want to see this event API become the de-facto way asynchronous & non-blocking I/O is done on Arduino libraries.
When events are actually processed involves many trade-offs. The more immediately an event handler function is called, the more disruptive it is to the flow of the rest of your program. Perhaps this is configurable? Maybe when attaching you can specify if you wish to handle events immediately, even at interrupt context. Or maybe there's an option for handling at a low priority interrupt, similar to how the audio library uses 2 interrupt priority levels. Of course, the option to handle events from main program context needs to be present and probably the default. Maybe even there a couple options make sense, like calling from yield() versus calling only after loop() completes? When used together with an RTOS, options for assigning the thread to receive the function call would make sense.
From a style point of view, the code people actually write in sketches wants to have the minimalist simplicity you'd expect from Arduino.
Many low-level implementation details exist. I've tried to keep this top-down. But believe me, I've thought a lot about little issues, like locking to prevent recursive event calls, linked lists between C++ objects to avoid malloc/new dynamic memory, calling static vs C++ class member functions, and so on. This stuff matters, so please by all means comment, but let's try to do so with can-do attitude to solve technical challenges, and with keeping the big picture in mind.
I want to collaborate with Arduino. Eventually I want to contribute this to all Arduino boards, or perhaps all 32 bit boards. But working with Arduino on even fairly simple features often goes very slowly and ends up stalled by many people disagreeing or bikeshedding. My goal is to move quickly and perhaps get a first experimental version into the 1.38 release. So instead of their developer list, I'm going to first compose my thoughts here and ask for your feedback. Contributing back to Arduino can come later...
In designing any API, you can take a top-down approach focusing on the features and functions provided to users, or a bottom-up method focusing on the implementation details. Some amount of bottom-up planning is needed to end up with an efficient implementation, but I'd prefer to start from the top where user needs come first. Over the last year I've talked with several people in person, where usually there's a strong tendency to immediately focus on the difficult low level details. Those things are important, but please for a moment let's think top-down...
Event processing is meant to provide at least 2 major features.
1: Timing sensitive functions can post events to be later processed by other functions, which typically aren't as timing sensitive. In particular, events provide a way for interrupt handlers to schedule non-interrupt functions to later run. Non-interrupt code can generate events too, but the interrupt to non-interrupt case is likely to be the most common usage.
2: A consistent API allows libraries that produce events to be connected to other libraries or user-written code that consumes events. Of course both sides have to agree on the meaning of the events to actually do something useful, but the general idea is some degree of "pluggable" modularity between separately developed code.
This ought to sound pretty similar to GUI programming, where controls or widgets generate events and most programming is done by attaching handler functions. Obviously I don't want to make this as large & complex as Microsoft WIN32. Perhaps events only need to be of a single no-data type? Or maybe some of the features of traditional GUI events would be valuable? I think it's at least worth considering.
Of course this needs to be kept fairly lightweight at runtime, which is where bottom-up design is needed. Posting an event needs to be quick. For example, the Wire library might offer a non-blocking endTransmission() function with posts an event when the transmission actually finishes. Long-term I want to see this event API become the de-facto way asynchronous & non-blocking I/O is done on Arduino libraries.
When events are actually processed involves many trade-offs. The more immediately an event handler function is called, the more disruptive it is to the flow of the rest of your program. Perhaps this is configurable? Maybe when attaching you can specify if you wish to handle events immediately, even at interrupt context. Or maybe there's an option for handling at a low priority interrupt, similar to how the audio library uses 2 interrupt priority levels. Of course, the option to handle events from main program context needs to be present and probably the default. Maybe even there a couple options make sense, like calling from yield() versus calling only after loop() completes? When used together with an RTOS, options for assigning the thread to receive the function call would make sense.
From a style point of view, the code people actually write in sketches wants to have the minimalist simplicity you'd expect from Arduino.
Many low-level implementation details exist. I've tried to keep this top-down. But believe me, I've thought a lot about little issues, like locking to prevent recursive event calls, linked lists between C++ objects to avoid malloc/new dynamic memory, calling static vs C++ class member functions, and so on. This stuff matters, so please by all means comment, but let's try to do so with can-do attitude to solve technical challenges, and with keeping the big picture in mind.
Last edited: