Forum Rule: Always post complete source code & details to reproduce any issue!
Page 1 of 3 1 2 3 LastLast
Results 1 to 25 of 64

Thread: Lightweight Teensy3.x Fibers Library (C++) Available

  1. #1
    Member
    Join Date
    Apr 2014
    Location
    St. Catharines, Ontario, Canada
    Posts
    78

    Lightweight Teensy3.x Fibers Library (C++) Available

    (Edit)The Fibers class has become a C++ template class(/Edit)

    For those who are looking for coroutine support without the use of an RTOS, I have made the initial version of the Fibers library in my git repository at:

    git@github.com:ve3wwg/teensy3_lib.git

    See subdirectory teensy3/fibers.

    This is a C++ Fibers template class library for use on the Teensy-3.x. This will only appeal to the non-Arduino IDE users unfortunately, since it is tricky to build into that toolchain (but can be done if someone wants to tackle it).

    Additionally, this library requires a symbol to be added to the mk20dx256*.ld loader script. This is necessary so that it can optionally determine how much stack a given fiber (or main routine) uses. Look for instructions at the end of the file fibers.hpp.

    Library Notes:

    In your main program, or suitable place:

    Code:
    #include <fibers.hpp>
    
    Fibers<> fibers;     // Make global to allow other modules to invoke fibers.yield()
    By default, the template class Fibers declares room for 15 fibers + main (16). To explicitly declare this use:

    Code:
    Fibers<16> fibers;
    If you want a non-default main stack size, use:

    Code:
    Fibers<> fibers(2000);  // Use 2000 bytes for main fiber stack (bytes)
    Create a new Fiber

    From any fiber:

    Code:
    fibers.create(foo,foo_arg,3000);  // Returns a fiber ID as unsigned
    Runs foo as a coroutine:

    Code:
    void foo(void *arg) { 
    ... 
    }
    Yield

    To yield control to the next fiber in round-robin fashion, from main fiber or a coroutine, simply:

    Code:
    fibers.yield();
    Determining Stack Size

    To determine the stack size(s), instrument the Fibers class first:

    Code:
    Fibers<> fibers(2000,true);    // true = "instrument" enable
    Allow your fibers to run for a while and at some point do:

    Code:
    bytes[0] = fibers.stack_size(0);   // Approx main stack bytes used
    bytes[1] = fibers.stack_size(1);   // Approx fiber #1 stack bytes used
    ...etc...
    It is not strictly necessary to create a fiber, if you just want to determine the main fiber's stack usage. In this case, just:

    Code:
    Fibers<> fibers(main_stack_size,true); // Instantiate with "instrumentation" enabled
    After main has run for a while, or prior to exit:

    Code:
    main_stack_bytes = fibers.stack_size(0);
    Testing

    April 29/2014: This library has been successfully tested on a Teensy-3.1, with the following caveat: Do not allow your fiber/coroutine to exit. This is an area that needs work.
    Last edited by wwg; 04-30-2014 at 02:26 AM.

  2. #2
    Member
    Join Date
    Apr 2014
    Location
    St. Catharines, Ontario, Canada
    Posts
    78
    Update - May 4, 2014


    The library has been enhanced to allow a fiber (coroutine) to return (exit) and remain stopped. In essence, it becomes trapped in a loop that just performs a yield() call. It's state also is changed to FiberReturned. In this state, it is now possible re-use that fiber, by invoking the method Fibers::restart(). Here you can change the function pointer and argument (the existing allocated stack however, cannot be changed).

    This restart capability makes it possible to set up "worker threads" and re-use them upon demand. The fiber must be in the FiberReturned state however.

    There are now additionally a Fiber::state() and Fiber::join() method calls. The FIber::join() is extremely useful for blocking the caller's control until the indicated fiber has terminated.

    All of these details can be found in the fibers.hpp function.


    The previously used assembler module has been replaced with inline asm .cpp module. This should make it easily possible to use from an Arduino sketch (to be tested). A minor modification to the mk20dx256.ld script is required, however to define where the stack begins (for instrumented stack measurements).
    Last edited by wwg; 05-04-2014 at 04:28 AM. Reason: Update

  3. #3
    Member
    Join Date
    Apr 2014
    Location
    St. Catharines, Ontario, Canada
    Posts
    78
    The lightweight fiber library has moved to its own project at github, for easier use by Arduino IDE users:

    https://github.com/ve3wwg/teensy3_fibers

    Git url:

    git@github.com:ve3wwg/teensy3_fibers.git

    There is an example sketch included in the above repo, but I'll copy it here for your reading convenience:

    Code:
    // Example Sketch illustrating the use of the fibers library.
    
    #include <fibers.h>
    
    // Allocate up to main + 5 fibers:
    Fibers<6> fibers(2000); // main loop() is allocated 2000 bytes of stack
    
    int ledm = 13;          // Main LED on teensy 3.x
    int led0 = 0;           // Digital pin 0
    int led1 = 1;           // Digital pin 1
    int led2 = 2;           // Alter to suit
    int led3 = 3;           //  your wiritng preference
    
    uint32_t ffoo = 0;      // fibers.create() index for the foo() fiber
    uint32_t fbar = 0;      // fibers.create() index for bar() etc.
    uint32_t fzoo = 0;
    
    void yield() {
      fibers.yield();      // Yield cpu to next fiber in round-robin sequence
    }
    
    static void
    toggle(int led,int& led_state) {
      led_state ^= 1;                              // Alternate state on/off
      digitalWrite(led,led_state&1 ? HIGH : LOW);  // LED on/off
    }
    
    // First fiber (coroutine)
    static void foo(void *arg) {
      unsigned count = 0;
      int led0_state = 0;
    
      while ( count++ < 35 ) {
        toggle(led0,led0_state);
        delay(230);
      }
      digitalWrite(led0,LOW);
    }
    
    // 2nd fiber (coroutine)
    static void bar(void *arg) {
      unsigned count = 0;
      int led1_state = 0;
    
      while ( count++ < 40 ) {
        toggle(led1,led1_state);
        delay(250);
      }
      digitalWrite(led1,LOW);
    }
    
    // 3rd fiber (coroutine)
    static void zoo(void *arg) {
      unsigned count = 0;
      int led2_state = 0;
    
      while ( count++ < 47 ) {
        toggle(led2,led2_state);
        delay(270);
      }
      digitalWrite(led2,LOW);
    }
    
    void setup() {
      pinMode(ledm,OUTPUT);
      pinMode(led0,OUTPUT);
      pinMode(led1,OUTPUT);
      pinMode(led2,OUTPUT);
      pinMode(led3,OUTPUT);
    
      digitalWrite(ledm,HIGH);
      digitalWrite(led0,LOW);
      digitalWrite(led1,LOW);
      digitalWrite(led2,LOW);
      digitalWrite(led3,LOW);
    
      ffoo = fibers.create(foo,0,1000);  // Create first coroutine foo(0)
      fbar = fibers.create(bar,0,1200);  // Create 2nd coroutine bar(0)
      fzoo = fibers.create(zoo,0,1100);  // Create 3rd coroutine zoo(0)
    }
    
    // main thread
    void loop() { 
      unsigned count = 0;
      int ledm_state = 0;
      
      digitalWrite(ledm,LOW);        // Turn off Teensy LED while main loops
      
      while ( count++ < 20 ) {      // Loop only 20 times in main
        toggle(led3,ledm_state);    // Toggle led3 while main thread runs
        delay(350);
      }
       
      digitalWrite(ledm,HIGH);      // Teensy LED on means it is doing joins
      digitalWrite(led3,HIGH);
      
      fibers.join(ffoo);            // Stop here until foo() returns
      fibers.join(fbar);            // Stop here until bar() returns
      fibers.join(fzoo);            // Stop here until zoo() returns
      
      digitalWrite(led3,LOW);       // Indicate pause before restarts
      
      delay(3000);
      
      fibers.restart(0,foo,0);      // Restart foo(0) at next yield()
      fibers.restart(1,bar,0);      // Restart bar(0) at next yield()
      fibers.restart(2,zoo,0);      // etc..
    }
     
    // End arduino_fibers.ino
    This sketch requires 4 LEDs to be wired up to see the full effect. LEDs 0, 1, and 2 blink as the fibers (coroutines) foo(), bar() and zoo() blink their respective LEDs at slightly different rates. When the given coroutine "returns", the respective LED will be lit solid. The main loop() blinks the LED3 for the shortest period of time. When it exits its blink loop, it then joins with fibers foo(), bar() and zoo() in that sequence. When the joins have completed, all four LEDs 0 thru 3 will be lit for 3 seconds before the sketch repeats.

    The teensy LED (13) is lit as it is doing joins with the other threads. It turns off, while it is performing its own blink action on LED3. I used a bar graph LED (RBG 1000), but you can of course use whatever you have on hand.
    Last edited by wwg; 05-05-2014 at 11:52 AM. Reason: Small typo

  4. #4
    Senior Member+
    Join Date
    Jul 2013
    Posts
    296
    It looks great, as I see in the github description, it's possible now to use it as a normal library in arduino? I'm a bit confused because you said before that it wasn't that easy.

  5. #5
    Member
    Join Date
    Apr 2014
    Location
    St. Catharines, Ontario, Canada
    Posts
    78
    When I first developed the library, I used a separate assembler language module, which I don't believe will be handled by the IDE. I've since converted it to a .cpp module, using inline asm where it was needed. That change made it possible to just #include it into a sketch.

  6. #6
    Senior Member
    Join Date
    Jun 2013
    Location
    So. Calif
    Posts
    2,825
    Read the API... are there API methods for MUTEXs, semaphores, queues, etc? Methods for ISRs to use to put data in queue, set a semaphore, etc?

  7. #7
    Member
    Join Date
    Apr 2014
    Location
    St. Catharines, Ontario, Canada
    Posts
    78
    Note sure about the point that you are making, but this class does not need to, nor is designed to replace mutexes, queues or semaphores. It is simply a way to provide cooperative multitasking in a lightweight object.

  8. #8
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    Hi wwg,

    I just started using your fibers library and I like what i see so far. It's actually the most compatible scheduler for the teensyunio core i've used through your use of yield. Though I'm seeing a possible bug with your example. If I comment out "bar" and "zoo" fibers the foo fiber does not restart. I've narrowed it down to restart fiber routine. In the example you have "fiberx" for "foo" set to 0, isn't that the main thread? So shouldn't the fiberx parameter be ffoo (1)? Here is whittled down example that shows what I mean.
    Code:
    #include <fibers.h>
    
    // Allocate up to main + 5 fibers:
    Fibers<6> fibers(2000); // main loop() is allocated 2000 bytes of stack
    
    uint32_t ffoo = 0;      // fibers.create() index for the foo() fiber
    uint32_t fbar = 0;      // fibers.create() index for bar() etc.
    uint32_t fzoo = 0;
    
    void yield() {
      fibers.yield();      // Yield cpu to next fiber in round-robin sequence
    }
    
    // First fiber (coroutine)
    static void foo(void *arg) {
      unsigned count = 0;
    
      while ( count++ < 35 ) {
        Serial.println("foo");
        delay(230);
      }
    }
    
    // 2nd fiber (coroutine)
    static void bar(void *arg) {
      unsigned count = 0;
      
      while ( count++ < 40 ) {
        //Serial.println("bar");
        delay(250);
      }
    }
    
    // 3rd fiber (coroutine)
    static void zoo(void *arg) {
      unsigned count = 0;
    
      while ( count++ < 47 ) {
        //Serial.println("zoo");
        delay(270);
      }
    }
    
    void setup() {
      pinMode(LED_BUILTIN, OUTPUT);
      while(!Serial);
      delay(100);
      ffoo = fibers.create(foo,0,1000);  // Create first coroutine foo(0)
      fbar = fibers.create(bar,0,1200);  // Create 2nd coroutine bar(0)
      fzoo = fibers.create(zoo,0,1100);  // Create 3rd coroutine zoo(0)
      Serial.printf("ffoo: %i | fbar: %i | fzoo: %i\n", ffoo, fbar, fzoo);
      fibers.join(ffoo);
      fibers.restart(0,foo,0);// error in restarting
      //fibers.restart(ffoo,foo,0);// resumes as normal 
    }
    
    // main thread 
    void loop() {
        int state = fibers.state(ffoo);
        Serial.print("Fiber State: ");
        Serial.println(state);
        delay(100);
    }
    Maybe I'm missing something?

    duff

  9. #9
    Member
    Join Date
    Apr 2014
    Location
    St. Catharines, Ontario, Canada
    Posts
    78
    Quote Originally Posted by duff View Post
    Hi wwg,

    I just started using your fibers library and I like what i see so far. It's actually the most compatible scheduler for the teensyunio core i've used through your use of yield. Though I'm seeing a possible bug with your example. If I comment out "bar" and "zoo" fibers the foo fiber does not restart. I've narrowed it down to restart fiber routine. In the example you have "fiberx" for "foo" set to 0, isn't that the main thread? So shouldn't the fiberx parameter be ffoo (1)? Here is whittled down example that shows what I mean.
    Code:
    #include <fibers.h>
    
    // Allocate up to main + 5 fibers:
    Fibers<6> fibers(2000); // main loop() is allocated 2000 bytes of stack
    ...
    void setup() {
      pinMode(LED_BUILTIN, OUTPUT);
      while(!Serial);
      delay(100);
      ffoo = fibers.create(foo,0,1000);  // Create first coroutine foo(0)
      fbar = fibers.create(bar,0,1200);  // Create 2nd coroutine bar(0)
      fzoo = fibers.create(zoo,0,1100);  // Create 3rd coroutine zoo(0)
      Serial.printf("ffoo: %i | fbar: %i | fzoo: %i\n", ffoo, fbar, fzoo);
      fibers.join(ffoo);
      // fibers.restart(0,foo,0);// error in restarting
      fibers.restart(ffoo,foo,0);// resumes as normal 
    }
    Maybe I'm missing something?

    duff
    No, I think you found typos! Yes, indeed 0 is the main thread. The value used in the restart method should be the index returned in the fibers.create() method. So yes, use ffoo in the example, not zero. I'll fix the example program on github in the mean time.

    Thanks for pointing that out.
    Warren

  10. #10
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    I'm trying to use this on a teensy 3.6, and I'm running into a linker problem. Maybe there's just a line missing in the instructions. I've added at the end of the teensyduino linker script (mk66fx1m0.ld):
    Code:
    	.stack :
    	{
    	    . = ALIGN(4);
            _sstack = .;
    	    . = . + _minimum_stack_size;
    	    . = ALIGN(4);
    	} >RAM
    
    	_estack = ORIGIN(RAM) + LENGTH(RAM);
    (the last line is part of the original file). However, _minimum_stack_size is missing and the linker complains. Do I need to define that in the linker script as well or is there another way to sneak it into the application?

    This is the sketch I'm trying to compile and test:
    Code:
    #include <fibers.h>
    
    Fibers<2> fibers(2000);
    
    void yield() {fibers.yield();}
    
    #define LED LED_BUILTIN
    static void stuff(void* arg)
    {
      pinMode(LED, OUTPUT);
      digitalWriteFast(LED, 1);
      int count = 0;
      while(count < 10)
      {
        delay(250);
        digitalWriteFast(LED, !digitalReadFast(LED));
        count++;
      }
    }
    
    void setup() {
      // put your setup code here, to run once:
      auto fStuff = fibers.create(stuff, nullptr, 1000);
      int count = 0;
      while(count < 10)
      {
        delay(250);
        Serial.printf("c = %d\n", count);
        count++;
      }
      fibers.join(fStuff);
    }
    
    void loop() {
      // put your main code here, to run repeatedly:
    
    }
    One question regarding this sketch: I'm not doing anything in loop(). Is it ok to use fibers just from setup() if it's a "one-shot" application?

  11. #11
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    Quote Originally Posted by christoph View Post
    I'm trying to use this on a teensy 3.6, and I'm running into a linker problem. Maybe there's just a line missing in the instructions. I've added at the end of the teensyduino linker script (mk66fx1m0.ld):

    One question regarding this sketch: I'm not doing anything in loop(). Is it ok to use fibers just from setup() if it's a "one-shot" application?
    I have a version that works that is based off this fibers library here. No linker scripts edits involved either.

  12. #12
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    thanks! That worked well (on Teensy 3.6) and it prints the expected output on Serial.

  13. #13
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    One thing to look out for is you need to call the 'yield' function in any loop you do or the other tasks will be starved. Luckily 'delay(x)' calls yield but delyMicroseconds does not so be careful on how you use that. Another thing is dynamic memory allocations won't work with this scheduler library, so no printf, String, Malloc, New etc... Besides that it works great for many projects I've done. I used it with the Audio Library with no issues since each task at running priority 255 (User Level). I actually used a modified version in my TeensyTracks library to play songs from samples and synthesized Audio Objects.

  14. #14
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    OK that's kinda bad because I need dynamic memory for my graphics library. Why can't I allocate memory dynamically? Can we somehow implement support for dynamic memory?

    Also it doesn't seem to like a task returning, but I'd like to do that as well.

  15. #15
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    Because this uses static memory for your tasks and the dynamic alloction could step all over that area. I did look at the Placement New allocation in that you allocate a memory space globably and do smaller memory allocation from that memory pool.

    If a task retuns it stays in paused state until you start it again.

  16. #16
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    In other words if tasks are created from a memory pool it could allow for dynamic memory with malloc?

  17. #17
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    Quote Originally Posted by christoph View Post
    In other words if tasks are created from a memory pool it could allow for dynamic memory with malloc?
    This might shed some light on it here, i haven't looked into this for awhile. Where do you do malloc calls in a library? If so where I'll take a look.

  18. #18
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    What I wrote above doesn't make sense given your description of a possible workaround.

    However, since malloc works on the heap and the fibers have statically allocated stack space (don't they?), I don't see the reason why malloc shouldn't work. The two regions would only overlap if one of them grows too much, but that is also the case when I don't use any kind of multitasking at all.

    I seem to be missing a crucial point here.

  19. #19
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    Quote Originally Posted by christoph View Post
    What I wrote above doesn't make sense given your description of a possible workaround.

    However, since malloc works on the heap and the fibers have statically allocated stack space (don't they?), I don't see the reason why malloc shouldn't work. The two regions would only overlap if one of them grows too much, but that is also the case when I don't use any kind of multitasking at all.

    I seem to be missing a crucial point here.
    yes it is statically allocated.

  20. #20
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    The article you kinked to above only explains placement new, alignment and related issues, but it doesn't explain why your library doesn't allow me to use malloc.

  21. #21
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    About my application: I have written my own widget classes on top of uGFX, and each widget can have a parent. They are created dynamically and parents can delete their children, so that I can open and close dialogues, for example. I basically do something like
    Code:
    Widget* pW = new Widget(nullptr); // root widget
    Widget* pB = new Button(pW, width, height); // button is now a child of the root widget, and the root widget takes care of deletion
    
    //... let it live for a while
    
    delete pW; // deletes pB as well
    There's no (public) library yet, but I might just upload the code so that you can have a look. It's in an early stage so it might be possible to restructure it entirely to make static allocation possible. As it is, parents delete their children and that makes use of the heap/malloc necessary unless I write my own memory manager. That would probably suck (both the task and the resulting code).

  22. #22
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    I've seen issues with printf so thats why I say that but maybe if your careful it won't hang. I did some tests with malloc and it ran fine but then the US election madness took over, now I'm in shock!

  23. #23
    Senior Member
    Join Date
    Sep 2013
    Location
    Hamburg, Germany
    Posts
    894
    I managed to take dynamic memory out of my widgets, but at the cost of signals and slots. I'll do some more testing with that, and then move on to using them in fibers. You say that your library is based on the Fibers library, but what does it actually provide? Fibers, coroutines, or something else? I'm just curious about the wording...

  24. #24
    Senior Member duff's Avatar
    Join Date
    Jan 2013
    Location
    Las Vegas
    Posts
    1,027
    Its cooperative multitasking, meaning that tasks must yield for other tasks to run so yes these are considered fibers still.

  25. #25
    Member
    Join Date
    Apr 2014
    Location
    St. Catharines, Ontario, Canada
    Posts
    78
    I'm late to the party here (sorry, but been busy).

    The fibres library should work just fine with malloc/free/printf et al. assuming stack and heap are properly set up. Because the context switch only happens when yield() is called, you'll never interrupt printf or malloc type functions. The only thing left that you have to be careful about is what your interrupt service routines are doing. Those occur at any time (when not blocked) and should thus not being doing anything with malloc/free etc.

    Warren.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •