Forum Rule: Always post complete source code & details to reproduce any issue!
Page 3 of 6 FirstFirst 1 2 3 4 5 ... LastLast
Results 51 to 75 of 141

Thread: Circular_Buffer

  1. #51
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    8,597
    H:# would be H:3 for Head at 3 and T:# would be T:1 for Tail at 1. That would show it wraps over the end [if you knew how many elements it had]

  2. #52
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    That is possible yes, anything else to add there?

    btw, the floats/doubles list() threw compile errors when I added list() support, due to the Type not being "int" when calling the array indices, I had to cast alot of (int)s in there to accept it and it works fine now

  3. #53
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    8,597
    Bummer on added forcing (casts). That is all I can see as useful - other than maybe allocated size of the CB - too much gets in the way and slows down - but when something is working funny - it just takes that one value to show what is wrong.

  4. #54
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    no choice to cast. This, also, if you want a circular array of floats/doubles, the "methods" that work for primitives need to be casted to int as well, otherwise it wont accept floats/doubles, and compiler will complain

    It's no biggie to sprinkle (int) for the array indice, we don't really have a choice..

    example to satisfy the compiler:

    Code:
        for ( uint16_t i = 2; i <= (((int)_cabuf[(int)_cbuf[(head)&(_size-1)]][0] << 8*sizeof(T)) | (int)_cabuf[(int)_cbuf[(head)&(_size-1)]][1])+1; i++ ) {
          Serial.print((int)_cabuf[(int)_cbuf[(head)&(_size-1)]][i]); Serial.print(" ");
        } Serial.print("("); Serial.print(((int)((int)_cabuf[(int)_cbuf[(head)&(_size-1)]][0] << 8*sizeof(T)) | (int)_cabuf[(int)_cbuf[(head)&(_size-1)]][1])); Serial.println(" entries.)");
    I didn't colour in all of them, but you get the point

  5. #55
    Senior Member+ defragster's Avatar
    Join Date
    Feb 2015
    Posts
    8,597
    Yeah - casting can be necessary - still a bummer to have to hack them in - especially when it gets involved with so many.

  6. #56
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Tim, like this?

    Code:
      Circular_Buffer<uint8_t, 8> k;
      k.push_back(1);
      k.push_back(2);
      k.push_back(3);
      k.push_back(4);
      k.push_back(5);
      k.push_back(6);
      k.push_back(7);
      k.push_back(8);
      k.push_back(9);
      k.list();
    Code:
    Queue Size: 8
    
    Indice:	[1]	[2]	[3]	[4]	[5]	[6]	[7]	[0]	
    Entry:	 2	 3	 4	 5	 6	 7	 8	 9
    that doesn't look bad... I should do an indice table for the array system...
    Last edited by tonton81; 04-21-2018 at 02:17 PM.

  7. #57
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Like this?

    Code:
    Queue Size: 5
    
    [Indice]  ___________________________Entries:___________________________
    
       0	  0 1 2 3 4 5 (6 entries.)
       1	  6 7 8 9 10 11 (6 entries.)
       2	  12 13 14 15 16 17 (6 entries.)
       3	  18 19 20 21 22 23 (6 entries.)
       4	  24 25 26 27 28 29 (6 entries.)
    I duplicated a few pushes to verify overwrite indice positioning:
    Code:
    Queue Size: 8
    
    [Indice]  [Entries]
    
       3	  18 19 20 21 22 23 (6 entries.)
       4	  24 25 26 27 28 29 (6 entries.)
       5	  18 19 20 21 22 23 (6 entries.)
       6	  24 25 26 27 28 29 (6 entries.)
       7	  18 19 20 21 22 23 (6 entries.)
       0	  24 25 26 27 28 29 (6 entries.)
       1	  18 19 20 21 22 23 (6 entries.)
       2	  24 25 26 27 28 29 (6 entries.)

  8. #58
    Senior Member+
    Join Date
    Jul 2014
    Location
    New York
    Posts
    3,345
    Nice one Tony. And since you are using indices it would work for floats as well?

  9. #59
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    I have to do a few casts for floats, but yes (casts for the indices, not the float itself

  10. #60
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Like this?

    Code:
    Queue Size: 7
    
    Indice:      [0]	      [1]	      [2]	      [3]	      [4]	      [5]	      [6]	      
    Entry:	 -3.1415901	 -12.3456001	 -78.9123383	 -7.9123402	 -11.9123402	 -58.9123383	 -18.9123402

  11. #61
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Fiddling around with it I came to this autoformatting based on floats/doubles vs ints:

    floats/doubles:
    Code:
    Queue Size: 7
    
    Indice:  	[0]      	[1]      	[2]      	[3]      	[4]      	[5]      	[6]      	
    Entries:	-3.1415901	-12.3456001	-78.9123383	-7.9123402	-11.9123402	-58.9123383	-18.9123402
    other primitives:
    Code:
    Queue Size: 8
    
    Indice:  	[1]      	[2]      	[3]      	[4]      	[5]      	[6]      	[7]      	[0]      	
    Entries:	2		99		4000		399		4000		4000		4000		4000
    I do believe the '\t' tabbing is working correctly

    how i determine when to switch printing decimals vs ints? Like this, KISS..

    Code:
        if ( (int)_cbuf[(head+i)&(_size-1)] != _cbuf[(head+i)&(_size-1)] ) { // possible float?
          Serial.print(_cbuf[(head+i)&(_size-1)],7); Serial.print("\t");
        }
        else {
          Serial.print(_cbuf[(head+i)&(_size-1)]); Serial.print("\t\t");
        }
    if the value does not match itself, it must mean the decimals are causing a mismatch, so print 7 decimals for it since its considered a float

  12. #62
    Senior Member+
    Join Date
    Jul 2014
    Location
    New York
    Posts
    3,345
    nice Tony. Will be a nice addition to the library. Between this and SPI_MSTransfer you are adding a lot of options and additional power to the Teensies I think.

  13. #63
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    I was pulled out of the house for an 8 IP camera installation so im not home, ill work on the list when i get back

  14. #64
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Code:
    Circular Ring Buffer Queue Size: 7 / 16
    
    Indice:  	[0]      	[1]      	[2]      	[3]      	[4]      	[5]      	[6]      	
    Entries:	-3.1415901	-12.3456001	-78.9123383	-7.9123402	-11.9123402	-58.9123383	-18.9123402	
    
    
    Circular Array Buffer Queue Size: 4 / 4
    
    First Entry: 6    7    8    9    10    11    (6 entries.)
    Last Entry:  24    25    26    27    28    29    (6 entries.)
    
    [Indice]      [Entries]
    
        1		6	7	8	9	10	11	(6 entries.)
        2		12	13	14	15	16	17	(6 entries.)
        3		18	19	20	21	22	23	(6 entries.)
        0		24	25	26	27	28	29	(6 entries.)
    
    Circular Array Buffer Queue Size: 3 / 4
    
    First Entry: 123.4567871    789.0123291    (2 entries.)
    Last Entry:  88.0199966    743.6900024    (2 entries.)
    
    [Indice]      [Entries]
    
        0		123.4567871	789.0123291	(2 entries.)
        1		333.7864380	69.1100006	(2 entries.)
        2		88.0199966	743.6900024	(2 entries.)
    
    Circular Ring Buffer Queue Size: 8 / 8
    
    Indice:  	[1]      	[2]      	[3]      	[4]      	[5]      	[6]      	[7]      	[0]      	
    Entries:	2		99		4000		399		4000		4000		4000		4000
    pop_front(buf,size) and push_back(buf,size) was casted to support array'd floats, in order to test the list() above..

    Others would need to be updated too if floats are involved for array usage
    Last edited by tonton81; 04-22-2018 at 12:46 AM.

  15. #65
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Code:
      Circular_Buffer<float, 4, 6> flt;
      float _f[] = { 123.456789, 789.012345 };
      float _f1[] = { 333.78643, 69.11 };
      float _f2[] = { 88.02, 743.69 };
      flt.push_back(_f, 2);
      flt.push_back(_f1, 2);
      flt.push_back(_f2, 2);
      flt.list();
    
      float dump[2];
      flt.pop_front(dump, 2);
      Serial.print(dump[0], 7); Serial.print(" : "); Serial.println(dump[1], 7);
    Output:
    Code:
    Circular Array Buffer Queue Size: 3 / 4
    
    First Entry: 123.4567871    789.0123291    (2 entries.)
    Last Entry:  88.0199966    743.6900024    (2 entries.)
    
    [Indice]      [Entries]
    
        0		123.4567871	789.0123291	(2 entries.)
        1		333.7864380	69.1100006	(2 entries.)
        2		88.0199966	743.6900024	(2 entries.)
    123.4567871 : 789.0123291

  16. #66
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Tim, i committed the list() with float array support for push_back and pop_front

    and check this post for another feature:
    https://forum.pjrc.com/threads/50008...l=1#post176931

  17. #67
    Many thanks for your great work.
    I'm trying to use it to aid re-sync of Can Bus data with a remote sensor signal that has huge latency. I'm slowly getting there, maybe.

    The circular buffer example does not compile on a T3.2 but if I change:
    Code:
    Circular_Buffer<uint32_t, 32, 250> print_test;
    to
    Code:
    Circular_Buffer<uint32_t, 32, 220> print_test;
    it then squeezes into the smaller memory of the 3.2

    I struggled a bit with the documentation and some of the syntax, but guess it was never intended for someone of my calibre ( I had to google Pop Push, and FILO or is it LIFA), and it is helping me to raise my game.

    Separate(d) examples for the Ring buffer and the Array buffer would help, when the Giants shoulders are this high, I need little steps between the ladder rungs and Sat-Nav-like documentation.

  18. #68
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    yes indeed, your initializing a array buffer of 32 arrays of 220 x uint32_t’s, this will surely eat alot of ram
    32x220 == 7,040. The library supports pushing to the back or the front, including dequeueing from front or back, and supports FIFO and LIFO capability this way with all buffers.

    are you looking for a ring buffer or circular ring buffer? do you really need 32bit primitives for your data? canbus should be 8 bits wide, as well as most sensors

    using uint8_t will save you 3x more ram, since you can have multiple buffers you can have different ones depending on your usages
    Last edited by tonton81; 04-27-2018 at 11:07 PM.

  19. #69
    I was just running the examples to try to get a little understanding of what was going on.
    I'm not sure what difference is between a ring buffer and a circular ring buffer? If you meant Ring or Array, then I'm pretty sure the Array functions are what I need.
    I have some of your Circular Array methods working using a uint8_t array of 8 bytes, this just covers the signals/message which is between 8 and 64bits wide but is as you correctly say always received in multiples of 8bits.
    When I decode a message it can start or end anywhere within the 64bits and can be Intel or Motorola or Backwards, so I do quite a bit of byte re-ordering / bit-shifting to get what I want after pulling from the queue.

    The TimeStamp and I.D are uint32_t which could possibly be handled with multiple buffers, but I'm doing OK without these at the moment. I can have a separate buffer for each ID that I'm interested in (only a couple), and time is just a relative thing isn't it?

    The one thing I'm struggling with is how to access and read a specific entry / index or alternatively Flush() or Clear() to an array entry. I'm currently just .readBytes() multiple times and discarding to get to the point in the buffer that I want, but I'm not sure if I will lose my 'alignment' when an array is shorter than 8 bytes.
    I've probably got to do some more reading and experimenting.....

  20. #70
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    for the circular array you got the constructor correct.

    to add an item to queue you can simply send its buffer in using push_back method:
    Code:
    uint32_t myBuf[24]; // buffer of 24 uint32_t’s
    // some code to setup your buffer data
    print_test.push_back(myBuf,24);
    the library is smart enough to return the length of the array your attempting to pull out, so you can do:
    Code:
    uint32_t data[print_test.length_front()];
    print_test.pop_front(data, print_test.length_front());
    if you prefer to read the data without pulling it out of the buffer, you may do so using the peek_front() or front() method:

    Code:
    print_test.peek_front()[8]; // shows you value in the 7th indice of the array queue
    or using a for loop to print out the first queue:
    Code:
    for ( uint16_t i = 0; i < print_test.length_front(); i++ ) {
      Serial.print(print_test.front()[i]);
      Serial.print(“ “);
    }
    Serial.println();
    Similarily, you can do the same from the back of the queue, using back(), peek_back(), and back_length()

  21. #71
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    I updated the library on github but I will post a test version here, the primary indexing array which was initialized between 0 -> max for the circular array system is not used anymore. I made the head and tail pointers directly without using the indexer array as a reference. Tests have been successful with a display library, SPI_MST, and some float/uint16_t/uint8_t examples as well, but would like some testing. It's more better this way than referring to an extra array as a reference, probably more efficient too since it's less lookups.

    This version is the github copy, with the head & tail pointers used as the indexing directly, rather than using the secondary buffer

    in other words, cabuf does not rely on cbuf anymore.
    Differences can be seen here: https://www.diffchecker.com/qEFtp8Sv

  22. #72
    Senior Member+
    Join Date
    Jul 2014
    Location
    New York
    Posts
    3,345
    Ok. Back at it. Decided to go back to basics with you example in post #65. Still playing but with the example I am using kind of illustrates how the circular buffer works if you just keep pushing values to it:
    Code:
    #include "circular_buffer.h"
    
    void setup() {
      Serial.begin(115200);
      delay(5000);
      // List example
      Circular_Buffer<float, 4, 6> flt;
    
      float _f[] = { 123.456789, 789.012345 };
      float _f1[] = { 333.78643, 69.11 };
      float _f2[] = { 88.02, 743.69 };
      float _f3[] = { 223.456789, 989.012345 };
      float _f4[] = { 433.78643, 79.11 };
      float _f5[] = { 99.02, 843.69 };
    
      flt.push_back(_f, 2);  flt.list();
      flt.push_back(_f1, 2);  flt.list();
      flt.push_back(_f2, 2);  flt.list();
      flt.push_back(_f3, 2);  flt.list();
      flt.push_back(_f4, 2);  flt.list();
      flt.push_back(_f5, 2);  flt.list();
    
    
      float dump[2];
      flt.pop_front(dump, 2);
      Serial.print(dump[0], 7); Serial.print(" : "); Serial.println(dump[1], 7);
    
    }
    
    void loop() {
      // put your main code here, to run repeatedly:
    }
    Should do a push back test to see how it works. I also assume that since you are using flt.pop_front(dump, 2); you could use dump[n] to access individual elements in the list. Now, I have to look up if there is way to get a particular indicia from the buffer (think you did that some where or is that where find is going to come in ?).

    Mike

  23. #73
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Yeah the find would have to either return an indice, or it sets up an internal iterator that enables some methods to behave differently

    if you dont want to pop, you can read the front or back using front(),back(),peek_front(),peek_back() as needed without dequeueing

    Ex.

    Code:
    flt.length_front(); <-- this would be the length of the queue your reading
    flt.peek_front()[2]; // read the first queue item 3rd position

    you could do something like this (psuedo code):
    Code:
      for ( uint16_t i = 0; i < flt.length_front(); i++ ) {
        Serial.print(flt.peek_front()[i],7); // print 7 decimals
        Serial.print(" ");
      }
      Serial.println();
      // that wont remove it from queue, but it'll print it. You'll have to pop it to remove it.
      flt.pop_front();
    The ::remove(indice) method works using the ::list() indice position, just need to feed the value to ::remove()

    Code:
    flt.remove(1); // removes indice 1 ( actually array 2 ) from queue
    the tail is untouched during the removal. all queues before the removal are shifted up with memmove, the head is increased, and available is decreased, this gives the producer his rights to the tail
    Last edited by tonton81; 04-30-2018 at 09:05 PM.

  24. #74
    Senior Member+
    Join Date
    Jul 2014
    Location
    New York
    Posts
    3,345
    Thanks. I am going to try some of those methods and then try to document what I am doing. Then you have another example.

    By the way did you notice that CB declaration is in the setup and not in the header. I wonder what would happen with MSTransfer if I redefined:
    Code:
    Circular_Buffer<uint16_t, SPI_MST_QUEUE_SLOTS, SPI_MST_DATA_BUFFER_MAX> SPI_MSTransfer::mtsca;
    this to different dimensions in the set up.... have to try it one of these days.

  25. #75
    Senior Member
    Join Date
    Dec 2016
    Location
    Montreal, Canada
    Posts
    3,020
    Setup?

    It's static in the H so it must be declared as well in the CPP, thats normal

    you have it in setup? hehe

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •