Is there a "line receiver" library for e.g. the serial ports?

Status
Not open for further replies.
C

christoph

Well-known member
The TinyGPS library updates the internal state when a line has been fully received. The user code just pushes characters into the object, like so:
Code: 
#define RXPIN 3
#define TXPIN 2
SoftwareSerial nss(RXPIN, TXPIN);
void loop()
{
  while (nss.available())
  {
    int c = nss.read();
    if (gps.encode(c)) // <---- this is what I mean by "pushing characters into the object"
    {
      // process new gps info here
    }
  }
}
Is there a library that provides just this feature, working like this:
  • create a "line receiver" object and give it a callback to use when a line is complete
  • push in received characters
I know this is actually dead simple and I write this kind of code quite often - so I think there should be a library around that facilitates this common task.
 
I'm betting you have searched pretty thoroughly and your question suggests to me that the answer is... not yet :)

I wonder if a flag on the Serial objects that a carriage return or line feed has just entered the buffer might suffice, escape would be a good flag to have as well - I always clear their buffers as .available() occurs so I haven't checked; I assume they have a 'buffer-nearly-full' flag?

Hmmm, sorry, sorta derailing your topic, especially if I ended on that previous paragraph's note;


I'm tempted to do it just for the sheer exercise - if nobody links to, or posts, anything suitable in the next couple/few days feel free to remind me I practically volunteered to make you one :D
 
To be honest I didn't do a thorough search because I couldn't think of a good term to search for. That's why I just described what I'm looking for. I can also write one, so let's work together if you want to!

I wonder if a flag on the Serial objects that a carriage return or line feed has just entered the buffer might suffice, escape would be a good flag to have as well
Click to expand...
What do you mean by "escape flag"? A flag indicating that there's an escape character in the buffer somewhere? I'd think that Paul wouldn't like to touch the Serial objects for those CR/LF/ESC flags, but I might be wrong.
 
lol; I couldn't think of any terms that wouldn't return too many redundant results either.

Whether or not we reach for crayons and make anybody (including ourselves) muck about with those stream objects, by drawing a gun for them or something, it shouldn't be too dramatic to at least attempt cooperation on an object that does a handful of handy things - we could just do it publicly in this thread, if you don't object (lol, pardon [not-even]pun); if push comes to shove: give this thread at least a couple of days to see if anyone happens know of, or have, something suitable.

I should go beddies, hush for now whooshing world! :)
 
I have also done my own stuff for this in the past. Example: Serial monitor as part of my Phoenix code base, that when idle, it checks to see if there are characters on the serial port and when it gets a CR it acts on what is in the buffer. No big deal. Now if I was simply doing this as part of the main line of code, I have also tried using built in code (Serial class).
Code: 
char abBuffer[80];
char cnt;
Serial.setTimeout(500);  // wait up to 500ms
cnt = Serial.readBytesUntil(13, abBuffer, sizeof(abBuffer));
 
It polls Serial.available() and the elapsed time. Is this what you're asking? There also a version for reading a number.
 
Something along these lines, but
  • Could be more versatile (detect different/custom line endings, skip leading whitespace, ...)
  • Shouldn't use malloc() - the size could be a template parameter
  • Overflow behavior might be odd for some users: the leading part of overflowing lines is simply ignored and overwritten
  • No timeout yet

Code: 
class LineReceiver
{
  public:
    typedef void (*callback_t)(const char*);
    static const size_t default_size = 100;

    LineReceiver(size_t n = default_size, callback_t callback = nullptr)
    {
      init(n, callback);
    }

    LineReceiver()
    {
      init(default_size, nullptr);
    }

    bool push(char c)
    {
      if (c != '\n')
      {
        if (p_ < &buf_[size_-1])
        {
          *p_ = c;
          p_++;
        }
        else
        { // overflow: discard, reset line MAYBE ADD AN OVERFLOW FLAG?
          reset();
          terminate();
        }
      }
      else // line ends
      {
        terminate();
        reset();
        if(callback_ != nullptr)
        {
          callback_(buf_);
          terminate();
        }
        return true;
      }
      return false;
    }

    const char* get() const
    {
      return buf_;
    }

    size_t size() const
    {
      return size_;
    }

    size_t length() const
    {
      return p_ - buf_;
    }

    size_t space() const
    {
      return size() - length() - 1;
    }
    
    ~LineReceiver()
    {
      free(buf_);
    }
  protected:
  private:
    void init(size_t n, callback_t callback)
    {
      buf_ = (char*)malloc(n);
      size_ = n;
      callback_ = callback;
      reset();
      terminate();
    }

    void reset() // could also be public
    {
      p_ = buf_;
    }

    void terminate()
    {
      *p_ = 0;
    }

    callback_t callback_;
    char* buf_;
    char* p_;
    size_t size_;
};

void lineReceived(const char* line)
{
  Serial.printf("callback; Line: %s\n", line);
}

void setup()
{
}

void loop()
{
  // switches back and forth between two line receiver object, one with a callback installed

  static LineReceiver line(10);
  static LineReceiver lineCallback(10, lineReceived);
  static LineReceiver* p = &line;
  if (Serial.available())
  {
    if (p->push(Serial.read()))
    {
      if(p == &line)
      {
        Serial.printf("Line: %s\n", p->get());
        p = &lineCallback;
      }
      else
      {
        p = &line;
      }
    }
  }
}
 
Last edited:
Thinking about "enlarging the bore" a bit I realized that this seemingly simple thing (assemble single characters into lines) can be quite compilcated. I'll leave it at that for now.
 
I like that christoph, sort of daunting to contribute now - pretty sure it won't look anywhere near as neat if I do ;)
 
Go ahead and contribute! Really, it doesn't have to look neat. Make whatever suggestion you would like to see considered.
 
regarding malloc(), why not just put the onus on the end user to supply the buffer?
Code: 
#include <LineReceiver.h>

void lineReceived(const char* line)
{
  Serial.printf("callback; Line: %s\n", line);
}

char myBuffer[255];

LineReceiver myLines(&myBuffer,256,myCallback);

void setup() {

...
Allthough the callback routine wouldn't need to pass the pointer so much, unless you are doing something more advanced with the buffer than I see a need for so far.
 
Last edited:
I merely posit it as an alternative to malloc() which shouldn't pose that great a challenge for folks to figure out how to use.

Although I've always made the buffer belong to the object which is going to manipulate it most (obvious owner, right?), when this alternative occured to me I did think there may be some benefits for the end user of the object if they get to own the buffer instead; also, I couldn't see how who owns it becomes a problem till somebody does something fairly daft with it.

It could be an alternative constructor in your current version, for people who pedantically avoid playing with malloc :)
 
The alternative constructor for user-supplied buffers wouldn't be a problem at all. One potential source of errors is the extra size argument, which might not be the actual size of the buffer - depending on the user's attention to this detail and how it is actually set. std::array has size information built-in, that's why I prefer it over plain arrays of char.

The destructor would be a problem though, because simply keeping a pointer to the buffer (same for a dynamically allocated buffer and a user-supplied buffer) wouldn't be sufficient any more. The destructor cannot simply call free() in that case, so a flag indicating if we are owning the buffer is necessary.
 
Here's the code that I use for interactive text entry over a serial connection with support for backspace, timeout, and restricting the character set based on a predicate, e.g. digits only, etc.:

Code: 
void readEdit(char* buffer, bool& isNotTooLate,
              const char* end, IntPredicate isIncluded, uint32_t timeout)
{
   const char ASCII_BS = '\b';
   const char ASCII_DEL = '\x7f';
   char* p(buffer);
   elapsedMicros elapsed;
   do {
      if (Serial.available()) {
         char in(Serial.read());
         if (isIncluded(in)) {
            *p = in;
            Serial.write(*p);
            ++p;
         } else if (in == '\r') {
            *p = 0;
            break;
         } else if (p != buffer && (in == ASCII_BS || in == ASCII_DEL)) {
            Serial.write(ASCII_BS); // back up one
            Serial.write(' ');      // erase by writing a space
            Serial.write(ASCII_BS); // back over the space
            *p = 0;
            --p;
         }
      }
      isNotTooLate = elapsed < timeout;
   } while (p != end && isNotTooLate);
   *p = 0;
}


bool readLine(char* buffer, size_t n, uint32_t timeout) {
  bool isNotTooLate(true);
  if (!n) {
    // No buffer space. You ask for nothing? You got it!
    return isNotTooLate;
  }
  const char* e(buffer + n - 1);


  readEdit(buffer, isNotTooLate, e, isprint, timeout);


  return isNotTooLate;
}


bool readNumber(uint32_t& v, uint32_t timeout) {
  bool isNotTooLate(true);
  char buffer[32];
  const char* e(buffer + sizeof buffer - 1);


  readEdit(buffer, isNotTooLate, e, isdigit, timeout);


  v = strtoul(buffer, 0, 0);
  return isNotTooLate;
}
 
pictographer, thanks! I already saw your code as part of didah, but it is much more than what I had in mind. Of course, it packs a lot more features and is already tested probably a lot more than my code. I was not thinking about user-interactive mode, but line-based devices such as GPS and small co-boards that send human-readable data line by line. I think we all build those things from time to time: Create some slave-like gadget, let it spit out readable data, and interpret that on a central "main" device.

So interactive mode might be over the top. A timeout, however, is probably a good idea. I'm just not sure how to handle such an event - flag it? Or a callback? The didah code uses the return value of a blocking function, which is totally different from my approach. My main method for LineReceiver acts immediately, and returns a state (line complete or not) and optionally calls a function. While we're at it: Just calling a function is somewhat restrictive; I'd prefer a more versatile delegate approach.
 
Flag is easy but why offer a destructor at all? I've never destroyed an object doing a job like the job this object is intended for in an embedded context; do you really ever instantiate similarly tasked objects and then destroy them when writing in embedded context?

As for a timeout I'm going to suggest you show the end user of the object how to manage timing out externally from the object because the alternative is to add time tracking variables to the object and then arrange some way for the oject to have a function called regularly when there are no incoming characters - timed isr for such a task is just YUCK (although premise is there to use a smallish counter instead of actual seconds in a float or something), making the end user call regularly when there is no new data isn't as awful as the isr route but, seriously, how would making the end user of the object time out externally from the object be a bad thing?

Alternatively, you could keep an elapsedMillis in the object, in the 'incoming chars routine', when the user calls with the next incoming character check the elapsedMillis for exceeding the limit and reset the buffer if it has, place the new character at the current pointer of the buffer and reset the elapsedMillis to 0.

Single callback function is not that restrictive if you hand end user tools to employ in it, although I can see premise for handing the object a char array containing 'keyword1\0keyword2\0keyword_last\0\0' and another array of a set of function pointers aligned with the keywords for a reasonably obvious outcome but I think this sort of thing more reasonably handled outside of the object.


I can see plenty of premise for adding functionality to it without even extending the current object, a couple of parsing and converting routines included in the source file(s) for the object and when the object is in use by including its .h file those helper routines become available.

I've some reasonably compact/efficient code for parsing commands and converting numeric variables back and forth to/from text that I am not too ashamed of, perhaps I will clean them up (if they need it, while since I looked at them) and just post them later in this thread, with a little help on using them.
 
I'll answer in more detail later because I only have my phone right now, but re destructor: simply omitting it would be really bad style and I have in the past used an object like this one with temporarily enabled sources of data, and improper cleanup would have led to quite some surprises I guess
 
Granted, especially in light of using malloc(), it is pretty dumb looking not to have a destructor but I won't admit my suggestion of ditching it in this context is that ridiculous yet :D

Edit: Tho, on the other hand, why ditch it when it isn't that hard to make?
 
Yes, if the object is never destructed it really doesn't matter if the destructor is implemented correctly (or if it is implemented at all).

External handling of timeouts: I like that idea! No extra memory needed which might be unused in some scenarios, and how a timeout is handled should be up to the user anyway. I would have placed an elapsedMillis on the class that is evaluated and - if ok - reset whenever a character is added.

Also having some supplementary functions and classes for handling input might be a good idea, but I'm afraid they would end up simply wrapping a lot of the character handling functions from <cstring>, <ctype> and similar headers. But let's see what comes, there's a repo and whoever wants something added can open a feature request.

I'm still not sure if I want LineReceiver to own a buffer, or not, or to do it optionally. I could use a shared_ptr but that sounds so fancy...
 
Status
Not open for further replies.
Back
Top