Forum Rule: Always post complete source code & details to reproduce any issue!
Results 1 to 25 of 25

Thread: A Second USB_HID On Teensy 3.2

  1. #1
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210

    A Second USB_HID On Teensy 3.2

    I have a project whereby 5120 bytes of sonar data are passed from the Teensy to a Lazarus program running on a laptop via an FTDI USB/serial adapter. The serial connection is very unpredictable with latency problems. I would like to abandon that approach and try USB_HID. I need to send a six byte command out to the Teensy and receive a 64 or 128 byte packet in return. I also need to know when the returning packet is available using an 'while' statement with timeout. I want to keep the existing USB port open for uploading and debugging. I've also considered the Due or Zero with it's native USB port.

    Is this possible and if so, where would I find some info? I've been searching for several days and found only slightly related and very confusing data. The Lazarus end of the link is another problem which I'll have to solve separately. I've used game joysticks in Lazarus but that's only one way with no handshaking.

  2. #2
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    I've made some headway. My wife said 'try this' which I rarely do but this part is working. I used the USBHID Basic example and changed the upload port to Raw_HID then changed the Serial.print to Serial1.print. The FTDI adapter was already connected to RX1/TX1 from the previous serial setup. I can now see text on a terminal monitor since I added a statement at the top of loop(). I wonder if I can move the wires back to RX0/TX0 and not affect the upload and then use the normal Serial.print statements.

    PS
    The normal Serial.print works on the IDE serial monitor so the FTDI is not needed.
    Last edited by Arctic_Eddie; 05-18-2016 at 04:36 PM.

  3. #3
    Senior Member+ MichaelMeissner's Avatar
    Join Date
    Nov 2012
    Location
    Ayer Massachussetts
    Posts
    4,090
    Quote Originally Posted by Arctic_Eddie View Post
    I've made some headway. My wife said 'try this' which I rarely do but this part is working. I used the USBHID Basic example and changed the upload port to Raw_HID then changed the Serial.print to Serial1.print. The FTDI adapter was already connected to RX1/TX1 from the previous serial setup. I can now see text on a terminal monitor since I added a statement at the top of loop(). I wonder if I can move the wires back to RX0/TX0 and not affect the upload and then use the normal Serial.print statements.

    PS
    The normal Serial.print works on the IDE serial monitor so the FTDI is not needed.
    I suspect you mean use the normal Serial0.print statements. On the Teensy, Serial.print only goes to USB, Serial0.print only goes to RX0/TX0, Serial1.print only goes to RX1/TX1, and Serial2.print only goes to RX2/TX2.

  4. #4
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    I don't see a TX0/RX0 pair on the pinout chart. There is a TX/RX set on pins 3 and 4, two sets of TX1/RX1 and single sets for ports 2 and 3. I'm sending text using Serial.print to the monitor after uploading via Raw_HID and also see it on the TyQt monitor. It shows two entries, RawHID and seremu(open). I don't have anything running that can send USB packets into the Teensy so don't know if that part is running.

  5. #5
    Member
    Join Date
    Apr 2016
    Location
    Honolulu, Hawaii
    Posts
    41
    The Tx/Rx on pins 3 and 4 are for CAN Bus, not RS serial.

    Pins 5 and 27 are alternatives for Tx1/Rx1 in case pins 0 and 1 are taken for some other purpose.
    Similarly, Pins 26 and 31 (on the back side) are alternatives for Tx2/Rx2 in case pins 9 and 10 are already in use.

    This is the first I've heard of Tx0/Rx0.

    But going back to your original problem: I too had a lot of difficulty moving data at high data rates to a PC over FTDI, but I was able to resolve the problems once I finally understood how to read the serial port properly on the PC. Initially I was losing data and seeing high latencies above 9600 baud, but I currently move something like 800,000 bps without any issues.

    In my case the Teensy was working fine, but when I read data byte by byte (as in all the published examples I could find) the performance was terrible. In order to read multiple bytes you need to provide a buffer of the correct length, and in my environment there was no reliable way to know how many bytes are *actually* available.

    The solution in my case was to set Read_Timeout to zero before opening the port, which causes the serial port to return all available bytes and report how many it gave you. This meant I could just pass a large byte buffer, get all available bytes plus the length, and then move the data into an array of the correct size. When I did this, the performance boost was amazing!

    I'm not suggesting that your problem is necessarily the same as mine, but I am suggesting that the problem likely is on the receiving end because the Teensy is capable of reasonably high throughput using FTDI.

    Finally, I may have misunderstood the issue, but you can use the USB virtual serial port without interfering with the bootloader. The Teensy uploader disables the virtual serial port while uploading, and then your sketch enables the "normal" serial port when you call Serial.begin(). You don't need to use any Teensy pins in this case because the virtual serial port is entirely running over USB. People have achieved 2Mbps transfers over the virtual serial port, so you should have plenty of headroom.

    gl!
    Last edited by CraigF; 05-18-2016 at 10:50 PM. Reason: added info, typo

  6. #6
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    My problem is very much like yours. I am sending 6 byte command packets and receiving 128 byte data packets at 2.25Mbps. Debug statements shows the data leaving the Teensy but receiving it in Lazarus is intermittent. Each packet uses a buffer array of the correct size. I also use the time out feature but often it seems not to work. Adding a sleep() function helps in some cases but slows everything down. There seems to be an internal buffer larger than the ones I use and sometimes it contains previous data. I've been fighting this for weeks and don't think it will ever work due to the hardware. That's why I decided to try the USB_HID method. My problem now is finding a library or component for Lazarus for the USB object. So far, nothing is clear or working. The functions on this page are just what I need for the Teensy side. Now looking for the similar functions on the Lazarus side.

    http://pjrc.com/teensy/rawhid.html

  7. #7
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    25,082
    Quote Originally Posted by Arctic_Eddie View Post
    I need to send a six byte command out to the Teensy and receive a 64 or 128 byte packet in return.
    I have some bad news for you. This type of send-query-wait-response approach always performs poorly over USB. Changing to HID protocol will not help. Nothing will. You must change your approach. Absolutely nothing will ever make this query-and-wait design ever work well over USB.

    I've also considered the Due or Zero with it's native USB port.
    Those boards definitely will not help! While Due has much faster 480 Mbit/sec speed, its software support is based on inefficient polling. But no matter how efficient the USB stack, query-wait-respond never achieves good performance.

    You must change your approach, to always send data as soon as possible. If you require acknowledgements, you must transmit them after the fact. You can add message IDs or other data as needed. I know this isn't as simple as handling 1 thing at a time in your code, but that's simply the way you *must* do thing if you ever want to achieve good performance over USB.

    On the Teensy side, if you're receiving fixed size message, Serial.readBytes() is highly optimized. Reading 1 byte at a time can work at nearly full USB speed (which works out to be about 8-9 Mbit/sec actual data rate with USB protocol overhead), but much CPU time is needed to handle the bytes individually. Serial.readBytes() can run at those speeds using only a small fraction of Teensy CPU, which leaves a lot more for you to actually do things with the data.

  8. #8
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    25,082
    Another thing you should know is Windows and Linux are not very efficient if you transmit small data sizes over USB serial. Macintosh will group small write together to efficiently fill up the outgoing (to Teensy) packets. But Windows and Linux do not. Windows has additional limitations in the number of transactions it can handle per USB frame, which makes small write severely slow.

    Here's some benchmarking I did a few years ago. You can find the benchmark code on that page.

    http://www.pjrc.com/teensy/benchmark...l_receive.html

    Maybe this benchmark code could give you a fresh starting point to develop your application to transmit data rapidly to Teensy?

  9. #9
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    25,082
    If you're primarily sending from Teensy to your PC, the good news is Teensy does efficiently fill up the USB packages, like Macintosh does for sending to Teensy. On the Teensy side, Serial.write(buffer, len) is also optimized, so you'll consume less Teensy CPU time that way than writing 1 byte at a time. But that's minor compared to the need to use a streaming approach, rather than query-wait-reply.

  10. #10
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    The serial problem at the moment is on the PC side. Sometimes, old data is not removed from the receive buffer even though it's been read. Even the purge command does not clear it. The libraries for USB_HID do not exist but use a wrapper for the C++ version. That attempt was even worse. I need to investigate the serial problem first before proceeding.

    The handshaking scheme is necessary due to the nature of the data source. The sonar data is accumulated at the shore station by nRF24 from the boat then sends a six byte message to the PC that the data is ready. Thereafter, the PC requests a particular packet from one of the four sonars and expects 128 bytes to be returned. This continues until all data has been recovered. In general, neither side knows when something will happen until informed by the other side.

    I've been trying to find an IDE/RAD software package that has the features of Lazarus(visual Pascal) but using C++. I just installed a trial version of C++ Builder by Embarcadero. It's so bloated with just a portion installed that I'm going to uninstall it without even trying it. I also have Ultimate++ and Code::Blocks to try but don't know if it's an IDE/RAD type of package. Most of these packages are very skimpy on the component menus and object inspector details. Lazarus has everything up front along with an Anchor Editor which I dearly love.

    I'll keep trying until I get something to work. Thank you Paul for a great product and support.
    Ed

  11. #11
    Member
    Join Date
    Apr 2016
    Location
    Honolulu, Hawaii
    Posts
    41

    Cool First things first

    Quote Originally Posted by Arctic_Eddie View Post
    The serial problem at the moment is on the PC side. Sometimes, old data is not removed from the receive buffer even though it's been read. Even the purge command does not clear it. The libraries for USB_HID do not exist but use a wrapper for the C++ version. That attempt was even worse. I need to investigate the serial problem first before proceeding.

    The handshaking scheme is necessary due to the nature of the data source. The sonar data is accumulated at the shore station by nRF24 from the boat then sends a six byte message to the PC that the data is ready. Thereafter, the PC requests a particular packet from one of the four sonars and expects 128 bytes to be returned. This continues until all data has been recovered. In general, neither side knows when something will happen until informed by the other side.

    I've been trying to find an IDE/RAD software package that has the features of Lazarus(visual Pascal) but using C++. I just installed a trial version of C++ Builder by Embarcadero. It's so bloated with just a portion installed that I'm going to uninstall it without even trying it. I also have Ultimate++ and Code::Blocks to try but don't know if it's an IDE/RAD type of package. Most of these packages are very skimpy on the component menus and object inspector details. Lazarus has everything up front along with an Anchor Editor which I dearly love.

    I'll keep trying until I get something to work. Thank you Paul for a great product and support.
    Ed

    Obviously everything Paul is saying is correct. Having recently gone through a similar process that was long and frustrating but ultimately successful I will attempt to add what I can in the way of practical suggestions.

    First of all, we need to know what data throughput and what latency your project really requires. From the description you give it does not appear that latency can possibly be an issue

    ... so I'll focus on throughput. You say you "receive 128 byte data packets at 2.25 Mbps" -- does that mean you have an average amount of 2.25 Mbits of sonar data that must be transferred each second?

    Is 128 bytes a complete sonar ping response for a single handshake, or do you receive two thousand 128 byte packets in response to a single handshake?

    Does the data need to be received in a certain amount of time, or is it just that you need to keep up with the volume of data so things don't pile up on the sending side?

    = = = = =
    Throughput will be determined by a combination of the actual data transfer time (related to the interface speed) and the time spent waiting for transfers to occur. Each handshake will consume a lot of coordination/waiting time that will waste transfer time, so the more handshaking you do per data aliquot, the lower your total throughput. By a lot.

    If you control the code at both ends you can maximize your throughput headroom if you do as Paul has suggested: make the data source simply send data as soon as it exists. Put a header on the data (or put multiple frame headers on chunks of data) and simply begin sending it as soon as you can. Make the PC listen for data all the time, read the header (or the frame headers) and route the data accordingly (analysis, write to disk, etc.). I would probably define a virtual "channel number" for each sonar data source, so I could add additional sources easily, but there are many other ways to approach that part of the problem.

    If you *do* control both sides, I would *definitely* discard the concept of handshaking right off the bat, because it really adds nothing. Just set a timer interrupt ISR to listen on the PC side and keep an inbound queue filled with data.

    Virtual serial port transfer over USB is very good at getting the data to you in the right order -- it handles handshaking and retransmission etc. invisibly and quite reliably. When I first read this I was skeptical myself, but it's actually true. I move large amounts of streaming medical data, and although I did put a lot of safety code in place I have never observed any data loss in transit using virtual serial over USB. If you want to be absolutely certain, you will want to include sequence numbering in your outbound frames and make provision to request a retransmission of any data frame that is not received on the PC side.

    In my case the limiting factor was that if the host PC did not read data fast enough, I would run out of buffer space on the sending device and would be forced to discard data. I used a circular buffer so this happened automatically, but I did monitor it and I put a "0x00, 0xFF" byte pair on the outbound feed every time there was data loss -- that allowed me to see data loss easily in my displays without requiring me to send any ascii error messages (additional serial print calls change the timing of the program). At first I followed the published examples (MSDN and everywhere else) and they were either way too slow or appeared to lose data.

    Way too slow:
    Code:
    while (port.bytes_available > 0)
         var blah = port.Read(byte);
         // do something with blah
    or this approach, which loses data because port.BytesToRead does not count ALL the bytes that are available to read.
    Code:
        int count = port.BytesToRead;
        byte[] ByteArray = new byte[count];
        port.Read(ByteArray, 0, count);
    The approach that suddenly gave me all my data and the promised transfer speeds was this (this is actual working code from a C# app doing well over 1Mbps sustained.) The async part of it is not necessary -- the magic is in knowing that IFF the timeout is zero and you request a big block of data, you will get back everything that exists and a count of bytes that were put into your buffer. Then when you know how much you got, just copy it into a properly sized buffer and use it as needed.

    Code:
            protected async Task<byte[]> ReadSerialBytes()
            {
                byte[] buffer = new byte[4096];
                try
                {
                    int actualLength = await serialPort1.BaseStream.ReadAsync(buffer, 0, buffer.Length);
                    byte[] received = new byte[actualLength];
                    Buffer.BlockCopy(buffer, 0, received, 0, actualLength);
                    return received;
                }
                catch (IOException ex)
                {
                    Console.Beep();
                    statusMessage1("Error: " + ex.ToString());
                    return null;
                }
            }
    that code example is in C#, but the correct approach is not just a .net thing, because I experienced exactly the same thing when using Python and I have seen linux discussions that were substantially the same.

    = = = = =
    If you do not control the code at the source end and you are absolutely locked into multi-step handshaking, you may still be able to accomplish your throughput needs. The USB hardware will move something like 64 bytes every millisecond, but the PC operating system will not communicate with the USB hardware anywhere nearly that often. When your PC application requests that your operating system send or receive a single data chunk (regardless of the size of the chunk) your non-realtime operating system typically will wait 20 msec - 500 msec before it gets around to actually executing your read data or write data request. If you are running incorrectly configured backup or antivirus software (or your machine has phantom drives or any of a host of other problems) you may occasionally encounter periods of several seconds during which USB and serial port calls are suspended. If that kind of thing will sink you, you may need to add really big buffers running in a RTOS somewhere before the PC

    So when you *do* get to read from the port, you *really* don't want to read or write a single byte at a time, even though MANY of the code examples show it this way.

    In my experience a round-trip handshake can take up to 1 second, during which time you are not receiving any data. However, once the data does start to flow, it comes very quickly -- provided you read ALL received bytes on each read, rather than trying to read a single byte at a time.

    So the bottom line is that if latency is not the problem, you may lose a second for each data handshake but you may still be able to receive the data fast enough to keep up with the sources. You can easily test this, of course.

    Finally, if the throughput and latency you need can be met with virtual serial over USB or with a serial FTDI interface, I recommend you pursue serial transfers as opposed to native USB transfers, This is for one simple reason: I found relatively little in the way of guidance, tutorials, and project-oriented documentation for USB data transfers, whereas there are nearly 30 years of documentation available related to serial transfers. The bugs and gotchas in the Windows and Linux serial implementations are well understood and well documented, once you find the right sources. For USB transfers I found plenty of material aimed at comm engineers, but very little on the host operating system APIs.

    Oh, yes -- some additional practical pointers:

    Don't add sleep statements. They really won't help with this. The hardware really does work and it is much faster than your code or the OS, and the problems are almost certainly going to turn out to be in your own code and the way you're approaching the transfer.

    Barring a bad USB cable, if the teensy successfully puts data on the virtual serial port (or an FTDI port), that data WILL arrive at the host PC and it WILL be readable from the serial port. The problem is NOT happening between the underlying USB transport and the virtual serial port, so moving to native USB will not make anything fundamentally better.

    If you read from a virtual serial port CORRECTLY, it will NOT fail to give you data it has received and it will NOT give you the same data twice. It also will not discard data that it has received. The only thing that will happen is that if you fail to read from the port fast enough you may not keep up with inbound data, in which case the port will fill up and will fail to request data from the sender, and the sender may overflow and discard data.

    If you read from the serial port INCORRECTLY, you may appear to lose data. It's easy to be misled into reading incorrectly, and many of the published code examples out there have got it all wrong. You cannot trust the data_available interrupt, because it does not fire based on what you think would be triggering it. You also cannot trust the bytes_available concept because it does not reflect ALL the bytes that have actually been received and are available to be read.

    If you read a virtual serial port, set the read timeout to zero (before opening the port). Then you have two choices. You can keep checking to see if there are *any* reported bytes available. If there are, the count of available bytes is not reliable so just pass a big buffer and request all bytes -- you'll get a return value telling how many you got. Alternatively just keep trying to read all bytes from the port, and check to see if you got any. The serial port likely will return data in chunks: when I'm streaming data, my OS serial port implementation returns 4096 bytes whenever it shows any bytes available, so 4096 is the useful buffer size in my case.

    If you build a test sketch that just sends sequential bytes from the teensy and a test PC application that just reads bytes from the virtual serial port in the manner I've described (without handshaking) you should be able to achieve throughput above 1 mbps. If you *DON'T* achieve that throughput, you haven't got it right yet and you should post your code and other details of your attempts.

    After you've got that working, THEN if you really need to, you can add handshaking and work to minimize the loss of throughput.

    Not sure any of this helps in any way, but I thought I'd at least try to point you in the right direction...

    glhf

  12. #12
    Senior Member PaulStoffregen's Avatar
    Join Date
    Nov 2012
    Posts
    25,082
    Quote Originally Posted by Arctic_Eddie View Post
    The handshaking scheme is necessary due to the nature of the data source.
    I'm sure if you try hard enough, you'll find a way.

    If this absolutely is a requirement of the serial devices, perhaps you'll need to have Teensy send this protocol and store the results temporarily in buffers, which you then stream to the PC using a different protocol (where you don't wait, but ack after the fact).

    No amount of wishful thinking can change the way USB host controllers and drivers allocate the USB bandwidth. If you want to sustain 2+ Mbit/sec speed, you're simply going to have to adopt an approach which works well with the reality of USB.

  13. #13
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,318
    Quote Originally Posted by Arctic_Eddie View Post
    I have a project whereby 5120 bytes of sonar data are passed from the Teensy to a Lazarus program running on a laptop via an FTDI USB/serial adapter. The serial connection is very unpredictable with latency problems. I would like to abandon that approach and try USB_HID
    In addition to GraigF's comment, it would be important to know, if this sonar application is active (ping driven) or passive (continuous recording) and what the distance of the sonar(teensy) is from PC. If it is underwater than distances are typically such that they exceed USB max distance. Only serial, or Ethernet can help you to cover distances in water.

    I would second Paul's comment to try everything you can to avoid handshake, especially is application is passive. I would also add: try everything you can to get data rate down (including runtime block encoding, most sonar data are noise and do not cover all bits). If necessary, use two teensies, one on each side of the cable allowing you to build a "smart cable".

  14. #14
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    Here is the sequence of events.
    1. PC sends serial command, 2Mbps, to shore station, adjacent, requesting a scan using 'n' cycles in the TX burst then waits with timeout for return message saying data ready.
    2. Shore station passes same command to boat by nRF24 then waits with timeout for return message saying data ready.
    3. Boat controller fires the TX burst of 'n' cycles and triggers data capture from two Teensies( dual ADC synced ) by interrupt then watches two digital lines for capture complete.
    4. Boat controller collects data from both Teensies by serial, 2.25Mbps, then sends message to shore by nRF24 that data is ready.
    5. Shore station requests a certain packet, 32 bytes, from boat and gets data back. This continues until all data is in shore station.
    6. Shore station tells PC that data is ready.
    7. Same procedure is used to pass data from shore station to PC in 128 byte packets. Typical command packet is three words, command, device, data, 3, 2, 11, which means, send from sonar 2 packet 11.

    All timeouts are set slightly longer than actual time needed.

    Operating modes are single scan, continuous scan, single listen, continuous listen.

    The number of cycles in each TX burst is variable and can change at any time depending on several conditions.

    The boat may be up to 250m from the shore station. Boat motion is by dual water jet controlled by independent nRF24 pair.

    Most of the problems seem to be in the PC Lazarus program as few problems are encountered using Mega2560 at shore station instead of Teensy. Code is almost identical but some of the Teensy serial functions are undefined such as send_now().

    The purpose of the project is to find the bodies of drowning victims for local law enforcement.

    I would gladly switch to C++ in the PC if it had an IDE/RAD with features of Lazarus. There's a lot of code to rewrite. Here's a screen shot of the PC app.
    Attached Thumbnails Attached Thumbnails Click image for larger version. 

Name:	BodyBargeEchoDisplay23.jpg 
Views:	154 
Size:	98.9 KB 
ID:	7239  

  15. #15
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    Some additional thoughts.
    I can now get the 5120 bytes into the shore station array in 291msec so this is acceptable for now. The nRF24 has it's own ACK handling and throughput is quite slow even though the RF link is running at 2Mbps. I may have to live with that. An optical link is not feasible as there can be a line of sight problem. The problem lies within the shore station Teensy and the Lazarus program. The packet sequence and quantity is always the same for a particular link. If I can store the data packets faster than they're received then only an initial send command is needed. The data is 10-bit using Nanos or 12-bit for Teensies. The upper 6 or 4 bits can be used as a sequence counter. Even though the sequence has to be repeated it will still tell me if something is missing or duped. I can change the PC/SS link to send each sonar data set, 2560 bytes, in 128 byte packets into the 4096 buffer until I receive the right quantity. Each pair is a structure of two uint16_t representing a simultaneous sampling of echoes from two receiver modules. That's 512 samples of two uint16_t or four bytes. One Teensy handles the front/back receivers and the other collects the right/left receivers. Time of arrival deltas deterrmines the X and Y angles off vertical. Average time of arrival determines distance at that angle, Za, so the 3D image can be created. A couple of atan() functions determines offsets and depth, the only non-integers in the whole project. Echo strength is mapped to a rainbow color and alpha. Weak echoes are nearly transparent pale blue to strong echoes nearly solid red. As long as I can store faster than I can receive, it might work.

    I had already considered using two video TX/RX channels at 2.4GHz as virtual serial running at 6Mbps. That was a last resort scheme.

  16. #16
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,318
    Quote Originally Posted by Arctic_Eddie View Post
    Some additional thoughts.
    I can now get the 5120 bytes into the shore station array in 291msec so this is acceptable for now.
    That is only 150 kBaud effective speed. I would look for another radiolink protocol.
    Meanwhile get a low datarate system up and running and move the boat slower or revisit the place multiple times. OK, this way you translate a HW problem into a analysis (SW) problem (data fusion).
    Concerning the teensy Lazarus link, you could buffer the data on PC and not on teensy. that is, PC gets data as fast as possible from teensy (usb/serial) and buffers it for async processing and display. commands from PC go directly to boat, i.e. keep the two way datastreams separate.

    Question: Do the teensies any processing or are they only the glue between sensors-radio link - PC?
    I would use the teensies, especially the boat based ones to do signal processing.

  17. #17
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    They're only the glue and pass the data through. Each transfer point is serial on one side and nRF24 on the other side. I would like to get rid of that radio but cannot find anything faster, and yes, it's one of the bottlenecks. The original data is a pair of uint32_t values. The two extra zero bytes in each value is stripped before sending it to the barge controller. From there on the data never changes, other than packet size. A shore station Teensy or Mega is needed as the nRF24 talks by SPI to an Arduino. I've never seen one talk directly to a PC.

    The signal processing could end up being extensive for 4D imaging so I need the CPU running at 3.2GHz.

    I think a good place to start is to stream the data from the shore station into the PC since that's most of the problem. Once I accumulate 2560 bytes from each sonar set and store then I can get the second set. That will eliminate 40 command packets going out and the turn around time. Once I send one command out to capture then an appropriate timeout for it's return I will either get it all or fail.

    I'm still looking at an IDE/RAD for C++. I installed Embarcadero C++ Builder trial but it was massive and very confusing. It was quickly removed. The website screen shots for the various packages are very skimpy on object anchoring. That's where Lazarus really shines. I have a window on the desktop for each feature I need.

  18. #18
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,318
    Quote Originally Posted by Arctic_Eddie View Post
    The signal processing could end up being extensive for 4D imaging so I need the CPU running at 3.2GHz.
    I would break it up into preprocessing and image processing (indeed my tetrahedral processing does exactly this) preprocessing on teensy and image processing on PC.

    I would run a time gated transient detector on each hydrophone channel and transferring only (matched filtered) arrival times and peak energy (range gates and detection thresholds are update from PC and driven adaptively by ambient noise)
    I even would try to do all 4 channels on a single teensy. In that case, you could do something with all 4 channels (e.g) determine if delays describe an acoustic arrival or are corrupted.
    Obviously, these ideas require you have control on the system design.

  19. #19
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    The entire project is mine so I can alter the design and code of any part.

    I made some progress this afternoon. I changed the data transfer from a comand/packet handshake per packet to one command, which sonar pair, and all data streamed into the serial buffer. However, it appears that I'm the victim of the whims of the USB port. The 5120 bytes of data leaves the shore station Teensy in 23msec but takes 1 to 1.2 secs to get it into the Lazarus arrays. I'm also plagued by the presence of a small command packet in the head of the buffer from the previous command even though it's been read and the buffer is purged. At this point, I'll create a small test program and send the same amount of phony data from Teensy to PC until I find the delay and residual trash. Until that is solved, the other cool ideas from ya'll won't help much.

    OT
    What IDE/RAD do you use for your PC work. I'm still entertaining a replacement for Lazarus.

  20. #20
    Senior Member
    Join Date
    Jul 2014
    Posts
    3,318
    If you send me a pm with your e-mail, we can discuss off-line
    Last edited by WMXZ; 05-26-2016 at 08:36 PM.

  21. #21
    Senior Member
    Join Date
    Jan 2013
    Posts
    843
    Quote Originally Posted by Arctic_Eddie View Post
    I made some progress this afternoon. I changed the data transfer from a comand/packet handshake per packet to one command, which sonar pair, and all data streamed into the serial buffer. However, it appears that I'm the victim of the whims of the USB port. The 5120 bytes of data leaves the shore station Teensy in 23msec but takes 1 to 1.2 secs to get it into the Lazarus arrays. I'm also plagued by the presence of a small command packet in the head of the buffer from the previous command even though it's been read and the buffer is purged.
    You most likely have bugs(s) in either the serial port library you are using or your code. While USB has quite a bit of latency, you can still get several hundred command/response round-trips per second. The Teensy USB virtual serial connection has excellent performance for me and has been 100% reliable.

    A possible C++ environment you may want to look at is Qt. There is an excellent IDE called Qt Creator. This is the closest you will find to Delphi/Lazarus on the C++ side of things.

    There is a serial port module that works perfectly fine with the Teensy USB virtual serial port:
    https://doc.qt.io/qt-5/qtserialport-index.html

  22. #22
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    The problem has been solved and was mostly within the Lazarus code. Streaming out of the Teensy to a USB/Serial buffer of 2560 bytes also saved a lot of handshaking. The TLazSerial package is built on top of Synaser on top of 5DPO. It is easy to insert methods that you would think necessary but in fact cause unusual delays. They've all been found and I can now get my 5120 bytes of data transferred in less than 32msec. The actual value is unknown because the call to 'getTickCount' has a time resolution of 16msec. I also tried TComport but got the same performance. To identify my actual serial port still needs elements of TLazSerial so I'll stay with that approach.

    I've been in direct contact with WMXZ and he's been of great help. I've installed QtCreator but will save it for the next project. Most of my Laz code is complete and far too much to translate to Qt, especially the graphic part. I've also looked at Code::Blocks and it also has potential. An ideal solution for me would be the Laz IDE and features in C++ language. I don't expect that to every happen so I'll make use of the tools presently available.

    Thank you to all posters for your comments. They are greatly appreciated.
    Ed

  23. #23
    Senior Member
    Join Date
    Jan 2013
    Posts
    843
    Quote Originally Posted by Arctic_Eddie View Post
    The actual value is unknown because the call to 'getTickCount' has a time resolution of 16msec.
    http://wiki.lazarus.freepascal.org/EpikTimer

    On Windows:
    http://wiki.freepascal.org/QueryPerformanceCounter

  24. #24
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    I think the EpikTimer will do quite well. It has the same functionality as millis() in testing for a timeout event like a port ready/read.

    Thanks for the links.
    Last edited by Arctic_Eddie; 06-13-2016 at 07:14 PM.

  25. #25
    Senior Member
    Join Date
    Apr 2016
    Location
    St. Petersburg, FL
    Posts
    210
    The EpikTimer works fine and is installed into Laz. A simple function with a few lines now returns elapsed time in milliseconds as a Uint32 and can be used exactly like getTickCount() or millis().

    Thanks again.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •