[posted] modular datalogger system

F

Fenichel

Well-known member
Several other forum members have described dataloggers for high-bandwidth datastreams, often status data from moving vehicles. I have completed a project that flexibly collects data from two or three dozen low-bandwidth sensors placed around my house, taking VC0706 pictures; measuring temperatures, pressures, water depth, power usage, and so on; and controlling relays.
The sensors are connected to peripheral stations, each built around a Teensy 3.5 and a real-time clock. The stations have identical hardware & software, but they can be configured through text files on the Teensies' uSD cards to work with different sensors and then either to use their XBee tranceivers to send the data to a central base station, or to timestamp & store received data locally.
The base station is built around another Teensy 3.5 and clock. It timestamps received data and then can store it locally for later transmission or transmit it immediately (through a USB connection) to a Windows program.
The Windows program is built around a relational database. Received data are stored in the database, and the database can be mined for graphs and statistics. Here's a sample graph:
raw graph outside.jpg
The system is described on my Web site at https://www.fenichel.net/pages/Indoor_Activities/electronics/datalogger/datalogger.pdf.
 
very impressive.

1. how often do you store readings? (every second for example)
2. how many readings do you use for averaging (assuming you do averaging)?
 
KrisKasprzak said:
very impressive.

1. how often do you store readings? (every second for example)
Click to expand...
This varies from sensor to sensor, but rarely as often as once a second. Remember, these are measurements of what is happening in & around an ordinary house. Why measure the temperature of a room every second? Why measure the temperature of a pond more often than every hour?

The reading/reporting distinction (Section 2.2.4.1 of my big document) is important here. For example, I look at my anemometer every minute, but I report an exponentially-damped running average only every 10 minutes. Also, as noted in Section 4.1 of the document, some data are recorded only when they change. For example, I have a photocell outside. It records a light level of zero for a few hours every day. The zeroes are duly passed in to the Base Station and then to the Windows program, but the Windows program is not silly enough to save all of them. In fact, it will save exactly four zeroes: first total darkness, daily signoff at 23:59:59, daily startup at 00:00:00, and just before first light.

2. how many readings do you use for averaging (assuming you do averaging)?
Click to expand...
The averaging is specified in the X_Config.txt files in terms of time, not number. If a reading interval is 1 minute and the reporting interval is 10 minutes, then the sensor driver will have 10 readings to think about (averaging, damping, determining the recent range, whatever it wants to do) between reports. Instead of specifying reporting intervals as times, I could have specified them as multiples of the reading interval, with the same result.
 
This expands on my previous answer about sampling rates. The key to tuning the system has been that all of the configuration information is on the uSD cards. The only numbers hardcoded into the sensor drivers are minimum read intervals and the like, taken from the sensors' datasheets. Every now and then, I decide that a graph looks too ragged, so I
  • pull the pertinent uSD card,
  • edit X_config.txt to increase the damping or the reporting interval/reading interval ratio,
  • reinstall the uSD card, and
  • reboot the peripheral station.

As of this morning, my current reading and reporting intervals are listed in the attached spreadsheet.
 

Attachments

  • reading & reporting.xls
    30 KB · Views: 165
Do you have any photos of the real hardware, ideally in use? I see a diagram of the PCB on page 5, which is close, but to show this on the PJRC website, at least one photo or video of the real hardware is needed.
 
pictures

PaulStoffregen said:
Do you have any photos of the real hardware, ideally in use? I see a diagram of the PCB on page 5, which is close, but to show this on the PJRC website, at least one photo or video of the real hardware is needed.
Click to expand...

I'm sorry; I hadn't been aware of that rule. Here are pictures (in use) of the base station and two peripheral stations.base station.jpgTV room.jpgoffice.jpg The design is always evolving, so the "TV room" peripheral board is of a slightly different (newer) design than the "office" peripheral board.
 
I was caught off guard by Paul's page drawing attention to my old home monitoring project. The project is in the middle of some fairly broad redesign, and I am trying to avoid getting sucked into a 2.0 syndrome. I was not planning to say anything new about it here until the redesign was further along.

To reduce the chance that people might read my old description and copy some decisions that I now think were suboptimal, I list three issues here.

(a) The old peripheral boards had too much sensor- and effector-specific stuff on them. For example, each of those boards reserved space for
  • two MID400 chips, so that the board could support detection of two different AC flows, and
  • a small relay.
In the redesign, the peripheral boards will have enough connectors to be prepared for any reasonable combination of my sensors & effectors, but the sensor-/effector-specific hardware (relays, debouncers, MID400s, and so on) will all be deported to daughter boards, to be deployed as needed.

(b) The CPUs on the various boards (then a Teensy 3.2 and multiple 3.5s; to be all 4.1s) were all socketed, so that they could be moved to a development environment for reprogramming. I am not using any of the Teensys' SMD pins, but the VUSB pins and the pins running crosswise (VBatt and so on) are not compatible with conventional socketing or breadboarding. Ad hoc socketing on my boards (to capture those abnormal pins) has made the Teensys difficult to pull out without damage. In the new version, each Teensy 4.1 will be soldered onto a breakout board bringing all of the pins of interest out to two conventional rows of breadboard-compatible pins. The breakout board will then be socketed onto one of my main boards. In every instance, this will mean loss of over a square inch of PCB real estate, but the increase in maintainability should be worth it.

(c) Some of the peripheral boards have to be in places where reading the TeensyViews is awkward for me (I'm not young). The new boards will have more LEDs, providing an information channel that under some circumstances has a higher effective bandwidth.
 
Last edited:
You must log in or register to reply here.
Back
Top