Recommended image sensor module with low latency, high fps?

[plengqui]

What would be the best camera module or image sensor for these requirements:
I want to detect movement at points along a wide (1m) and very narrow (1cm) area, from about 30cm distance. I think about 1x400 pixel resolution is ok. End-to-end latency needs to be very low, preferrably <5 ms. Both frame rate and latency are important.

The project goal is to create a MIDI keyboard out of an old traditional piano, by a camera visually detecting the movement of the felted "hammers" that hit the strings in the piano when a key is pressed. Acceptable latency end-to-end including sending the "key down" MIDI message is <10 ms. The camera is to be placed inside the piano about 30cm above the row of hammers facing down. To improve visual contrast, the hammers could be spray painted black and equipped with reflex tape on top, and a bright LED can be placed beside the camera. While this is possible to achieve with mechanical or electromagnetic sensors, a visual approach is less invasive and easier to install. Since only a single pixel row needs to be monitored, this should be possible with a teensy, provided I can find the right camera module.

 

[MarkT]

There are single pixel wide arrays that might be relevant from what I recall. Due to the small number of pixels these
are probably easy to use at high frame rates. Using IR illumination might be less distracting.

 

[KurtE]

Sorry not really sure what would work for you here. Hopefully others have better ideas. But if it were me, some things I might look at include:
a) an array of Time Of Flight (TOF) sensors, examples include: https://www.adafruit.com/product/3316
My guess is that this might be a pain to setup...

b) Some form of Lidar? My guess though is most spinning ones will be too slow for your usage: Again like: https://www.adafruit.com/product/4010

c) Not sure if any of the RealSense or like sensors would work? https://www.sparkfun.com/products/14946

d) likewise not sure if any of the OpenCV type devices could learn to do some of this?

Sorry again just throwing some random things. Hopefully others will have a real solution.

 

[mlu]

Do I get the picture right, the camera is above the hammers, is this a standing piano so the camera sees the edge of the hammers, or is like a grand piano so the camera sees the top/back of the hammers ?

How do you intend to see the movement of the hammers from a 1 pixel wide sensor ? By hammer seen or not seen, essentially on/off ?

 

[plengqui]

@mlu: yes, correct. It is a standing piano, so the movement is in the picture plane. And the next iteration would be to monitor a couple of such 1x400-ish pixel rows and detect when a hammer "enters" and "leaves" each row, register the exact timing and from that determine the velocity.
@MarkT: great idea. I googled around and while having great frame rates (8000 fps) these are expensive. The only affordable one I could find was the MLX75306 and the TSL1401CL.

I think a perfect component would be a mainstream (=cost efficient) monochrome image sensor like the OV7175, set to view a thin 1x640 window (Region of Interest = ROI) and then deliver good framerate. The problem is that all image sensors I have found so far don't increase the frame rate when set to a more narrow ROI. With the right scan control logic, it should be possible for a module that is capable of 30pfs at 640x480 resolution to deliver a 1x480 ROI at 30*640 = 19.200 fps. Right?
Do you know of any image sensor that can increase framerate whan ROI decreases?

 

[mlu]

Not necessarily, your frame rate calculation is just valid for the image transmission, but the sensor array has a minimum exposure time in order to create a useful image, that would be the frame time for a single pixel image. I guess a very careful reading of the data sheet for the actual image sensor chip is needed to answer these things.

 

[plengqui]

I just found out there are framerate calculator tools online. This one says a PYTHON480 sensor could do 1600 fps using a 4x600 pixel Region of Interest. Neat!