The Oneirophone: Feasibility of MIDI controller with insane number of sensors

Status
Not open for further replies.
O

Oneiros

Member
Hello,
(please bear with me, my english skills are kinda rusty)

I started to actually plan a project i had in mind for a long time and i am not sure if it can be realized (without me going bankrupt).
I have done smaller (MIDI) projects with teensy in the past but i am neither good in electronics nor in programming yet.

I want to build a MIDI controller/instrument with great polyphonic articulation possibilities/control (like e.g. the "Eigenharp", "LinnStruments" or "Seaboard") (at as low cost as possible).
Therefore i want to use a keypad with 4 pressure sensors per key (selfmade using velostat).
With using many keys (around 120 would be nice but i think i have to lower my expectations here quite a bit) it adds up very quick (to almost 500 sensors with 120 Keys). That should be problematic in more than one way.

Here are (some of) my thoughts so far:

Teensy 3.6 has 25 analog ins. I need some of them for other stuff so i calculate with 20 ins available for the keys.

With multiplexing using 74HCT4051 i could expand it to 160 analog ins (or is it possible to "stack" multiplexers somehow?) = 40 keys. With 3 boards that would be the desired 120 keys (+15 analog ins for other stuff... more than enough).
What gives me a little headache is the necessary delay between 74HC4051 change and analogRead because with MIDI instruments one would like to have as low latency as possible and the delays will also add up fairly quick i think (I am not sure if it is a real problem, will have to test it but want to fathom the feasibility of the whole project before ordering parts to do so. One thing that came to my mind is to use a capacitive switch to recognize which of the keys are touched and cycle only through the corresponding multiplexers instead of all (but it is only a vague idea and i need to do much research and thinking first to understand if it is possible and reasonable to do so... atm my guts say, it may not).

I would then connect these 3 teensy to one Rhaspberry PI (either as USB-MIDI device or just transfering the data and doing most of the math/MIDI-programming/conversion with the PI) (I want to use the Rhaspberry PI running Csound as a Soft-Synth anyway). I have not read into it yet and am not sure if it would work this way though?

So my biggest concerns atm are if it is possible to connect more than one Teensy to the Raspberry (ideally via an USB hub), and, the biggest concern, if this would be fast enough for an usable MIDI instrument.

Ideas and opinions on this are highly welcome.

Greetings, Lars
 
Depending on your budget/time using an array of Teensy LCs as your sensor readers may be an option over multiplexing chips. Then you can build one module, tinker with it till it works then duplicate using i2c/SPI/CAN/Serial to pull the data into an T3.2 for composing into USB Midi.

Total parts cost is higher but means you don't have a massive number of wires trying to run into a single module. Instead you have some form of common wired bus running along modules that can be physically close to the sensors. This will certainly be faster to build and may even cost less in overall needed hardware.
 
I think it might depend on how this thing will be played... but if there is a contact switch on each key to indicate it's in play you could then direct the analog scanner to make that a key of interest for extracting the additional parameters from the presure sensors.

Scanning a switch network can be done much more effectively than scanning thru analog values with noise.

Otherwise it's multiplexing analog signals that will be the bottleneck if you are scanning hundreds of them all the time (and all but a few will be sending background noise only). And you will have to write some kind of hysteresis (smoothing) method that can deal with all those values efficiently to make them stable enough to produce clean output.

With directed scanning you would have a polyphony limit based on the size of the array you use to track the matrix coordinates of the sensors in play. The analog section would direct it's muxes to switch to the sensors of interest based on the active note stack.

You can use mux for both row and column of a switch/sensor matrix. If you have four analog and on digital on each key I would try five matrixes of 8 x 8. The digital one would need separate control signals as it is full scanning while the rest are directed.

But this would be a very challenging project. The modular approach might be preferable if the instrument you have in mind is compatible with it.
 
Oneiros said:
So my biggest concerns atm are if it is possible to connect more than one Teensy to the Raspberry (ideally via an USB hub),
Click to expand...

Yes, you can connect many through hubs to a Raspberry Pi.

You can even connect many USB MIDI devices through hubs to the USB host port on a Teensy 3.6. You might be surprised how well a 180 MHz microcontroller can compete with a 1 GHz Linux-based SBC when it comes to latency when processing real-time events. Linux is a non-realtime system which adds quite a lot of scheduling overhead.

and, the biggest concern, if this would be fast enough for an usable MIDI instrument.
Click to expand...

Yes, USB should be plenty fast enough. Just make sure you use Multi-TT hubs. The cheaper Single-TT types don't properly share bandwidth when converting 12 to 480 Mbit/sec. Then again, even Single-TT might be able to do it, but good Multi-TT hubs usually don't cost much more.
 
Thx for the fast, nice and reeeallly helpfull answers :) (i hesitated a little bit posting about my plan because i worried to be rediculed ... even to me 500 analog sensors sounds like bigheaded foolishness every time it comes to my mind).

oddson said:
I think it might depend on how this thing will be played...
Click to expand...
It will be (by accident) close to the Eigenharp (my first vague ideas are actual older than the Eigenharp and derived from Chapman Stick, Bassoon and Cello but the Eigenharp is a huge inspiration now, too).

oddson said:
but if there is a contact switch on each key to indicate it's in play you could then direct the analog scanner to make that a key of interest for extracting the additional parameters from the presure sensors.
Click to expand...
That is great news.
I would love to use the modular approach (never thought of that before GremlinWrangler suggested it) but i think in the end it will be too expensive (500 Analog Inputs would be ca. 40 Teensy LCs = ca. 480€ and initially i wanted to stay under 150€ (Raspberry excluded), knowing that it will not be possible by far most likely (trying it nonetheless by using as much scrap material as possible etc.).

oddson said:
You can use mux for both row and column of a switch/sensor matrix. If you have four analog and on digital on each key I would try five matrixes of 8 x 8. The digital one would need separate control signals as it is full scanning while the rest are directed.
Click to expand...
Does this mean i could make an "analog sensor matrix"? I somehow assumed (what is kinda narrow-minded when i think of it now) a matrix would only be possible with digital sensors (switches). (So maybe one teensy 3.6 would be enough or do i have an error in reasoning (highly probable): 5 8x8 matrixes = 10 74HCT4051 so only 10 analog ins for this?). (Sorry beeing dumb concernic electronics... I am working on it).

PaulStoffregen said:
Yes, you can connect many through hubs to a Raspberry Pi.
You can even connect many USB MIDI devices through hubs to the USB host port on a Teensy 3.6
Click to expand...
Great news again. :)

So it seems the project is possible though not easy. (if it would not be a challenge it would be no fun i think).
Then i will order parts to start i think (with baby steps of course ... prototyping a single key first - have two ideas how to (maybe) reduce the amount of analog sensors to three per key.)
 
74HCT4067 has 16 channels. You can cascade analog muxes. I.e. connect one 4067 to a Teensy analog pin and use it to switch between 16 4067s. This will give you 256 channels (or 512 duplicating things on a second Teensy pin). Depending on your analog setup, a 100ns switching time may be long enough for things to settle.

'analogRead()' can run at up to 360'000 samples per second, configured with:
analogReadResolution(8);
analogReadAveraging(1);

Allowing for channel switching and processing, 150'000 samples per second may be realistic (that could be 500 sensors with a 3ms update rate). If you don't have a clean signal and need additional filtering / averaging things will quickly look a lot worse...

\\

The big question for me is, can you you build a decent analog sensor with velostat? I expect, you will need individual calibration for each sensor.
 
I don't think you'd need 40 Teensy to do a modular version...

I think the biggest mistake you could make is to buy all the stuff for a full version before you know you can deal with the challenges... this is a VERY complex build.

I would focus on the key design and functionality and see how a small number of them can work together and provide the output you are looking for and only afterwards consider muxes and greatly increased key counts.
 
tni said:
74HCT4067 has 16 channels.
Click to expand...
I cannot solder smd (and did not find a through-hole version) and a 4067 breakout board is considerably more expensive.

tni said:
The big question for me is, can you you build a decent analog sensor with velostat? I expect, you will need individual calibration for each sensor.
Click to expand...
I am pretty sure that individual calibration is inevitable (and not only once most likely). It will be a funny pain in the ... :eek:

I also have not much illusion about the precision etc. of the sensors. It will be by faaaa(...)ar not as good as the keys on commercial products. The first tests will show if it works adequate at all (never worked with Velostat but there are some youtubevideos (e.g. [1] [2] ) out there that look mildly promising). If it does not work i have to come up with alternatives. (i already dumped the idea of a matrix of hall sensors (would need many and it will be too expensive... but cool somehow) and a glove with sew in magnets in the fingertips).
I am very open to suggestion.

oddson said:
I think the biggest mistake you could make is to buy all the stuff for a full version before you know you can deal with the challenges... this is a VERY complex build.
I would focus on the key design and functionality and see how a small number of them can work together and provide the output you are looking for and only afterwards consider muxes and greatly increased key counts.
Click to expand...
I fully agree, babysteps is the key and i have no time-limit. Just wanted to check if it is possible and collect some ideas and suggestions (thank you all for taking the time and help me :) ) before starting (with a single key).
 
BTW, 74HC4051 has very bad performance with 3.3V Vcc (which may start to matter, if you cascade them). With 5V Vcc, it may not switch reliably with the Teensy 3.3V IO voltage.

74HCT4051 must be powered from 5V and will work reliably with the Teensy 3.3V IO.

If you use 8 Teensy pins, you can get 8 x 8 x 8 = 512 channels with the 4051s cascaded one level.
 
I always prefer muxing digital than analog sooo..

I have never use one of these (http://www.hobbytronics.co.uk/mcp3008-adc) but using one for every other key then you would only need to mux the chip select pin which is digital so should be a bit easier. You will have to watch for capacitance/inductance etc on the SPI line but you will have to be careful about such things to get good ADC readings from the teensys too.

they seem quite cheep each but £2.04 * 250 =£525 for all 120 keys seems a lot. For a more modest project these would seem like a good idea, has anyone run these with success? or know of any cheaper ones?
 
I thought the project through thoroughly and came to the conclusion, that there is no proportion between the effort and the expected result.

So I changed the design of the instrument from beeing a keyboard with 120 keys to be a ribbon controller with 4 "strings", (very close to a cello - even planning on using a bow beside the breath sensor, have some nice ideas).

I will not glue softpot membrane potentiometers ontop of long force sensitive resistors for the strings (that would be over 200€ for all needed sensors and i have doubts about the longevity - and everybody else does it that way... booooring) but i will use some load cells from china with some HX711 breakout boards for force-sensing and some VL53L0X time of flight distance sensor breakout boards for position sensing (should be around 50€ - plans are not yet 100% done).

Staying under 150€ seems possible again. :)

I will post more details here soon.

Edit:
Found out that VL53L0X has a FOV of 25 degree, so most likely it will not work the way i hoped it would.
 
Last edited:
Status
Not open for further replies.
Back
Top