I'm absolutely astonished! It all worked the first time.
For the first time in my life I soldered an SMD chip with hot air gun: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1
Using low temperature soldering paste: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1

Click image for larger version. 

Name:	T20191012_002_0007.jpg 
Views:	9 
Size:	99.1 KB 
ID:	17873

I just put a little paste to cover the contacts, put the chip on top, set the blower to 137C and blew on the contacts until the paste turned silver.
I have mutilated an extra breakout board with the USB host I had and used it just for the USB host.

Click image for larger version. 

Name:	20191012_132842.jpg 
Views:	6 
Size:	324.3 KB 
ID:	17874

And this is my Teensy 4.0 with USB host wires just soldered to the contacts at the bottom and bent to the side.
The Teensy is on top of the new Audio Shield. The big silver round thing is the microphone, extremely sensitive, by the way.

Click image for larger version. 

Name:	20191012_135222.jpg 
Views:	5 
Size:	139.7 KB 
ID:	17875

Then I just started this example:
C:\Program Files (x86)\Arduino\hardware\teensy\avr\libraries\USBHos t_t36\examples\Serial\MIDI\InputFunctions
Thanks @neurofun

Then I connected the cheapest MIDI keyboard I could find: https://www.amazon.com/gp/product/B0...?ie=UTF8&psc=1
And it just worked, even the pitch bend, modulation, key velocity and volume:
USB Host InputFunctions example
Control Change, ch=1, control=7, value=94
SysEx Message: 5F 4B B2 (end)
SysEx Message: C8 A7 15 (end)
Control Change, ch=1, control=7, value=94
Note On, ch=1, note=57, velocity=54
Note Off, ch=1, note=57, velocity=64
Note On, ch=1, note=57, velocity=127
Note Off, ch=1, note=57, velocity=64
Control Change, ch=1, control=7, value=93
Control Change, ch=1, control=7, value=92
Control Change, ch=1, control=7, value=91
Control Change, ch=1, control=7, value=89
Control Change, ch=1, control=7, value=88
Pitch Change, ch=1, pitch=128
Pitch Change, ch=1, pitch=512
Pitch Change, ch=1, pitch=896
Pitch Change, ch=1, pitch=1344
Pitch Change, ch=1, pitch=1728
Control Change, ch=1, control=1, value=2
Control Change, ch=1, control=1, value=5
Control Change, ch=1, control=1, value=2
Control Change, ch=1, control=1, value=0
Control Change, ch=1, control=1, value=2

Sometimes you spend days fighting something totally stupid, and sometimes you are ready to spend days on something and it just works in half hour.
I'm so happy! Will try to play with some synthesis today.

Here is my whole thing with guitar input, screen, touch sensors and battery. Still testing the A2DP Bluetooth.
Click image for larger version. 

Name:	20191012_140319.jpg 
Views:	22 
Size:	348.3 KB 
ID:	17876

And here is my old thing, with Teensy 3.6 and MIDI synth, with a horrible sound samples:
http://quantograph.com/