I have ordered myself a Teensy 3.2 to get around this.
The 128 buttons are working fine by editing the header file. However the axis do not seem to respond, even when not assigned to analogreads but just sending values. As soon as a button is sent as a joystick "command", all axis get reset to 0...