Using Teensy as Keyboard/Trackpad (multi touch) HID combo device

[Eric Damon Stahl]

Hi,so I've been working on this project of mine since over a year now. Short story short: I want to convert a 2011 MacBook Pro keyboard and trackpad to an external usb device for use on Android. I got it working and it works everywhere (Windows, MacOS, Linux, Android_x86) except on native Android devices.

After spending WAY too much time on this I would like to tackle it another way and I wonder if what I plan is possible with (any) a Teensy.

I want to make the Teensy act as a Keyboard/Trackpad USB HID device and I have two ideas for it (and have no idea which of it is do-able):

Version A)
Take the USB signal from my "device", interpret it on the Teensy or pass it through directly (with a different USB device header?).

Version B)
Bypass the original circuit for the k+t and feed it the data lines directly (should be 26 for the keyboard and 14 or 15? for the trackpad), process them with the Teensy and output it as USB. Same questions as above:




For both version I have the same questions:
1. Is this possible in general?
2. Can the Teensy produce a multi-touch output for the trackpad (up to 5 fingers)?
3. Will the output work on Android? I'm asking specifically because of my experience..... as the original "device" works EVERYWHERE except on native Android devices...



I would prefer if version A was possible as this wouldn't require me to find a way to get breakout boards for the tiny and flimsy k+t connectors.
If possible, which Teensy model would I need? I am willing to do whatever it takes to make this work. Any help appreciated.

Thanks in advance for your time!



The long story is below:

**********************************************
With the help of the schematics and the original mainboard I was able to isolate the circuitry needed for the k+t. It uses a CY8C24794 to convert the SPI signals to USB internally. After a bunch of very delicate soldering I got it to work.
It shows up as "Apple Internal Keyboard / Trackpad".


The keyboard works everywhere I tried (Windows, Linux (Ubuntu), MacOS, PC bios, Android_x86, Android (2 phones and 1 tablet)).

The trackpad works as follows:
Windows -> works as mouse... moving cursor + click, no gestures / multitouch
MacOS -> works fully including all gestures
Linux (Ubuntu) -> works fully including all gestures
Android_x86 -> works fully including all gestures

Android (Samsung Galaxy S9, S10+, Lenovo P11 Pro tablet ...., Android versions 10, 11, 12) -> never works AT ALL, it appears fully dead (only the trackpad part, keyboard always works fully).

So for the love of god I can't get this to work on Android for whatever reason.
**********************************************

[KurtE]

Sorry I am probably not much help here for this, as I don't really do anything with Android setups.

So some of this may not make any sense as regards to Android, but will throw them out there on the off chance they do.

Things like: Wondering if you may need to install some drivers on the Android to handle some of the different USB configurations of your device.
Might help to see what all is in your USB Descriptor.

Options.
First thing I would experiment with is to plug your device into the USB Host port on either T3.6 or 4.x. Probably easiest to setup 3.6 or 4.1 using something like:
https://www.pjrc.com/store/cable_usb_host_t36.html

You can then run the USBHost_t36 example which prints our HID information and we can hopefully see what all we currently recognize.

From there you may then need to setup a custom USB type to output to the PC/Android.

As for some of the details on things like multi-touch, I don't know as I have never tried it. I have mostly played on the USBHost side, and a few other USB Types like MTP, Dual Serial.

Again probably not much help, but if you do plug it into USBHost, will be interested to see how well we recognize all of your current pieces.

[Eric Damon Stahl]

So I know this is not on a Teensy (need to figure out where to get one first as they seem to be out of stock, at least the 3.X and 4.1), but maybe the device descriptors that I got (with "Free Device Monitoring Studio" for Windows) will help in some way already.



The first Touchpad shows:

Touchpad (Internal Keyboard/Trackpad (ISO))

Device Descriptor:
----------------------------------------------
This device is part of a composite device.

Connection Information
 Port: 2
 Speed: Full Speed
 Device address: 6
 Open pipes: 3
 Connection status: Device connected

Device Descriptor
 USB version: 2.0
 Device class: 0x0 - (Defined at Interface level)
 Device subclass: 0x0 - Unknown
 Device protocol: 0x0 - Unknown
 Control pipe max size: 8 bytes
 Vendor ID: 0x5ac (Apple, Inc.)
 Product ID: 0x246 (Internal Keyboard/Trackpad (ISO))
 Product version: 1.8
 Manufacturer: Apple Inc.
 Product: Touchpad
 Serial Number: Not specified
 Configurations: 1



Configuration Descriptor:
----------------------------------------------
descriptors[0] = "Configuration Descriptor" 
bLength = 9 
bDescriptorType = USB_CONFIGURATION_DESCRIPTOR_TYPE (2) 
wTotalLength = 34 
bNumInterfaces = 1 
bConfigurationValue = 1 
iConfiguration = 0 
Reserved = 0 
SupportsRemoteWakeup = 1 
SelfPowered = 0 
PoweredByBus = 1 
MaxPower = 0x14 -> 40 mA 

descriptors[1] = "Interface Descriptor" 
bLength = 9 
bDescriptorType = USB_INTERFACE_DESCRIPTOR_TYPE (4) 
bInterfaceNumber = 1 
bAlternateSetting = 0 
bNumEndpoints = 1 
bInterfaceClass = UsbHID (3) 
bInterfaceSubClass = 0 
bInterfaceProtocol = 0 
iInterface = 4 

descriptors[2] = "HID Descriptor" 
bLength = 9 
bDescriptorType = HID_HID_DESCRIPTOR_TYPE (33) 
bcdHID = 1.11 
bCountry = NotDefined (0) 
bNumDescriptors = 1 
reports =  
	reports[0] =  
		bReportType = 34 
		wReportLength = 27 
		raw =  7, 5, 129, 3, 64…  



HID Descriptor:
----------------------------------------------
This device is part of a composite device.

HID Descriptor
Usage Page: 65280 (Vendor Specific)
Usage: 1 (Unknown)
Collection 
Usage Page: 65280 (Vendor Specific)
Usage: 1 (Unknown)
Logical Minimum: 0
Logical Maximum: 255
Report ID: 68
Report Size: 8
Report Count: 511
Input: 0 (Data,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

End Collection

The second Touchpad:

Device Descriptor:
----------------------------------------------
This device is part of a composite device.

Connection Information
 Port: 2
 Speed: Full Speed
 Device address: 6
 Open pipes: 3
 Connection status: Device connected

Device Descriptor
 USB version: 2.0
 Device class: 0x0 - (Defined at Interface level)
 Device subclass: 0x0 - Unknown
 Device protocol: 0x0 - Unknown
 Control pipe max size: 8 bytes
 Vendor ID: 0x5ac (Apple, Inc.)
 Product ID: 0x246 (Internal Keyboard/Trackpad (ISO))
 Product version: 1.8
 Manufacturer: Apple Inc.
 Product: Touchpad
 Serial Number: Not specified
 Configurations: 1



Configuration Descriptor:
----------------------------------------------
descriptors[0] = "Configuration Descriptor" 
bLength = 9 
bDescriptorType = USB_CONFIGURATION_DESCRIPTOR_TYPE (2) 
wTotalLength = 34 
bNumInterfaces = 1 
bConfigurationValue = 1 
iConfiguration = 0 
Reserved = 0 
SupportsRemoteWakeup = 1 
SelfPowered = 0 
PoweredByBus = 1 
MaxPower = 0x14 -> 40 mA 

descriptors[1] = "Interface Descriptor" 
bLength = 9 
bDescriptorType = USB_INTERFACE_DESCRIPTOR_TYPE (4) 
bInterfaceNumber = 2 
bAlternateSetting = 0 
bNumEndpoints = 1 
bInterfaceClass = UsbHID (3) 
bInterfaceSubClass = 1 
bInterfaceProtocol = 2 
iInterface = 4 

descriptors[2] = "HID Descriptor" 
bLength = 9 
bDescriptorType = HID_HID_DESCRIPTOR_TYPE (33) 
bcdHID = 1.11 
bCountry = NotDefined (0) 
bNumDescriptors = 1 
reports =  
	reports[0] =  
		bReportType = 34 
		wReportLength = 52 
		raw =  7, 5, 132, 3, 8…  
 



HID Descriptor:
----------------------------------------------
This device is part of a composite device.

HID Descriptor
Usage Page: 1 (Generic Desktop Controls)
Usage: 2 (Mouse)
Collection 
Usage: 1 (Pointer)
Collection 
Usage Page: 9 (Buttons)
Usages (1-3): Button 1 (Primary), Button 2 (Secondary), Button 3 (Tertiary)
Logical Minimum: 0
Logical Maximum: 1
Report ID: 2
Report Count: 3
Report Size: 1
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Count: 1
Report Size: 5
Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage Page: 1 (Generic Desktop Controls)
Usage: 48 (Direction-X)
Usage: 49 (Direction-Y)
Logical Minimum: -127
Logical Maximum: 127
Report Size: 8
Report Count: 2
Input: 6 (Data,Variable,Relative,No Wrap,Linear,Preferred State,No Null Position)

End Collection 
End Collection

And the keyboard (although irrelevant I think as it works normal)

Apple Internal Keyboard (Internal Keyboard/Trackpad (ISO))

Device Descriptor:
----------------------------------------------
This device is part of a composite device.

Connection Information
 Port: 2
 Speed: Full Speed
 Device address: 6
 Open pipes: 3
 Connection status: Device connected

Device Descriptor
 USB version: 2.0
 Device class: 0x0 - (Defined at Interface level)
 Device subclass: 0x0 - Unknown
 Device protocol: 0x0 - Unknown
 Control pipe max size: 8 bytes
 Vendor ID: 0x5ac (Apple, Inc.)
 Product ID: 0x246 (Internal Keyboard/Trackpad (ISO))
 Product version: 1.8
 Manufacturer: Apple Inc.
 Product: Apple Internal Keyboard
 Serial Number: Not specified
 Configurations: 1



Configuration Descriptor:
----------------------------------------------
descriptors[0] = "Configuration Descriptor" 
bLength = 9 
bDescriptorType = USB_CONFIGURATION_DESCRIPTOR_TYPE (2) 
wTotalLength = 34 
bNumInterfaces = 1 
bConfigurationValue = 1 
iConfiguration = 0 
Reserved = 0 
SupportsRemoteWakeup = 1 
SelfPowered = 0 
PoweredByBus = 1 
MaxPower = 0x14 -> 40 mA 

descriptors[1] = "Interface Descriptor" 
bLength = 9 
bDescriptorType = USB_INTERFACE_DESCRIPTOR_TYPE (4) 
bInterfaceNumber = 0 
bAlternateSetting = 0 
bNumEndpoints = 1 
bInterfaceClass = UsbHID (3) 
bInterfaceSubClass = 1 
bInterfaceProtocol = 1 
iInterface = 3 

descriptors[2] = "HID Descriptor" 
bLength = 9 
bDescriptorType = HID_HID_DESCRIPTOR_TYPE (33) 
bcdHID = 1.11 
bCountry = International (13) 
bNumDescriptors = 1 
reports =  
	reports[0] =  
		bReportType = 34 
		wReportLength = 156 
		raw =  7, 5, 131, 3, 10…  




HID Descriptor:
----------------------------------------------
This device is part of a composite device.

HID Descriptor
Usage Page: 1 (Generic Desktop Controls)
Usage: 6 (Keyboard)
Collection 
Report ID: 1
Usage Page: 7 (Keyboard)
Usages (224-231): Control Left, Shift Left, Alt Left, GUI Left, Control Right, Shift Right, Alt Right, 
GUI Right
Logical Minimum: 0
Logical Maximum: 1
Report Size: 1
Report Count: 8
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Count: 1
Report Size: 8
Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Count: 5
Report Size: 1
Usage Page: 8 (LEDs)
Usages (1-5): NumLock, CapsLock, Scroll Lock, Compose, Kana
Output: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position,Non Volatile)

Report Count: 1
Report Size: 3
Output: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position,Non Volatile)

Report Count: 6
Report Size: 8
Logical Minimum: 0
Logical Maximum: 255
Usage Page: 7 (Keyboard)
Usages (0-255): No Event, Keyboard ErrorRollOver, Keyboard POSTfail, Keyboard Error Undefined, A, 
B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, 
1 and ! (One and Exclamation), 2 and @ (2 and at), 3 and # (3 and Hash), 4 and $ (4 and Dollar Sign), 
5 and % (5 and Percent Sign), 6 and ^ (6 and circumflex), 7 and & (Seven and Ampersand), 
8 and * (Eight and asterisk), 9 and ( (Nine and Parenthesis Left), 0 and ) (Zero and Parenthesis Right), 
Return (Enter), Escape, Delete (Backspace), Tab, Space Bar, - and _ (Minus and underscore), 
= and + (Equal and Plus), [ and { (Bracket and Braces Left), ] and } (Bracket and Braces Right), 
\ and | (Backslash and Bar), # and ~ (Hash and Tilde, Non-US Keyboard near right shift), 
; and : (Semicolon and Colon), � and " (Accent Acute and Double Quotes), 
` and ~ (Accent Grace and Tilde), , and < (Comma and Less), . and > (Period and Greater), 
/ and ? (Slash and Question Mark), Caps Lock, F1, F2, F3, F4, F5, F6, F7, F8, 
F9, F10, F11, F12, Print Screen, Scroll Lock, Pause, Insert, Home, Page Up, 
Delete Forward (without Changing Position), End, Page Down, Right Arrow, Left Arrow, 
Down Arrow, Up Arrow, Num Lock and Clear, Keypad / (Division Sign), Keypad * (Multiplication Sign), 
Keypad - (Subtraction Sign), Keypad + (Addition Sign), Keypad Enter, Keypad 1 and END, 
Keypad 2 and Down Arrow, Keypad 3 and Page Down, Keypad 4 and Left Arrow, Keypad 5 (Tactilei Raised), 
Unknown, Keypad 6 and Right Arrow, Keypad 7 and Home, Keypad 8 and Up Arrow, 
Keypad 8 and Page Up, Keypad . (decimal delimiter) and Delete, \ and | (Backslash and Bar, UK and Non-US Keyboard near left shift), 
Keyboard Application (Windows Key for Win95 or Compose), Power (not a key), 
Keypad = (Equal Sign), F13, F14, F15, F16, F17, F18, F19, F20, F21, F22, F23, 
F24, Execute, Help, Menu, Select, Stop, Again, Undo, Cut, Copy, Paste, Find, 
Mute, Volume Up, Volume Down, Locking Caps Lock, Locking Num Lock, Locking Scroll Lock, 
Keypad Comma, Keypad Equal Sign (AS/400), International 1 (PC98), International 2 (PC98), 
International 3 (PC98), International 4 (PC98), International 5 (PC98), International 6 (PC98), 
International 7 (Toggle Single/Double Byte Mode), International 8, International 9, 
LANG 1 (Hangul/English Toggle, Korea), LANG 2 (Hanja Conversion, Korea), LANG 3 (Katakana, Japan), 
LANG 4 (Hiragana, Japan), LANG 5 (Zenkaku/Hankaku, Japan), LANG 6, LANG 7, 
LANG 8, LANG 9, Alternate Erase, SysReq/Attention, Cancel, Clear, Prior, Return, 
Separator, Out, Open, Clear/Again, CrSel/Props, ExSel, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Control Left, Shift Left, Alt Left, GUI Left, Control Right, 
Shift Right, Alt Right, GUI Right, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown
Input: 0 (Data,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage Page: 12 (Consumer)
Report Size: 1
Report Count: 1
Usage: 184 (Eject)
Logical Minimum: 0
Logical Maximum: 1
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage Page: 255 (Vendor Specific)
Usage: 3 (Unknown)
Report Size: 7
Report Count: 1
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

End Collection 
Usage Page: 12 (Consumer)
Usage: 1 (Consumer Control)
Collection 
Report ID: 82
Logical Minimum: 0
Logical Maximum: 1
Report Size: 1
Report Count: 1
Usage: 205 (Play/Pause)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 179 (Fast Forward)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 180 (Rewind)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 181 (Scan Next Track)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 182 (Scan Previous Track)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report ID: 9
Logical Minimum: 0
Logical Maximum: 1
Report Size: 8
Report Count: 1
Usage Page: 65281 (Vendor Specific)
Usage: 11 (Unknown)
Feature: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Size: 8
Report Count: 2
Feature: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

End Collection

The USB Controller seems to have listed everything from the above combined I guess:

USB-Controller

Apple Internal Keyboard / Trackpad (Internal Keyboard/Trackpad (ISO))

Device Descriptor:
----------------------------------------------
Connection Information
 Port: 2
 Speed: Full Speed
 Device address: 6
 Open pipes: 3
 Connection status: Device connected

Device Descriptor:
 USB version: 2.0
 Device class: 0x0 - (Defined at Interface level)
 Device subclass: 0x0 - Unknown
 Device protocol: 0x0 - Unknown
 Control pipe max size: 8 bytes
 Vendor ID: 0x5ac (Apple, Inc.)
 Product ID: 0x246 (Internal Keyboard/Trackpad (ISO))
 Product version: 1.8
 Manufacturer: Apple Inc.
 Product: Apple Internal Keyboard / Trackpad
 Serial Number: Not specified
 Configurations: 1

Configuration Descriptor:
----------------------------------------------
descriptors[0] = "Configuration Descriptor" 
bLength = 9 
bDescriptorType = USB_CONFIGURATION_DESCRIPTOR_TYPE (2) 
wTotalLength = 84 
bNumInterfaces = 3 
bConfigurationValue = 1 
iConfiguration = 0 
Reserved = 0 
SupportsRemoteWakeup = 1 
SelfPowered = 0 
PoweredByBus = 1 
MaxPower = 0x14 -> 40 mA 

descriptors[1] = "Interface Descriptor" 
bLength = 9 
bDescriptorType = USB_INTERFACE_DESCRIPTOR_TYPE (4) 
bInterfaceNumber = 0 
bAlternateSetting = 0 
bNumEndpoints = 1 
bInterfaceClass = UsbHID (3) 
bInterfaceSubClass = 1 
bInterfaceProtocol = 1 
iInterface = 3 

descriptors[2] = "HID Descriptor" 
bLength = 9 
bDescriptorType = HID_HID_DESCRIPTOR_TYPE (33) 
bcdHID = 1.11 
bCountry = International (13) 
bNumDescriptors = 1 
reports =  
	reports[0] =  
		bReportType = 34 
		wReportLength = 156 
		raw =  7, 5, 131, 3, 10…  


HID Descriptor:
----------------------------------------------
HID Descriptor
Usage Page: 1 (Generic Desktop Controls)
Usage: 6 (Keyboard)
Collection 
Report ID: 1
Usage Page: 7 (Keyboard)
Usages (224-231): Control Left, Shift Left, Alt Left, GUI Left, Control Right, Shift Right, Alt Right, 
GUI Right
Logical Minimum: 0
Logical Maximum: 1
Report Size: 1
Report Count: 8
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Count: 1
Report Size: 8
Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Count: 5
Report Size: 1
Usage Page: 8 (LEDs)
Usages (1-5): NumLock, CapsLock, Scroll Lock, Compose, Kana
Output: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position,Non Volatile)

Report Count: 1
Report Size: 3
Output: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position,Non Volatile)

Report Count: 6
Report Size: 8
Logical Minimum: 0
Logical Maximum: 255
Usage Page: 7 (Keyboard)
Usages (0-255): No Event, Keyboard ErrorRollOver, Keyboard POSTfail, Keyboard Error Undefined, A, 
B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, 
1 and ! (One and Exclamation), 2 and @ (2 and at), 3 and # (3 and Hash), 4 and $ (4 and Dollar Sign), 
5 and % (5 and Percent Sign), 6 and ^ (6 and circumflex), 7 and & (Seven and Ampersand), 
8 and * (Eight and asterisk), 9 and ( (Nine and Parenthesis Left), 0 and ) (Zero and Parenthesis Right), 
Return (Enter), Escape, Delete (Backspace), Tab, Space Bar, - and _ (Minus and underscore), 
= and + (Equal and Plus), [ and { (Bracket and Braces Left), ] and } (Bracket and Braces Right), 
\ and | (Backslash and Bar), # and ~ (Hash and Tilde, Non-US Keyboard near right shift), 
; and : (Semicolon and Colon), � and " (Accent Acute and Double Quotes), 
` and ~ (Accent Grace and Tilde), , and < (Comma and Less), . and > (Period and Greater), 
/ and ? (Slash and Question Mark), Caps Lock, F1, F2, F3, F4, F5, F6, F7, F8, 
F9, F10, F11, F12, Print Screen, Scroll Lock, Pause, Insert, Home, Page Up, 
Delete Forward (without Changing Position), End, Page Down, Right Arrow, Left Arrow, 
Down Arrow, Up Arrow, Num Lock and Clear, Keypad / (Division Sign), Keypad * (Multiplication Sign), 
Keypad - (Subtraction Sign), Keypad + (Addition Sign), Keypad Enter, Keypad 1 and END, 
Keypad 2 and Down Arrow, Keypad 3 and Page Down, Keypad 4 and Left Arrow, Keypad 5 (Tactilei Raised), 
Unknown, Keypad 6 and Right Arrow, Keypad 7 and Home, Keypad 8 and Up Arrow, 
Keypad 8 and Page Up, Keypad . (decimal delimiter) and Delete, \ and | (Backslash and Bar, UK and Non-US Keyboard near left shift), 
Keyboard Application (Windows Key for Win95 or Compose), Power (not a key), 
Keypad = (Equal Sign), F13, F14, F15, F16, F17, F18, F19, F20, F21, F22, F23, 
F24, Execute, Help, Menu, Select, Stop, Again, Undo, Cut, Copy, Paste, Find, 
Mute, Volume Up, Volume Down, Locking Caps Lock, Locking Num Lock, Locking Scroll Lock, 
Keypad Comma, Keypad Equal Sign (AS/400), International 1 (PC98), International 2 (PC98), 
International 3 (PC98), International 4 (PC98), International 5 (PC98), International 6 (PC98), 
International 7 (Toggle Single/Double Byte Mode), International 8, International 9, 
LANG 1 (Hangul/English Toggle, Korea), LANG 2 (Hanja Conversion, Korea), LANG 3 (Katakana, Japan), 
LANG 4 (Hiragana, Japan), LANG 5 (Zenkaku/Hankaku, Japan), LANG 6, LANG 7, 
LANG 8, LANG 9, Alternate Erase, SysReq/Attention, Cancel, Clear, Prior, Return, 
Separator, Out, Open, Clear/Again, CrSel/Props, ExSel, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Control Left, Shift Left, Alt Left, GUI Left, Control Right, 
Shift Right, Alt Right, GUI Right, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, Unknown, 
Unknown
Input: 0 (Data,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage Page: 12 (Consumer)
Report Size: 1
Report Count: 1
Usage: 184 (Eject)
Logical Minimum: 0
Logical Maximum: 1
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage Page: 255 (Vendor Specific)
Usage: 3 (Unknown)
Report Size: 7
Report Count: 1
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

End Collection 
Usage Page: 12 (Consumer)
Usage: 1 (Consumer Control)
Collection 
Report ID: 82
Logical Minimum: 0
Logical Maximum: 1
Report Size: 1
Report Count: 1
Usage: 205 (Play/Pause)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 179 (Fast Forward)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 180 (Rewind)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 181 (Scan Next Track)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage: 182 (Scan Previous Track)
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report ID: 9
Logical Minimum: 0
Logical Maximum: 1
Report Size: 8
Report Count: 1
Usage Page: 65281 (Vendor Specific)
Usage: 11 (Unknown)
Feature: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Size: 8
Report Count: 2
Feature: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

End Collection 
HID Descriptor
Usage Page: 65280 (Vendor Specific)
Usage: 1 (Unknown)
Collection 
Usage Page: 65280 (Vendor Specific)
Usage: 1 (Unknown)
Logical Minimum: 0
Logical Maximum: 255
Report ID: 68
Report Size: 8
Report Count: 511
Input: 0 (Data,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

End Collection 
HID Descriptor
Usage Page: 1 (Generic Desktop Controls)
Usage: 2 (Mouse)
Collection 
Usage: 1 (Pointer)
Collection 
Usage Page: 9 (Buttons)
Usages (1-3): Button 1 (Primary), Button 2 (Secondary), Button 3 (Tertiary)
Logical Minimum: 0
Logical Maximum: 1
Report ID: 2
Report Count: 3
Report Size: 1
Input: 2 (Data,Variable,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Report Count: 1
Report Size: 5
Input: 1 (Constant,Array,Absolute,No Wrap,Linear,Preferred State,No Null Position)

Usage Page: 1 (Generic Desktop Controls)
Usage: 48 (Direction-X)
Usage: 49 (Direction-Y)
Logical Minimum: -127
Logical Maximum: 127
Report Size: 8
Report Count: 2
Input: 6 (Data,Variable,Relative,No Wrap,Linear,Preferred State,No Null Position)

End Collection 
End Collection

If going with the "take in USB data", "send out USB data" approach, would I need all the pins from a Teensy 4.1 or would a 4.0 work as well? (and would the 4.0 also have the needed functionality?)

Thanks in advance for your time, cheers!

[Eric Damon Stahl]

Alright, I got myself a Teensy 4.1 (only one available without a 6-8 months waiting time here at the moment).
I ran the HIDDeviceInfo from the USBHost_t36 example. It spits out the following:

USB HID Device Info Program

This Sketch shows information about plugged in HID devices

*** You can control the output by simple character input to Serial ***
R - Turns on or off showing the raw data
C - Toggles showing changed data only on or off
<anything else> - toggles showing the Hid formatted breakdown of the data


USBDeviceInfo claim this=2000CDF4

****************************************
** Device Level **
  vid=5AC
  pid=246
  bDeviceClass = 0
  bDeviceSubClass = 0
  bDeviceProtocol = 0
09 04 00 00 01 03 01 01 03 09 21 11 01 0D 01 22 9C 00 07 05 83 03 0A 00 08 09 04 01 00 01 03 00
00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00
01 22 34 00 07 05 84 03 08 00 08 

USBDeviceInfo claim this=2000CDF4

****************************************
** Interface Level **
09 04 00 00 01 03 01 01 03 09 21 11 01 0D 01 22 9C 00 07 05 83 03 0A 00 08 09 04 01 00 01 03 00
00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00
01 22 34 00 07 05 84 03 08 00 08 
 bInterfaceNumber = 0
 number end points = 1
 bInterfaceClass =    3
 bInterfaceSubClass = 1
    HID (BOOT)
 bInterfaceProtocol = 1
report descriptor size = 156
  endpoint = 83
    attributes = 3 Interrupt
    size = 10
    interval = 8

USBDeviceInfo claim this=2000CDF4

****************************************
** Interface Level **
09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01
02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
 bInterfaceNumber = 1
 number end points = 1
 bInterfaceClass =    3
 bInterfaceSubClass = 0
    HID
 bInterfaceProtocol = 0
report descriptor size = 27
  endpoint = 81
    attributes = 3 Interrupt
    size = 64
    interval = 2

USBDeviceInfo claim this=2000CDF4

****************************************
** Interface Level **
09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
 bInterfaceNumber = 2
 number end points = 1
 bInterfaceClass =    3
 bInterfaceSubClass = 1
    HID (BOOT)
 bInterfaceProtocol = 2
report descriptor size = 52
  endpoint = 84
    attributes = 3 Interrupt
    size = 8
    interval = 8
HIDDumpController Claim: 5ac:246 usage: ff000001 - Yes
*** Device HID1 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** Device HID2 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** HID Device hdc1 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
HIDDumpController Claim: 5ac:246 usage: 10002 - NO (Usage: 0)
HIDDumpController Claim: 5ac:246 usage: 10002 - Yes
*** HID Device hdc2 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
HID(ff000001): 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
END:

When I touch any keys on the keyboard, nothing happens.
When I move my finger on the touchpad I get outputs like this:

HID(10002): 02 00 00 01 00 00 00 00 
Begin topusage:10000 type:2 min:0 max:1
  usage=90001, value=0 (BUTTON 1)
  usage=90002, value=0 (BUTTON 2)
  usage=90003, value=0 (BUTTON 3)
  Begin topusage:10000 type:6 min:-127 max:127
    usage=10030, value=0(X)
    usage=10031, value=1(Y)
  END:

Comparing the USBDeviceInfo with the HID descriptors I don't exactly know what to make of it.

The first interface has a report descriptor size of 156, this is the keyboard (as the keyboard part on Windows also shows a wReportLength of 156).
The second interface has a size of 27, which fits with the "first" touchpad on Windows which is also 27. (This one would be the one reporting multi-touch)
The third interface has a size of 52, which again fits with the "second" touchpad on Windows which is also 52. (This one would be the one acting as a simple mouse)

The packages I receive when moving my finger on the touchpad looks like the "second" touchpad that I see on windows. At least it has the same -127 to +127 min/max. And it seems to ignore additional fingers.

How can I - in the HIDDeviceInfo sketch - receive the input from the other two interfaces / endpoints (aka keyboard and the "first" touchpad)?

[KurtE]

You might try running it again, but turn on more debug stuff in the library

In usbhost_t36.h uncomment the line at about line 62
//#define USBHOST_PRINT_DEBUG

rebuild the sketch and it should print out a lot more details, which might help

[Eric Damon Stahl]

Cheers, now it spits out much more stuff indeed:

USB HID Device Info Program

This Sketch shows information about plugged in HID devices

*** You can control the output by simple character input to Serial ***
R - Turns on or off showing the raw data
C - Toggles showing changed data only on or off
<anything else> - toggles showing the Hid formatted breakdown of the data

USB2 PLL running
 reset waited 4
USBHS_ASYNCLISTADDR = 0
USBHS_PERIODICLISTBASE = 20005000
periodictable = 20005000
port change: 10001803
    connect
  begin reset
port change: 10001805
  port enabled
  end recovery
new_Device: 12 Mbit/sec
new_Pipe
enumeration:
enumeration:
enumeration:
Device Descriptor:
  12 01 00 02 00 00 00 08 AC 05 46 02 08 01 01 02 00 01 
    VendorID = 05AC, ProductID = 0246, Version = 0108
    Class/Subclass/Protocol = 0 / 0 / 0
    Number of Configurations = 1
enumeration:
enumeration:
Manufacturer: Apple Inc.
enumeration:
Product: Apple Internal Keyboard / Trackpad
enumeration:
Config data length = 84
enumeration:
Configuration Descriptor:
  09 02 54 00 03 01 00 A0 14 
    NumInterfaces = 3
    ConfigurationValue = 1
  09 04 00 00 01 03 01 01 03 
    Interface = 0
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 1(Boot) / 1(Keyboard)
  09 21 11 01 0D 01 22 9C 00 
    HID, 1 report descriptor
  07 05 83 03 0A 00 08 
    Endpoint = 3 IN
    Type = Interrupt
    Max Size = 10
    Polling Interval = 8
  09 04 01 00 01 03 00 00 04 
    Interface = 1
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 0 / 0
  09 21 11 01 00 01 22 1B 00 
    HID, 1 report descriptor
  07 05 81 03 40 00 02 
    Endpoint = 1 IN
    Type = Interrupt
    Max Size = 64
    Polling Interval = 2
  09 04 02 00 01 03 01 02 04 
    Interface = 2
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 1(Boot) / 2(Mouse)
  09 21 11 01 00 01 22 34 00 
    HID, 1 report descriptor
  07 05 84 03 08 00 08 
    Endpoint = 4 IN
    Type = Interrupt
    Max Size = 8
    Polling Interval = 8
enumeration:
USBHub memory usage = 960
USBHub claim_device this=2000C760
USBHub memory usage = 960
USBHub claim_device this=2000CB20

USBDeviceInfo claim this=2000DDF4

****************************************
** Device Level **
  vid=5AC
  pid=246
  bDeviceClass = 0
  bDeviceSubClass = 0
  bDeviceProtocol = 0
09 04 00 00 01 03 01 01 03 09 21 11 01 0D 01 22 9C 00 07 05 83 03 0A 00 08 09 04 01 00 01 03 00
00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00
01 22 34 00 07 05 84 03 08 00 08 
HIDParser claim this=2000A5A0
HIDParser claim this=2000AC60
HIDParser claim this=2000B320
HIDParser claim this=2000B9E0
HIDParser claim this=2000C0A0
Descriptor 4 = INTERFACE

USBDeviceInfo claim this=2000DDF4

****************************************
** Interface Level **
09 04 00 00 01 03 01 01 03 09 21 11 01 0D 01 22 9C 00 07 05 83 03 0A 00 08 09 04 01 00 01 03 00
00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00
01 22 34 00 07 05 84 03 08 00 08 
 bInterfaceNumber = 0
 number end points = 1
 bInterfaceClass =    3
 bInterfaceSubClass = 1
    HID (BOOT)
 bInterfaceProtocol = 1
report descriptor size = 156
  endpoint = 83
    attributes = 3 Interrupt
    size = 10
    interval = 8
HIDParser claim this=2000A5A0
 bInterfaceNumber =   0
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 1
HIDParser claim this=2000AC60
 bInterfaceNumber =   0
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 1
HIDParser claim this=2000B320
 bInterfaceNumber =   0
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 1
HIDParser claim this=2000B9E0
 bInterfaceNumber =   0
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 1
HIDParser claim this=2000C0A0
 bInterfaceNumber =   0
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 1
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
Descriptor 4 = INTERFACE

USBDeviceInfo claim this=2000DDF4

****************************************
** Interface Level **
09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01
02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
 bInterfaceNumber = 1
 number end points = 1
 bInterfaceClass =    3
 bInterfaceSubClass = 0
    HID
 bInterfaceProtocol = 0
report descriptor size = 27
  endpoint = 81
    attributes = 3 Interrupt
    size = 64
    interval = 2
HIDParser claim this=2000A5A0
 bInterfaceNumber =   1
 bInterfaceClass =    3
 bInterfaceSubClass = 0
 bInterfaceProtocol = 0
HID Parser Claim: 09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
report descriptor size = 27
Single endpoint HID:
  endpoint = 81
   size = 64
   interval = 2
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 5, at offset = 0, shift= 0
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
Descriptor 4 = INTERFACE

USBDeviceInfo claim this=2000DDF4

****************************************
** Interface Level **
09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
 bInterfaceNumber = 2
 number end points = 1
 bInterfaceClass =    3
 bInterfaceSubClass = 1
    HID (BOOT)
 bInterfaceProtocol = 2
report descriptor size = 52
  endpoint = 84
    attributes = 3 Interrupt
    size = 8
    interval = 8
HIDParser claim this=2000AC60
 bInterfaceNumber =   2
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 2
HID Parser Claim: 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
report descriptor size = 52
Single endpoint HID:
  endpoint = 84
   size = 8
   interval = 8
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 3, at offset = 1, shift= 0
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
control callback (hid)
06 00 FF 09 01 A1 03 06 00 FF 09 01 15 00 26 FF 00 85 44 75 08 96 FF 01 81 00 C0 
  mesg = 22000681
  got report descriptor
Found top level collection FF000001
find_driver
  driver 200051E0
HIDDumpController Claim: 5ac:246 usage: ff000001 - Yes
*** Device HID2 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** HID Device hdc1 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** Device HID1 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
control callback (hid)
05 01 09 02 A1 01 09 01 A1 00 05 09 19 01 29 03 15 00 25 01 85 02 95 03 75 01 81 02 95 01 75 05 81 01 05 01 09 30 09 31 15 81 25 7F 75 08 95 02 81 06 C0 C0 
  mesg = 22000681
  got report descriptor
Found top level collection 10002
find_driver
  driver 200051E0
HIDDumpController Claim: 5ac:246 usage: 10002 - NO (Usage: 0)
  driver 200062A0
HIDDumpController Claim: 5ac:246 usage: 10002 - Yes
*** HID Device hdc2 5ac: 246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
HID(ff000001): 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
END:

And the print for the data package is now:

HID(10002): 02 00 00 01 00 00 00 00 
begin, usage=10000
       type= 2
       min=  0
       max=  1
       reportcount=3
       usage count=255
Begin topusage:10000 type:2 min:0 max:1
Input, total bits=3
  usage = 90001  data = 0
  usage=90001, value=0 (BUTTON 1)
  usage = 90002  data = 0
  usage=90002, value=0 (BUTTON 2)
  usage = 90003  data = 0
  usage=90003, value=0 (BUTTON 3)
begin, usage=10000
       type= 6
       min=  -127
       max=  127
       reportcount=2
       usage count=2
  Begin topusage:10000 type:6 min:-127 max:127
Input, total bits=16
  usage = 10030  sdata = 0
    usage=10030, value=0(X)
  usage = 10031  sdata = 1
    usage=10031, value=1(Y)
  END:

What does it tell me? That it detects the keyboard as keyboard, the second touchpad as mouse, and the first touchpad as nothing?
That it only did a "control callback (hid)" for the second touchpad and not for the first touchpad and the keyboard?

What exactly does that mean? I'd guess that it's the reason I only receive data from 1 of the 3 interfaces?

[KurtE]

For the heck of it, try the USBHost_t36 example mouse and see what it gives you.

The mouse sketch is setup to handle up to 2 keyboards, a Mouse a joystick...

And it should be able to talk to the keyboard and the like.

[Eric Damon Stahl]

The mouse sketch also gives me input only of the "second" touchpad, nothing from the keyboard.

In the HIDDeviceInfo I read something about drivers. Personally, at this point, I wouldn't care wether it works with a driver or not. I can probably spend a few days on trying to figure out the packages myself (I already have the formulas for detecting move, tap, tap and hold, swipe, pinch, rotate gestures... for 1-10 fingers... from a different project of mine), as long as I know how to get them (from the interfaces) into my sketch.

For the record.... the keyboard part ALWAYS works everywhere else... in my BIOS, Windows PE, Windows 10, Ubuntu, Android, MacOS, any VM. Never had any issues with it.
But yea I need the raw packages anyways for the trackpad itself.

Oh, also I don't know if that would be of any help but, the driver being used on Android_x86 and Linux would be available I guess. It's for the multi touch controller BCM5974 which pretty much handles every MacBook Pro from 2008 to 2015 I believe.

[Eric Damon Stahl]

(I tried to edit the previous post, but it seems after a while it's not possible?)

I forgot the output from the mouse sketch.

USB Host Testing
960
USB2 PLL running
 reset waited 4
USBHS_ASYNCLISTADDR = 0
USBHS_PERIODICLISTBASE = 20005000
periodictable = 20005000
port change: 10001803
    connect
  begin reset
port change: 10001805
  port enabled
  end recovery
new_Device: 12 Mbit/sec
new_Pipe
enumeration:
enumeration:
enumeration:
Device Descriptor:
  12 01 00 02 00 00 00 08 AC 05 46 02 08 01 01 02 00 01 
    VendorID = 05AC, ProductID = 0246, Version = 0108
    Class/Subclass/Protocol = 0 / 0 / 0
    Number of Configurations = 1
enumeration:
enumeration:
Manufacturer: Apple Inc.
enumeration:
Product: Apple Internal Keyboard / Trackpad
enumeration:
Config data length = 84
enumeration:
Configuration Descriptor:
  09 02 54 00 03 01 00 A0 14 
    NumInterfaces = 3
    ConfigurationValue = 1
  09 04 00 00 01 03 01 01 03 
    Interface = 0
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 1(Boot) / 1(Keyboard)
  09 21 11 01 0D 01 22 9C 00 
    HID, 1 report descriptor
  07 05 83 03 0A 00 08 
    Endpoint = 3 IN
    Type = Interrupt
    Max Size = 10
    Polling Interval = 8
  09 04 01 00 01 03 00 00 04 
    Interface = 1
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 0 / 0
  09 21 11 01 00 01 22 1B 00 
    HID, 1 report descriptor
  07 05 81 03 40 00 02 
    Endpoint = 1 IN
    Type = Interrupt
    Max Size = 64
    Polling Interval = 2
  09 04 02 00 01 03 01 02 04 
    Interface = 2
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 1(Boot) / 2(Mouse)
  09 21 11 01 00 01 22 34 00 
    HID, 1 report descriptor
  07 05 84 03 08 00 08 
    Endpoint = 4 IN
    Type = Interrupt
    Max Size = 8
    Polling Interval = 8
enumeration:
USBHub memory usage = 960
USBHub claim_device this=20007AA0
USBHub memory usage = 960
USBHub claim_device this=20007E60
KeyboardController claim this=20008780
KeyboardController claim this=20008A20
HIDParser claim this=200058E0
HIDParser claim this=20005FA0
HIDParser claim this=20006660
HIDParser claim this=20006D20
HIDParser claim this=200073E0
JoystickController claim this=20008220
BluetoothController claim this=200051E0
Descriptor 4 = INTERFACE
KeyboardController claim this=20008780
09 04 00 00 01 03 01 01 03 09 21 11 01 0D 01 22 9C 00 07 05 83 03 0A 00 08 09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
ep = 83
packet size = 10
polling interval = 8
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 3, at offset = 0, shift= 0
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
Descriptor 4 = INTERFACE
KeyboardController claim this=20008A20
09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
HIDParser claim this=200058E0
 bInterfaceNumber =   1
 bInterfaceClass =    3
 bInterfaceSubClass = 0
 bInterfaceProtocol = 0
HID Parser Claim: 09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
report descriptor size = 27
Single endpoint HID:
  endpoint = 81
   size = 64
   interval = 2
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 5, at offset = 0, shift= 3
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
Descriptor 4 = INTERFACE
KeyboardController claim this=20008A20
09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
HIDParser claim this=20005FA0
 bInterfaceNumber =   2
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 2
HID Parser Claim: 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
report descriptor size = 52
Single endpoint HID:
  endpoint = 84
   size = 8
   interval = 8
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 3, at offset = 1, shift= 0
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
control callback (keyboard)
  mesg = A21
control callback (hid)
06 00 FF 09 01 A1 03 06 00 FF 09 01 15 00 26 FF 00 85 44 75 08 96 FF 01 81 00 C0 
  mesg = 22000681
  got report descriptor
Found top level collection FF000001
find_driver
  driver 2000878C
  driver 20008A2C
  driver 20009CC4
  driver 2000822C
  driver 20008CC0
Rawhid Claim: 5ac:246 usage: ff000001
  driver 20008D60
Rawhid Claim: 5ac:246 usage: ff000001
No Driver claimed topusage: FF000001
*** Device KB1 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** Device HID1 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** Device HID2 5ac:246 - connected ***
  manufacturer: Apple Inc.
control callback (hid)
05 01 09 02 A1 01 09 01 A1 00 05 09 19 01 29 03 15 00 25 01 85 02 95 03 75 01 81 02 95 01 75 05 81 01 05 01 09 30 09 31 15 81 25 7F 75 08 95 02 81 06 C0 C0 
  mesg = 22000681
  got report descriptor
Found top level collection 10002
find_driver
  driver 2000878C
  driver 20008A2C
  driver 20009CC4
  product: Apple Internal Keyboard / Trackpad
*** HID Device Mouse1 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
KeyboardController Callback (member)
  KB Data: 01 06 00 00 00 00 00 00 
ERROR Followup

[KurtE]

Again sort of interesting that the keyboard is not working... I am curious why.

That is with the information: Class/Subclass/Protocol = 3(HID) / 1(Boot) / 1(Keyboard)

There are specific things this should specify... That it follows the USB boot keyboard... 4 key rollover... That the code is setup to handle. So it would be interesting to see if the keyboard driver is claiming the device or not... The debug output should show it.

If it claims it, does it show any debug output when you type a key? Again would be interesting to see what it generates.

I would try seeing if sending message to force keyboard into boot mode would help. A few keyboards say they are boot, but don't initially work that way. again in keyboard.cpp about line 97
there is a list of PID/VID, to send the IOCTL to force into boot mode... Try adding you PID to list and see if that makes difference.


Interface 1 - again would be interesting to see what this one is:
Very likely it is part of the keyboard. That is as BOOT keyboard is very specific, for normal keys, it does not handle extended things for example like multi-media keys or system level keys... So many(most) keyboards have 2nd HID interface defined, where they have some standard USB USAGE pages defined where many of these keys are defined. Currently our Keyboard class is looking at the two pages:
#define TOPUSAGE_SYS_CONTROL 0x10080
#define TOPUSAGE_CONSUMER_CONTROL 0x0c0001

I don't see these here, but the simple HID sketch is there to hopefully give some of the simple data... So is it now showing any data coming in at all for keyboard. Are there any keys on the keyboard like volume up? if so pressing these does it show anything.

If I were debugging with hardware, I would uncomment some addition debug data. like in keyboard.cpp in the claim_collection member I would uncomment the couple of prints.


Interface 2 shows up as Class/Subclass/Protocol = 3(HID) / 1(Boot) / 2(Mouse)
So it again should have a specific defined data...

Are we seeing the data correctly. And/or would the DigitizerController class work better. We have only tried a few of these.

Hope some of this helps

[Eric Damon Stahl]

Ah, you've sent your message while I was editing my previous one.

So I went through the code and found the following:

        // Note: with some keyboards there is an issue that they don't output in boot protocol mode
        // and may not work.  The above code can try to force the keyboard into boot mode, but there
        // are issues with doing this blindly with combo devices like wireless keyboard/mouse, which
        // may cause the mouse to not work.  Note: the above id is in the builtin list of
        // vendor IDs that are already forced
        if (drivers[i] == &keyboard1) {
          if (keyboard1.idVendor() == 0x04D9) {
            Serial.println("Gigabyte vendor: force boot protocol");
            // Gigabyte keyboard
            keyboard1.forceBootProtocol();
          }
        }

I changed it so it forces it:

if (drivers[i] == &keyboard1) {
  keyboard1.forceBootProtocol();
}

Now the keyboard works. It was actually the same thing you suggested, just that I did it in a different place.
The MacBook has multimedia keys (FN + F-keys), but it is set to use the F-keys as standard. And since I damaged some of the original circuit the keyboard is somewhat broken (for example FN doesn't work), so I can't test if the multimedia keys would work normally.

Interface 0 is the keyboard, interface 2 is the "second" touchpad with pure mouse funcionality.
Interface 1 is the "first" touchpad containing the multi-touch that I need. At least I'm pretty sure it is, I am comparing the descriptor sizes of the 3 interfaces, and they match up with the ones I read out in Windows.


In Windows, without the Apple Touchpad drivers installed it ignores interface 1 and uses interface 2 as a plain mouse. With the drivers installed it ignores interface 2 and uses interface 1 instead.

I'd lke to also ignore (not take ownership?) of interface 2... and read out the raw packages from interface 1 (or maybe a DigitizerController would be able to interpret the packages? If yes I have no idea how to use it).

And yea, it does help indeed. Thanks a lot already for your time!

[KurtE]

Note: The data showed: Found top level collection FF000001

So if you look at the USage Hid tables, which you can download from:

https://usb.org/document-library/hid-usage-tables-13

TopWord: FF00 that this is in the range FF00=FFFF which is vendor defined...

i.e. no standard... But that does not stop anyone from setting up class to claim it it..

For example the hid info sketch, my hid dumper looks like it did receive one packet like:

HID(ff000001): 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
END:


But I am not seeing anything that breaks this down into fields...

[Eric Damon Stahl]

I tried reading up a little on all of it, but as I've never done anything at driver and/or USB level it's quite difficult for me.

I have the source code od the driver (Android... ported from Linux from what I know) for the Keyboard/Trackpad combination with the BCM5974 controller here:
https://android.googlesource.com/ker...ouse/bcm5974.c

My product id is 0246 which is included in there.

If you have a look at lines 593+, it does some initializing upon connecting ("starting traffic", line 731). May it be that the trackpad by default runs in "mouse-mode" and, if there is a driver available, switches upon request by the host to multi-touch? This would explain why on Windows interface 2 is used without driver and interface 1 as soon as the driver is installed. Could that further explain why your hid dumper got a packet that looks kind of "empty", because the trackpad didn't switch to interface_1-mode and therefore doesn't bother to send data there?

(And out of curiosity, any idea why native Android devices can't / don't use the mouse-mode of the trackpad whereas literally every other device/OS does?)


In the Windows driver .inf file I found:

[AmtPtpDeviceUsbUm_Install.NT.Services]
AddService=mshidumdf, 0x000001fa, MSHIDUMDF_ServiceInstall ; flag 0x2 sets this as the service for the device
AddService=WUDFRd,0x000001f8,WUDFRD_ServiceInstall         ; this service is installed because its a filter.

Not sure this belongs to the top level collection FF000001 ?
Following the mshidumdf (supposed to be "Pass-through Driver for HID-UMDF Interface") service, later in the file there is:

[mshidumdf_ServiceInstall]
ServiceType    = 1                  ; SERVICE_KERNEL_DRIVER
StartType      = 3                  ; SERVICE_DEMAND_START
ErrorControl   = 1                  ; SERVICE_ERROR_NORMAL
ServiceBinary  = %12%\mshidumdf.sys

[Eric Damon Stahl]

Note: The data showed: Found top level collection FF000001

So if you look at the USage Hid tables, which you can download from:

https://usb.org/document-library/hid-usage-tables-13

TopWord: FF00 that this is in the range FF00=FFFF which is vendor defined...

i.e. no standard... But that does not stop anyone from setting up class to claim it it..

For example the hid info sketch, my hid dumper looks like it did receive one packet like:

HID(ff000001): 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
END:


But I am not seeing anything that breaks this down into fields...

Alright, I had a lot of stuff going on in my live lately, but I should have some free time for my project again now.
I was in the process of writing a huge, huge reply with what I tried and my findings when I... well, kind of got it (but not completely).
So let me summarize and ask some questions.

I used a USB sniffer on Windows and Linux and went through the packets (and learning a bit about how USB communication works) with Wireshark. In the following image it shows the first 51 packets after plugging in the device.
In packet #43 we can see the above packet from your hid dumper: 60 02
(In the first hours I missed this 60 02 packet! That cost me quite some time)

After that a GET_REPORT, SET_REPORT (Data: 08 05 00 00), GET_REPORT and another SET_REPORT (Data: 01 05 00 00)



The corresponding part in the Linux driver is this:

/* Wellspring initialization constants */
#define BCM5974_WELLSPRING_MODE_READ_REQUEST_ID		1
#define BCM5974_WELLSPRING_MODE_WRITE_REQUEST_ID	9
#define BCM5974_WELLSPRING_MODE_REQUEST_VALUE		0x300
#define BCM5974_WELLSPRING_MODE_REQUEST_INDEX		0
#define BCM5974_WELLSPRING_MODE_VENDOR_VALUE		0x01
#define BCM5974_WELLSPRING_MODE_NORMAL_VALUE		0x08

static int bcm5974_wellspring_mode(struct bcm5974 *dev, bool on)
{
	char *data = kmalloc(8, GFP_KERNEL);
	int retval = 0, size;
	if (!data) {
		err("bcm5974: out of memory");
		retval = -ENOMEM;
		goto out;
	}
	/* read configuration */
	size = usb_control_msg(dev->udev, usb_rcvctrlpipe(dev->udev, 0),
			BCM5974_WELLSPRING_MODE_READ_REQUEST_ID,
			USB_DIR_IN | USB_TYPE_CLASS | USB_RECIP_INTERFACE,
			BCM5974_WELLSPRING_MODE_REQUEST_VALUE,
			BCM5974_WELLSPRING_MODE_REQUEST_INDEX, data, 8, 5000);
	if (size != 8) {
		err("bcm5974: could not read from device");
		retval = -EIO;
		goto out;
	}
	/* apply the mode switch */
	data[0] = on ?
		BCM5974_WELLSPRING_MODE_VENDOR_VALUE :
		BCM5974_WELLSPRING_MODE_NORMAL_VALUE;
	/* write configuration */
	size = usb_control_msg(dev->udev, usb_sndctrlpipe(dev->udev, 0),
			BCM5974_WELLSPRING_MODE_WRITE_REQUEST_ID,
			USB_DIR_OUT | USB_TYPE_CLASS | USB_RECIP_INTERFACE,
			BCM5974_WELLSPRING_MODE_REQUEST_VALUE,
			BCM5974_WELLSPRING_MODE_REQUEST_INDEX, data, 8, 5000);
	if (size != 8) {
		err("bcm5974: could not write to device");
		retval = -EIO;
		goto out;
	}
	dprintk(2, "bcm5974: switched to %s mode.\n",
		on ? "wellspring" : "normal");
 out:
	kfree(data);
	return retval;
}

While comparing the packets with the driver code, I found the values marked in red and they fit with the packets.



I then found this in the hid.cpp of USBHost_t36...

	// request the HID report descriptor
	bInterfaceNumber = descriptors[2];	// save away the interface number; 
	mk_setup(setup, 0x81, 6, 0x2200, descriptors[2], descsize); // get report desc
	queue_Control_Transfer(dev, &setup, descriptor, this);

... and tried different places in the code to send it. After a few hours (as I missed the packet in Wireshark at first) I settled to put it in the hid.cpp USBHIDPARSER::in_data :

	// Check for the first HID packet (60 02) and switch mode - UXP
	if ((buf[0] == 0x60) && (buf[1] == 0x02))
	{
		USBHDBGSerial.printf("BCM5974: Received packet (60 02), initiating mode switch\n");
		// 0x01 = Vendor mode = wellspring mode (multitouch)
		// 0x08 = Normal mode = mouse mode
		uint8_t	txbuf8[8];
		txbuf8[0] = 0x01; txbuf8[1] = 0x05; txbuf8[2] = 0x00; txbuf8[3] = 0x00;
		txbuf8[4] = 0x00; txbuf8[5] = 0x00; txbuf8[6] = 0x00; txbuf8[7] = 0x00;

		mk_setup(uxpsetup, 0x21, 0x09, 0x0300, 0, 8);
		queue_Control_Transfer(uxpdevice, &uxpsetup, txbuf8, this);
		USBHDBGSerial.printf("BCM5974: Wellspring mode activated\n");
	}

Then I cloned the mouse.ino and mouse.cpp.
Named .cpp touchpad.cpp and replaced every occurance of "mouse" with "touchpad".
Under TouchpadController::claim_collection I set it to claim the topusage FF000001.

	// only claim Touchpad
	if (topusage != 0xff000001) return CLAIM_NO;

In the new touchpad.ino I copied every occurance of "mouse" and added a "touchpad".



Without the mode switch I get the normal mouse packages:

HID: 1 - 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
HID: 1 - 02 00 03 FD 00 00 00 00 
begin, usage=10000
       type= 2
       min=  0
       max=  1
       reportcount=3
       usage count=255
Input, total bits=3
  usage = 90001  data = 0
  usage = 90002  data = 0
  usage = 90003  data = 0
begin, usage=10000
       type= 6
       min=  -127
       max=  127
       reportcount=2
       usage count=2
Input, total bits=16
  usage = 10030  sdata = 3
  usage = 10031  sdata = -3
Mouse: buttons = 0,  mouseX = 3,  mouseY = -3,  wheel = 0,  wheelH = 0

After the mode switch:
("Relevant" stuff marked in red)

USB Host Testing
960
USB2 PLL running
 reset waited 4
USBHS_ASYNCLISTADDR = 0
USBHS_PERIODICLISTBASE = 20005000
periodictable = 20005000
port change: 10001803
    connect
  begin reset
port change: 10001805
  port enabled
  end recovery
new_Device: 12 Mbit/sec
new_Pipe
enumeration:
enumeration:
enumeration:
Device Descriptor:
  12 01 00 02 00 00 00 08 AC 05 46 02 08 01 01 02 00 01 
    VendorID = 05AC, ProductID = 0246, Version = 0108
    Class/Subclass/Protocol = 0 / 0 / 0
    Number of Configurations = 1
enumeration:
enumeration:
Manufacturer: Apple Inc.
enumeration:
Product: Apple Internal Keyboard / Trackpad
enumeration:
Config data length = 84
enumeration:
Configuration Descriptor:
  09 02 54 00 03 01 00 A0 14 
    NumInterfaces = 3
    ConfigurationValue = 1
  09 04 00 00 01 03 01 01 03 
    Interface = 0
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 1(Boot) / 1(Keyboard)
  09 21 11 01 0D 01 22 9C 00 
    HID, 1 report descriptor
07 05 83 03 0A 00 08 
    Endpoint = 3 IN
    Type = Interrupt
    Max Size = 10
    Polling Interval = 8
  09 04 01 00 01 03 00 00 04 
    Interface = 1
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 0 / 0
  09 21 11 01 00 01 22 1B 00 
    HID, 1 report descriptor
  07 05 81 03 40 00 02 
    Endpoint = 1 IN
    Type = Interrupt
    Max Size = 64
    Polling Interval = 2
  09 04 02 00 01 03 01 02 04 
    Interface = 2
    Number of endpoints = 1
    Class/Subclass/Protocol = 3(HID) / 1(Boot) / 2(Mouse)
  09 21 11 01 00 01 22 34 00 
    HID, 1 report descriptor
  07 05 84 03 08 00 08 
    Endpoint = 4 IN
    Type = Interrupt
    Max Size = 8
    Polling Interval = 8
enumeration:
USBHub memory usage = 960
USBHub claim_device this=20007AA0
USBHub memory usage = 960
USBHub claim_device this=20007E60
KeyboardController claim this=20008780
KeyboardController claim this=20008A20
HIDParser claim this=200058E0
HIDParser claim this=20005FA0
HIDParser claim this=20006660
HIDParser claim this=20006D20
HIDParser claim this=200073E0
JoystickController claim this=20008220
BluetoothController claim this=200051E0
Descriptor 4 = INTERFACE
KeyboardController claim this=20008780
09 04 00 00 01 03 01 01 03 09 21 11 01 0D 01 22 9C 00 07 05 83 03 0A 00 08 09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
ep = 83
packet size = 10
polling interval = 8
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 3, at offset = 0, shift= 0
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
Descriptor 4 = INTERFACE
KeyboardController claim this=20008A20
09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
HIDParser claim this=200058E0
 bInterfaceNumber =   1
 bInterfaceClass =    3
 bInterfaceSubClass = 0
 bInterfaceProtocol = 0
HID Parser Claim: 09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
report descriptor size = 27
Single endpoint HID:
  endpoint = 81
   size = 64
   interval = 2
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 5, at offset = 0, shift= 3
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
Descriptor 4 = INTERFACE
KeyboardController claim this=20008A20
09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
HIDParser claim this=20005FA0
 bInterfaceNumber =   2
 bInterfaceClass =    3
 bInterfaceSubClass = 1
 bInterfaceProtocol = 2
HID Parser Claim: 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
report descriptor size = 52
Single endpoint HID:
  endpoint = 84
   size = 8
   interval = 8
new_Pipe
allocate_interrupt_pipe_bandwidth
 best_bandwidth = 3, at offset = 1, shift= 0
Descriptor 33 = HID
Descriptor 5 = ENDPOINT
control callback (keyboard)
  mesg = A21
control callback (hid)
06 00 FF 09 01 A1 03 06 00 FF 09 01 15 00 26 FF 00 85 44 75 08 96 FF 01 81 00 C0 
  mesg = 22000681
  got report descriptor
Found top level collection FF000001
find_driver
  driver 2000878C
  driver 20008A2C
  driver 2000C998
  driver 2000CA44
UXP Touchpad claim ff000001
*** Device HID1 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** Device HID2 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** HID Device Touchpad1 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
*** Device KB1 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
control callback (hid)
05 01 09 02 A1 01 09 01 A1 00 05 09 19 01 29 03 15 00 25 01 85 02 95 03 75 01 81 02 95 01 75 05 81 01 05 01 09 30 09 31 15 81 25 7F 75 08 95 02 81 06 C0 C0 
  mesg = 22000681
  got report descriptor
Found top level collection 10002
find_driver
  driver 2000878C
  driver 20008A2C
  driver 2000C998
control callback (keyboard)
  mesg = B21
*** HID Device Mouse1 5ac:246 - connected ***
  manufacturer: Apple Inc.
  product: Apple Internal Keyboard / Trackpad
control callback (keyboard)
  mesg = A21
KeyboardController Callback (member)
  KB Data: 02 00 45 51 00 00 00 00 
  press, key=69
  unicode = 205
key 'F12'  205 MOD: 2 OEM: 45 LEDS: 0
  press, key=81
  unicode = 217
key 'DN'  217 MOD: 2 OEM: 51 LEDS: 0
KeyboardController Callback (member)
  KB Data: 00 00 45 51 00 00 00 00 
KeyboardController Callback (member)
  KB Data: 02 00 45 51 00 00 00 00 
UXP in_callback
UXP in_callback TRANSFER->DRIVER = TRUE
HID: 1 - 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
UXP Touchpad hid_input_end
BCM5974: Received packet (60 02), initiating mode switch
BCM5974: Wellspring mode activated
control callback (hid)
00 00 00 00 C0 8D 00 20 
  mesg = 3000921
UXP in_callback
UXP in_callback TRANSFER->DRIVER = TRUE
HID: 1 - 74 47 1C 03 9D 3A 00 00 00 17 07 97 1C 00 01 00 80 08 00 00 0B 00 00 00 00 00 00 00 01 02 02 01 90 00 00 04 90 FF B0 00 4A 04 5E 03 90 59 34 00 42 00 00 00 00 00 00 00 FD 07 00 00 00 00 00 00 
UXP Touchpad hid_input_end

So the "new" touchpad-driver is connected, the mode switch gets applied. Now the device doesn't send mouse packets anymore and the packets I have been looking for all the time.





Now for my questions:

1) The Windows/Linux drivers there is a GET_REPORT before a SET_REPORT is sent.

/* read configuration */
	size = usb_control_msg(dev->udev, usb_rcvctrlpipe(dev->udev, 0),
			BCM5974_WELLSPRING_MODE_READ_REQUEST_ID,
			USB_DIR_IN | USB_TYPE_CLASS | USB_RECIP_INTERFACE,
			BCM5974_WELLSPRING_MODE_REQUEST_VALUE,
			BCM5974_WELLSPRING_MODE_REQUEST_INDEX, data, 8, 5000);
	if (size != 8) {
		err("bcm5974: could not read from device");
		retval = -EIO;
		goto out;
	}

Can I do this in USBHost_t36 ? I mean I can send the packet, but I won't have the response.
static bool queue_Control_Transfer(Device_t *dev, setup_t *setup, void *buf, USBDriver *driver) just implies a return value wether the packet has been sent or not.



2) How can I forward the packets I receive now to my driver? I put debug messages all over my touchpad.cpp

void TouchpadController::hid_input_begin(uint32_t topusage, uint32_t type, int lgmin, int lgmax) {
	USBHDBGSerial.printf("Touchpad hid_input_begin\n");

void TouchpadController::hid_input_data(uint32_t usage, int32_t value) {
	USBHDBGSerial.printf("Touchpad hid_input_data\n");

void TouchpadController::hid_input_end() {
	USBHDBGSerial.printf("UXP Touchpad hid_input_end\n");

However, hid_input_begin and hid_input_data never get called. hid_input_end gets called after every single "HID: 1 - 74 47 .. .. .. .. " packet.



3) Is it a "problem" that all the packets received from HID seem to be 64 bytes? The "60 02" packet is only two bytes. (In the screenshot above, packet #43 it states: Packet data length)
Also the multi-touch packets are supposed to be 58 bytes (the last 6 "00" are not part of the packet).



4) Following 3), The multi-touch packets are 58 bytes if one finger is used. Each additional finger increases the packet-length by 28 bytes. However instead of one longer packet I get - with four fingers - multiple HID packets like this:

HID: 1 - 74 9E 1C 03 96 8A 00 00 00 17 07 97 70 00 04 00 10 08 00 00 FB FF 00 00 00 00 00 00 06 04 02 01 E8 04 99 0D F7 FF 5F FF E5 03 9C 03 00 40 A5 00 EE 00 00 00 00 00 00 00 07 04 04 01 11 13 BB 0B 
UXP Touchpad hid_input_end
UXP in_callback
UXP in_callback TRANSFER->DRIVER = TRUE
HID: 1 - C2 F7 BE 01 71 05 52 03 02 6F 6C 00 56 00 00 00 00 00 00 00 08 04 03 01 0D 0B FC 0C D4 FF 89 FF D8 03 2B 02 F9 3D 3F 00 8F 00 00 00 00 00 00 00 09 07 00 00 A2 0F 52 0D 00 00 00 00 00 00 00 00 
UXP Touchpad hid_input_end
UXP in_callback
UXP in_callback TRANSFER->DRIVER = TRUE
HID: 1 - C5 39 00 00 00 00 00 00 00 00 00 00 FE 1C 04 00 10 08 00 00 FB FF 00 00 00 00 00 00 06 04 02 01 E8 04 99 0D F7 FF 5F FF E5 03 9C 03 00 40 A5 00 EE 00 00 00 00 00 00 00 07 04 04 01 11 13 BB 0B 
UXP Touchpad hid_input_end
UXP in_callback
UXP in_callback TRANSFER->DRIVER = TRUE

As seen, each one triggers the TouchpadController::hid_input_end and USBHIDParser::in_callback .
How/where can I increase the maximum packet size from 64?




Thanks a lot for your time in advance again! And also for the time last time, I would have not come so far otherwise!
So after all the post is huge anyways...

[KurtE]

A lot of these types of questions, can best be answered by the relevant documents.

For example, the GET_REPORT, SET_REPORT details on how they work and what messages you can send and expected responses.
Are best answered by the HID_v1.1.1.pdf document which you can download from the USB HID document page: https://www.usb.org/hid

Likewise, from that same web page, you can download the HID usage table document, which will show you details about the different IDs. Like what IDs
are associated with Usage Page 1 (Generic Desktop Page), which most of the Mouse and like IDs come from.
Keyboard come from page 6... and shows FF00-FFFF - as Vendor-defined

2) forwarding packets. If your class has the method: hid_process_in_data
And you return true from it, it says you processed the data, so the HID parsing code is not called, to parse the data and call functions on a field by field level.

Note some of classes choose to do their own parsing. For example, our Wacom Tablet code, does its own parsing. We decided to do that as the Linux driver more or less said the HID definitions were bogus.

3) Many devices will send back whole packets (64 bytes) regardless of how much they use of it. They will typically somehow encode how much they actually used.

4) increase the size of packets. You cannot. That was set by the device:
That is, you will see in some of your previous posts, things like:

** Interface Level **
09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01
02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
 bInterfaceNumber = 1
 number end points = 1
 bInterfaceClass =    3
 bInterfaceSubClass = 0
    HID
 bInterfaceProtocol = 0
report descriptor size = 27
  endpoint = 81
    attributes = 3 Interrupt
    size = 64
    interval = 2
HIDParser claim this=2000A5A0
 bInterfaceNumber =   1
 bInterfaceClass =    3
 bInterfaceSubClass = 0
 bInterfaceProtocol = 0
HID Parser Claim: 09 04 01 00 01 03 00 00 04 09 21 11 01 00 01 22 1B 00 07 05 81 03 40 00 02 09 04 02 00 01 03 01 02 04 09 21 11 01 00 01 22 34 00 07 05 84 03 08 00 08 
report descriptor size = 27
Single endpoint HID:
  endpoint = 81
   size = 64
   interval = 2
new_Pipe

And likewise in one of your first posts:

Touchpad (Internal Keyboard/Trackpad (ISO))

Device Descriptor:
----------------------------------------------
This device is part of a composite device.

Connection Information
 Port: 2
 Speed: Full Speed
 Device address: 6
 Open pipes: 3
 Connection status: Device connected

Full speed (12 MBS) USB 2 - has a max of 64 bytes per packet. This is for example how all of the Teensy 3.x work, where if it said High Speed (480MBS) then the packet size can go up to 512.

There is a difference then on logical packets where lets say: 3 fingers, so 58+28+28 = 114 won't fit in 64. So, they will encode that somehow.
Not sure of what approach they may use or not.
They for example could in this case decide to send 3 different HID reports or the like, in three packets. They could decide to have the first several bytes of a "primary" instruction have some information, like total size of data, and then the receiving side would need to build the complete packet.
...

For example, in our newer bluetooth code, there are at least two different ways this is done. In that case one way is each physical packet started with a header and one byte in it, was if the certain bit is set, it is a primary message, if a different bit is set, it is a continuation.
A second way is the data is contained in known structures: Where the first structure, has a total length of the physical packet, and then goes to the contained structure(L2CAP) which has a length of its data. And if its length won't fit in the first size, we know that there will be more packets coming in to fill out that logical packet.

And some Bluetooth classes used both ways, depending on which message.

Hope this helps some.

[Eric Damon Stahl]

Hope this helps some.

Yes it did, cheers mate!

2) forwarding packets. If your class has the method: hid_process_in_data
And you return true from it, it says you processed the data, so the HID parsing code is not called, to parse the data and call functions on a field by field level.

Note some of classes choose to do their own parsing. For example, our Wacom Tablet code, does its own parsing. We decided to do that as the Linux driver more or less said the HID definitions were bogus.

So to be honest, I had/have no idea how to make the parse function ( void USBHIDParser:arse(uint16_t type_and_report_id, const uint8_t *data, uint32_t len) ) work, so I completely commented it's content out.
Instead I do my checking and driver calling directly in the in_data ( void USBHIDParser::in_data(const Transfer_t *transfer) ).

Checking for the 60 02 packet: if (buf[0] == 0x60 && buf[1] == 0x02 && buf[57] == 0x00), then sending my control packet.

Then checking for multi-touch packets: if (buf[2] == 0x1C && buf[3] == 0x03 && buf[11] == 0x97)
Those bytes are always exactly the same so I can use them to easily identify a (first) multi-touch packet.

4) There is a difference then on logical packets where lets say: 3 fingers, so 58+28+28 = 114 won't fit in 64. So, they will encode that somehow.
Not sure of what approach they may use or not.
They for example could in this case decide to send 3 different HID reports or the like, in three packets. They could decide to have the first several bytes of a "primary" instruction have some information, like total size of data, and then the receiving side would need to build the complete packet.

Well, the "method" is that... it just appends the bytes of the following finger(s).
So for 1 finger I need: Packet 1: bytes 30-58
2 fingers: Packet 1: bytes 30-58 (finger 1) + bytes 59-64 (beginning finger 2) , Packet 2: bytes 1-22 (rest finger 2)
etc... 5 fingers will use almost 3 full packets.

I created a 'touchpoint' struct that contains a timestamp and the data from the packets (xAbs, yAbs, xRel, yRel etc) and send them to my driver, one "finger" at a time. Works all fine, it's awesome so far!



But there is one major issue I encounter...

In my touchpad sketch I have the following in the main loop:

if (touchpad1.available()) 
{
switch (touchpad1.getGesture()) 
{
case 1: 
Serial.print("Touchpad 1 finger move\n");
Mouse.move(touchpad1.getMove1fX() / 5, touchpad1.getMove1fY() / 7);
break;
case 2: 
Serial.print("Touchpad 2 finger move\n");
//if (!isTouchActive) { isTouchActive = true; }
//touchScrollDelta += touchpad1.getMove2fY();
//TouchscreenUSB.press(0, 16000, 16000 + touchScrollDelta);
break;
case 3: 
Serial.print("Touchpad 1 finger tap\n");
Mouse.set_buttons(true, 0, 0);
break;
case 4: 
Serial.print("Touchpad 2 finger tap\n");
Mouse.set_buttons(0, 0, true);
break;
case 0: 
break;
}
touchpad1.touchpadDataClear();
}

if (!touchpad1.isCleared() && (millis() - touchpad1.lastPacket() > 10))
{
Serial.print("touchpad1.isCleared() and > 10: ");
Serial.print((millis() - touchpad1.lastPacket()));
Serial.println();
touchpad1.lastFingerUp();

if (isTouchActive)
{
TouchscreenUSB.release(0);
isTouchActive = false;
touchScrollDelta = 0;
}
}

The switch handles action depending on what my driver set my gesture to.
The if-statement checks if more than 10ms have passed since the last packet arrived at my driver and - if true - assumes that no more fingers are on the touchpad. It then resets everything (gesture, values etc) and starts fresh with the next finger_down.

So, if I tap on the touchpad with 1 finger, the Teensy sends a left mouse click.
Tapping with 2 fingers sends a right mouse click.
Moving around with 1 finger moves the mouse cursor.
Moving with 2 fingers does nothing but printing the "2 finger move".

It all works completely fine and as intended. I can tap and move with 1, 2, 3, 4, 5 fingers (3-5 fingers have nothing implemented so far, but it easily scales and works) all day long and everything works.
The moment I uncomment the 3 lines I would expect a touch input. (I'm using Windows 11 on my main machine, Windows 10 on a secondary test machine and an Android 11 tablet)

On Android I get my touch input and scrolling up and down with 2 fingers moves the touch on the y-axis. But after releasing my fingers it is "stuck" in the 2-finger-move-gesture. Tapping with 1 or 2 fingers does nothing and moving with 1 finger does nothing either. I can continue to use 2 finger scrolling up and down to control the touch input, but that's it.

On Windows... well.. first of all TouchscreenUSB.press does absolutely nothing on Windows 11 or Windows 10. In the device manager I can see the Teensy show up as a "HID-compliant touch screen", but no touch is registered.
Furthermore, immediately after the first TouchscreenUSB.press fires my serial monitor becomes unresponsives, sluggish and skips/breaks output.

For example it should look like this (and this is super quick, poll rate for the touchpad is 2ms):

BCM5974: Wellspring packet (1 finger) received
BCM5974: currentGesture = 0
UXP touchpadDataClear
BCM5974: Wellspring packet (1 finger) received
BCM5974: currentGesture = 0
Touchpad 1 finger move
UXP touchpadDataClear
BCM5974: Wellspring packet (1 finger) received
BCM5974: currentGesture = 1
Touchpad 1 finger move
UXP touchpadDataClear

But after TouchscreenUSB.press it's like this:

BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: gesture is now Gesture_Move2f
Touchpad 2 finger move
UXP touchpadDataClear
BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_Move 2f: -183
Touchpad 2 finger move
UXP touchpadDataClear
BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_Move 2f: -185
Touchpad 2 finger move
UXP touchpadDataClear
BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_Move 2f: -188
Touchpad 2 finger move
UXP touchpadDataClear
BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_Move 2f: -184
Touchpad 2 finger move
UXP touchpadDataClear
BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_Move 2f: -185
Touchpad 2 finger move
UXP touchpadDataClear
BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_Move 2f: -179
Touchpad 2 finger move
UXP touchpadDataClear
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received
BCM5974: checkGestureMove2f activeFingers = 2
BCM5974: Gesture_M

So the first few milliseconds it's fine. Detects my 2 finger gesture, spits out my relative y numbers (those prints are from my driver), then it's becoming weird...
The input in the serial monitor becomes super slow.
"BCM5974: Gesture_MBCM5974: Wellspring packet (2 fingers) received" is from my 'void USBHIDParser::in_data(const Transfer_t *transfer)' where I do my parsing.
"BCM5974: checkGestureMove2f activeFingers = 2" is from the function below. The first print still makes it, but not further. Keep in mind that this works totally fine if I comment out the TouchscreenUSB.press.

void TouchpadController::checkGestureMove2f(int number)
{
	USBHDBGSerial.printf("BCM5974: checkGestureMove2f activeFingers = %i\n", bcm.tp[1].fingers);
	if (bcm.tp[1].fingers == 2)
	{
		if ((currentGesture == Gesture_None) && //|| currentGesture == Gesture_Move1f) &&
			abs(bcm.tp[1].xAbs0 - bcm.tp[1].xAbs) < 500 &&
			abs(bcm.tp[2].xAbs0 - bcm.tp[2].xAbs) < 500 &&
			abs(bcm.tp[1].yAbs0 - bcm.tp[1].yAbs) > bcm.moveDelta &&
			abs(bcm.tp[2].yAbs0 - bcm.tp[2].yAbs) > bcm.moveDelta)
		{
			USBHDBGSerial.printf("BCM5974: gesture is now Gesture_Move2f\n");
			currentGesture = Gesture_Move2f;
		}
		else if (currentGesture == Gesture_Move2f)
		{
			USBHDBGSerial.printf("BCM5974: Gesture_Move 2f: %i\n", (bcm.tp[1].yRel + bcm.tp[2].yRel));
			move2fY1 = bcm.tp[1].yRel;
			move2fY2 = bcm.tp[2].yRel;
		}
	}
}

After a few more milliseconds it stops responding all together. The keyboard driver still works at this point. I can type and it changes the LED's when pressing capslock for example. But for the keyboard there is no more output in the serial monitor either.
Also it seems to soft-brick the Teensy. In that state, when trying to upload code to it it says "error sending reboot command (HID). Teensy did not respond to a USB-based request to enter program mode.
Please press the PROGRAM MODE BUTTON on your Teensy to upload your sketch." and I have to either unplug and plug it back in, or press the button.

I have no idea why that is, and like I stated:
No TouchscreenUSB.press -> everything works
TouchscreenUSB.press -> broken as hell

Any idea what I might be doing wrong? (Also why TouchscreenUSB.press seems to not do anything on Windows 11 / 10 at all? Even when using a new/empty sketch. I found the 10-finger multi touch sketch in the forum somewhere and it does nothing in Windows, but works on Android)


Edit:
So just for my sanity, I double checked to replace the TouchscreenUSB.press with 'Mouse.scroll(1)' and 'Keyboard.press(KEY_DOWN)' + 'Keyboard.release(KEY_DOWN)' and that works just fine (but, obviously, is super janky and not neeeearly as smooth as I would want it with the touch...

[Eric Damon Stahl]

Let me attach my sketch. It's the modified mouse.ino
The five functions on the bottom are not actually empty, I just wanted to show that they are in the sketch (and unmodified from the mouse.ino).

Whenever my driver reports a 1/2/3 finger tap or move, I now put the led on the teensy on, and when the gesture is over I turn it off. On the tap I can't actually see it as it's only on for about 10ms, but for 1/2/3 finger movements (1 finger can go in any direction, 2/3 finger move needs to go vertically only) the led is on as soon as the gesture is recognized and turned off when I remove my fingers. Works perfectly.

When I remove the comment on 'TouchscreenUSB' in my OnMove2f function, the sketch breaks, but "less" than I thought earlier. Serial monitor outputs are super slow and inomplete/cut off. Also my gestures (move/tap) won't work as intended anymore. But the underlaying code (my touchpad.cpp, hid.cpp etc) still works fine. The led still turns on/off instantly when my gestures are recognized / fingers are removed.

The behaviour, as soon as 2 finger move WITH TouchscreenUSB is active the first time is as follow:

1. Touchpoint is pressed down in the center of the screen and moved up and down along the y-axis
2. I can lift the fingers and end the gesture
3. a) Using 2 finger move again works and executes everything from 1. again.
3. b) Using another gesture (1/2/3 finger tap or 1/3 finger move) does nothing - for now.
3. c) Using 2 finger NOW executes everything that should have happened in b) and 'stops' the current 2 finger move
4. b) Now I can do a) or b) again

So basically, all gestures always work totally fine without TouchscreenUSB usage.
With it, all gestures work totally fine until the one with TouchscreenUSB is executed and breaks the sketch until I restart the Teensy.

#include "USBHost_t36.h"

int ledPin = 13;

int touchScrollDelta;
bool isTouchActive;
bool tp_isCleared;

USBHost myusb;
USBHub hub1(myusb);
KeyboardController keyboard1(myusb);
USBHIDParser hid1(myusb);
TouchpadController touchpad1(myusb);

USBDriver *drivers[] = {&hub1, &keyboard1};
#define CNT_DEVICES (sizeof(drivers)/sizeof(drivers[0]))
const char * driver_names[CNT_DEVICES] = {"Hub1", "KB1"};
bool driver_active[CNT_DEVICES] = {false, false};

USBHIDInput *hiddrivers[] = {&touchpad1};
#define CNT_HIDDEVICES (sizeof(hiddrivers)/sizeof(hiddrivers[0]))
const char * hid_driver_names[CNT_DEVICES] = {"Touchpad1"};
bool hid_driver_active[CNT_DEVICES] = {false};
bool show_changed_only = false;

void setup()
{
  pinMode(ledPin, OUTPUT);
  //while (!Serial) ; // wait for Arduino Serial Monitor
  Serial.println("\n\nUSB Host Testing");
  Serial.println(sizeof(USBHub), DEC);
  myusb.begin();
  keyboard1.forceBootProtocol();
  //keyboard1.attachPress(OnPress);
  //keyboard1.attachRelease(OnRelease);
  touchpad1.attachMouseLeftButton(OnMouseLeftButton);
  touchpad1.attachMove1f(OnMove1f);
  touchpad1.attachMove2f(OnMove2f);
  touchpad1.attachMove3f(OnMove3f);
  touchpad1.attachTap1f(OnTap1f);
  touchpad1.attachTap2f(OnTap2f);
  touchpad1.attachTap3f(OnTap3f);
  keyboard1.attachExtrasPress(OnHIDExtrasPress);
  keyboard1.attachExtrasRelease(OnHIDExtrasRelease);
  Mouse.screenSize(1920, 1080);
}

void loop()
{
  myusb.Task();

  for (uint8_t i = 0; i < CNT_DEVICES; i++) {
    if (*drivers[i] != driver_active[i]) {
      if (driver_active[i]) {
        Serial.printf("*** Device %s - disconnected ***\n", driver_names[i]);
        driver_active[i] = false;
      } else {
        Serial.printf("*** Device %s %x:%x - connected ***\n", driver_names[i], drivers[i]->idVendor(), drivers[i]->idProduct());
        driver_active[i] = true;

        const uint8_t *psz = drivers[i]->manufacturer();
        if (psz && *psz) Serial.printf("  manufacturer: %s\n", psz);
        psz = drivers[i]->product();
        if (psz && *psz) Serial.printf("  product: %s\n", psz);
        psz = drivers[i]->serialNumber();
        if (psz && *psz) Serial.printf("  Serial: %s\n", psz);
      }
    }
  }

  for (uint8_t i = 0; i < CNT_HIDDEVICES; i++) {
    if (*hiddrivers[i] != hid_driver_active[i]) {
      if (hid_driver_active[i]) {
        Serial.printf("*** HID Device %s - disconnected ***\n", hid_driver_names[i]);
        hid_driver_active[i] = false;
      } else {
        Serial.printf("*** HID Device %s %x:%x - connected ***\n", hid_driver_names[i], hiddrivers[i]->idVendor(), hiddrivers[i]->idProduct());
        hid_driver_active[i] = true;

        const uint8_t *psz = hiddrivers[i]->manufacturer();
        if (psz && *psz) Serial.printf("  manufacturer: %s\n", psz);
        psz = hiddrivers[i]->product();
        if (psz && *psz) Serial.printf("  product: %s\n", psz);
        psz = hiddrivers[i]->serialNumber();
        if (psz && *psz) Serial.printf("  Serial: %s\n", psz);
      }
    }
  }

if (!touchpad1.isCleared() && (millis() - touchpad1.lastPacket() > 10))
{
  Serial.print("touchpad1.isCleared() and > 10: ");
  Serial.print((millis() - touchpad1.lastPacket()));
  Serial.println();
  touchpad1.lastFingerUp();
  digitalWrite(ledPin, LOW);

  if (isTouchActive)
  {
    Serial.print("release touchscreenusb 0");
    Serial.println();
  TouchscreenUSB.release(0);
  TouchscreenUSB.end();
  isTouchActive = false;
  }
}
}

void OnTap1f(bool pressed)
{
  digitalWrite(ledPin, HIGH);
  Serial.print("one finger tap ");
  if (pressed) { Serial.print("down"); }
  else { Serial.print("up"); }
  Serial.println();
  Mouse.set_buttons(pressed, 0, 0);
}

void OnTap2f(bool pressed)
{
  digitalWrite(ledPin, HIGH);
  Serial.print("two finger tap ");
  if (pressed) { Serial.print("down"); }
  else { Serial.print("up"); }
  Serial.println();
  Mouse.set_buttons(0, 0, pressed);
}

void OnTap3f(bool pressed)
{
  digitalWrite(ledPin, HIGH);
  Serial.print("3 finger tap ");
  if (pressed) { Serial.print("down"); }
  else { Serial.print("up"); }
  Serial.println();
}

void OnMouseLeftButton(bool pressed)
{
  Serial.print("mouse_left_button ");
  if (pressed) { Serial.print("down"); }
  else { Serial.print("up"); }
  Serial.println();
  Mouse.set_buttons(pressed, 0, 0);
}

void OnMove1f(int xRel, int yRel)
{
  digitalWrite(ledPin, HIGH);
  Serial.print("move 1 finger: ");
  Serial.print(xRel);
  Serial.print(", ");
  Serial.print(yRel);
  Serial.println();
  Mouse.move(xRel / 5, yRel / 7);
}

void OnMove2f(int yRel)
{
  digitalWrite(ledPin, HIGH);
   Serial.print("move 2 fingers: ");
   Serial.print(yRel);
   Serial.println();

 // if (yRel > 0)
//  {
//  //Mouse.scroll(1);
//  }
//  else if (yRel < 0)
//  {
//    // Keyboard.press(KEY_DOWN);
//    // Keyboard.release(KEY_DOWN);
//      //Mouse.scroll(-1);
//  }

  if (!isTouchActive)
  {
    isTouchActive = true;
    touchScrollDelta = 0;
    TouchscreenUSB.begin();
    //TouchscreenUSB.press(0, 16000, 16000);
  }
  else
  {  
    touchScrollDelta += yRel;
    //TouchscreenUSB.press(0, 16000, 16000 + touchScrollDelta);
  }
}

void OnMove3f(int yRel)
{
  digitalWrite(ledPin, HIGH);
   Serial.print("move 3 fingers: ");
   Serial.print(yRel);
   Serial.println();
  if (yRel > 0)
  {
    Keyboard.press(KEY_MEDIA_VOLUME_INC);
    Keyboard.release(KEY_MEDIA_VOLUME_INC);
  }
  else
  {
    Keyboard.press(KEY_MEDIA_VOLUME_DEC);
    Keyboard.release(KEY_MEDIA_VOLUME_DEC);
  }
}

void OnPress(int key) {}

void OnRelease(int key) {}

void OnHIDExtrasPress(uint32_t top, uint16_t key) {}

void OnHIDExtrasRelease(uint32_t top, uint16_t key)  {}

bool OnReceiveHidData(uint32_t usage, const uint8_t *data, uint32_t len) {}

I tried putting different TouchscreenUSB commands at different locations.
TouchscreenUSB.begin(); in the setup() or in my 2 finger move function or not at all
Using TouchscreenUSB.end(); after each .press or when all fingers are off the touchpad or not at all
Using TouchscreenUSB.release(i) anywhere or not at all
Using TouchscreenUSB.press(i) with and without pressure (.press(0, X, Y, 0-255)
Using TouchscreenUSB.press(i) with 0/1 pressure instead of .release and/or before .release

Any combination that came to my mind...
Also worth nothing (I guess), that TouchscreenUSB.begin .end .release do NOT break the sketch, only .press.