Goal of USB Host Mouse/Keyboard example code

illera88

Member
Hi,

I've been checking the example code for USBHost_t36 called Mouse and I'm wondering why are all those devices declared:

Code:
USBHost myusb;
USBHub hub1(myusb);
USBHub hub2(myusb);
KeyboardController keyboard1(myusb);
KeyboardController keyboard2(myusb);
USBHIDParser hid1(myusb);
USBHIDParser hid2(myusb);
USBHIDParser hid3(myusb);
USBHIDParser hid4(myusb);
USBHIDParser hid5(myusb);
MouseController mouse1(myusb);
JoystickController joystick1(myusb);
//BluetoothController bluet(myusb, true, "0000");   // Version does pairing to device
BluetoothController bluet(myusb);   // version assumes it already was paired
int user_axis[64];
uint32_t buttons_prev = 0;
RawHIDController rawhid1(myusb);
RawHIDController rawhid2(myusb, 0xffc90004);

USBDriver *drivers[] = {&hub1, &hub2,&keyboard1, &keyboard2, &joystick1, &bluet, &hid1, &hid2, &hid3, &hid4, &hid5};
#define CNT_DEVICES (sizeof(drivers)/sizeof(drivers[0]))
const char * driver_names[CNT_DEVICES] = {"Hub1","Hub2", "KB1", "KB2", "JOY1D", "Bluet", "HID1" , "HID2", "HID3", "HID4", "HID5"};
bool driver_active[CNT_DEVICES] = {false, false, false, false};

// Lets also look at HID Input devices
USBHIDInput *hiddrivers[] = {&mouse1, &joystick1, &rawhid1, &rawhid2};
#define CNT_HIDDEVICES (sizeof(hiddrivers)/sizeof(hiddrivers[0]))
const char * hid_driver_names[CNT_DEVICES] = {"Mouse1", "Joystick1", "RawHid1", "RawHid2"};
bool hid_driver_active[CNT_DEVICES] = {false, false};

Why 2 KeyboardControllers if there is just 1 USB Female port and just 1 keyboard can be connected? Is it to make the USB Host act as a Hub of USB ports?

In my scenario I just want to detect if a Keyboard (both a wired and a Bluetooth keyboard) has been connected and do something with the typed keystrokes and do nothing and let the USB packets go through if the device is anything else (mouse, storage, joystick, ...). Can I just declare 1 single KeyboardController in this case? How detect that a keyboard has been inserted in the usb host? Should I use the "product()" method? And what should I do to just let through the USB data that is not a keyboard?

Thank you in advance :cool:
 
I'm wondering why are all those devices declared:

Each line creates a device driver instance which may be used when you connect a USB device.

Also an issue is the growing complexity of the example code. We really should have simpler examples and more advanced examples. As libraries develop is the examples tend to become sort of test benches for all the ways more complex hardware can be supported. Complexity grows. Eventually simpler examples get written again. USBHost_t36 is at a stage of development where a lot of functionality has been added, and it's now in need of simpler example and more comprehensive documentation with a "getting started, then more complex stuff" structure.


Why 2 KeyboardControllers if there is just 1 USB Female port and just 1 keyboard can be connected? Is it to make the USB Host act as a Hub of USB ports?

Yes, exactly, the 2 USBHub lines allow for supporting 2 hubs. Usually 2 or more hub lines are used, because many larger USB hubs are actually a network of 4 port hubs. When you plug that type of hub into a normal computer, Windows Device Manager, MacOS System Report or Linux "lsusb" will also detect multiple smaller hubs.


In my scenario I just want to detect if a Keyboard (both a wired and a Bluetooth keyboard) has been connected and do something with the typed keystrokes and do nothing and let the USB packets go through if the device is anything else (mouse, storage, joystick, ...). Can I just declare 1 single KeyboardController in this case?

For a "normal" wired USB keyboard connected directly to Teensy's USB host port, usually just 1 KeyboardController instance is enough.
 
First of all, thank you for your response. Things are more clear now.

What I still don't know is:
- Since I want to support any kind of keyboard, I guess I should create at least a KeyboardController and a BluetoothController (for bluethooth keyboards). Is there any other device driver I should declare to handle any kind of keyboard keystroke?
- In the example I don't see any "attachPress()" callback setup for on BluetoothController so I guess no keystroke will be captured if a keyboard Bluetooth dongle is connected to the teensy USB host. Am I right?
- How can I detect if a device is a keyboard or some other type of device?
- How can I do to pass through any data that doesn't come from a keyboard to the teensy USB device port? I know that I can use "Keyboard.Print()" to send the captured keystrokes to the USB device port. What if I just want to blindly pass the info to the USB device end?

Thank you for your time.
 
Hi PaulStoffregen,

sorry to bother you again, do you mind taking a look at my previous questions?

Thank you in advance.
 
Sorry I am not Paul, but ...

As mentioned by Paul, several of our examples were setup as big test cases to try out several different things, and the one named Mouse is/was the kitchen sink, especially for testing anything related to HID (Human Interface Device) type devices, like mice, joysticks, keyboards, etc..

Over time we have tried to add some more specific, so about 10 months ago I added the KeyboardForward sketch. Which is a simple I receive data from a wired keyboard and send most of the data to the host. In this one the only USBHost type devices I have included are:

Code:
USBHost myusb;
USBHub hub1(myusb);
KeyboardController keyboard1(myusb);
USBHIDParser hid1(myusb);
USBHIDParser hid2(myusb);

Why each one:
Hub - The keyboard might be connected through a hub. Either explicitly plugging into external hub and hub plugged into Teensy. Or some keyboards have built-in hub, so you can then plug other USB devices into the keyboard.

Keyboard controller - obviously because we are trying to process a keyboard.

USBHIDParser - Needed at least one if you are going to try to process some of the keys on most keyboards. More in a minute... I put in two as some of these devices have multiple things, so in case one gets eaten by something else...

But understand this sketch does not handle Bluetooth devices. There is another example under USBHost under Bluetooth called KeyboardBT.ino. Which still could use some code cleanup, as
it was setup to only handle one keyboard, but still has commented out other devices and code that should just be removed. This one can work for either wired or BT keyboards. Assuming our bluetooth code handles the specific keyboard.

But for now lets stick with the Forward example as I believe that is closest to your questions. Warning I am using USB terminology sort of loosely. And this not cover all of the other complications, like some keyboards don't by default produce standard boot keyboard data...

When you plug in a keyboard (to the host or your PC), the device is typically just do one Interface, but most do at least 2 interfaces some more.

Lets assume the first one has it's Interface Descriptor is marked as a (HID, BOOT device, KEYBOARD), which are keyboard1, will see and claim that Interface, and process the data that comes from the device directly, and does not use a HID parser. This part of the code handles most all of the standard normal keys like (A-Z, 0-9, Shift, control, Alt, ...) keys

However many/most keyboards have some other keys on them, like multi-media keys, (volume up, down...) Power buttons, Maybe other keys, like to launch a calculator or Web app, ...
This is where many of these keyboards, define a second HID interface, which we may wish to claim. This is where the hid1 or hid2 object come into play. If they see a standard HID interface that they can claim, they claim it. Then they ask the device for their descriptor that describes the HID data they generate, and the HID parser... parses it and looks for the collections that they generate and then ask any objects which are derived from USBHIDInput, if they are interested in this information. The keyboard object is derived this and as such it looks for a couple of specific pages (SYS_CONTROL, CONSUMER_CONTROL) and claims them... All others will not be claimed by the keyboard.

Note: Some keyboards might define their data marked vendor specific page, and we will not understand or process it. Also some keyboards may define another Interface in the descriptor, that again is marked something else vendor specific, where for example on PC maybe you install some software for the Keyboard, which adds a device driver that process that data. Again we will not understand or claim these interfaces.

I know the above is not overly clear, but I mention this to describe what/why some of the code in example:
Code:
void setup()
{
  myusb.begin();
#ifdef SHOW_KEYBOARD_DATA
  while (!Serial) ; // wait for Arduino Serial Monitor
  Serial.println("\n\nUSB Host Testing");
  Serial.println(sizeof(USBHub), DEC);

  // Only needed to display...
  keyboard1.attachPress(OnPress);
#endif
  keyboard1.attachRawPress(OnRawPress);
  keyboard1.attachRawRelease(OnRawRelease);
  keyboard1.attachExtrasPress(OnHIDExtrasPress);
  keyboard1.attachExtrasRelease(OnHIDExtrasRelease);
}
attachPress - is probably what is mostly used if you simply want to process keyboard coming in. That is if you typed: Input Line<cr>
You will get the keystrokes I n p .... <cr> one per callback...

However if you need to know more details like: when the first: 'i' was pressed and released and the state of the shift key... then you may need to connect up to:
attachRawPress, attachRawRelease: Where these functions will be called with an 8 bit: keycode.

And in the example to forward to PC, I needed to process this raw data, and then needed to translate the data back into the Teensy USB type containing Keyboard interface data and calls.

The calls for: attachExtrasPress, attachExtrasRelease - are the functions to call when we receive data on one of the Second HID interface for one of the two pages we understand.

Note: I am not sure if we ever added in the translation code for this to figure out how to forward this data to the PC...

Bluetooth: if you are wanting to support you need to include a bluetooth controller object. Currently code is setup where you either startup normal or in paring mode. At some point this code should be reworked to be cleaner and have API to enter pairing mode... But not there yet and not sure if/when anyone will.

BluetoothController bluet(myusb, true, "0000"); // Version does pairing to device
//BluetoothController bluet(myusb); // version assumes it already was paired

The Bluetooth controller code works sort of like a HIDParser code in that when it receives a connection request, they look for a class that supports the BTHIDInput interface and sees if they are interested in the connection... The KeyboardController does support this. And if they Bluetooth dongle and the keyboard are properly paired and we respond with the appropriate responses and handle the type of bluetooth Interfaces that they speak it should work. But there are limitations. For example we have not added the code that works with the BLE interface.

As for questions: hopefully some of the above gives you some insights.

- Since I want to support any kind of keyboard, I guess I should create at least a KeyboardController and a BluetoothController (for bluethooth keyboards). Is there any other device driver I should declare to handle any kind of keyboard keystroke?
Covered most of this above.

Note: Not all wireless keyboards/mice are Bluetooth. For example some of the Microsoft or Logitech keyboards, they have their own special USB dongles, which communicate with their keyboard and/or Mouse. With these, they will simply tell USB that they will look to USB like a normal Mouse and/or keyboard.


- In the example I don't see any "attachPress()" callback setup for on BluetoothController so I guess no keystroke will be captured if a keyboard Bluetooth dongle is connected to the teensy USB host. Am I right?

Lots above, but simply put, if the Bluetooth code is working right, it should just use the functions mentioned above.,

- How can I detect if a device is a keyboard or some other type of device?
To fully understand you would need to read through the USB specifications and likewise the Bluetooth specifications.
Example when I talk about usage pages, they are defined: https://usb.org/sites/default/files/hut1_3_0.pdf and top level HID documentation page: (https://www.usb.org/hid)


That is what each of the USB device classes do with their claim method: like:
Code:
bool KeyboardController::claim(Device_t *dev, int type, const uint8_t *descriptors, uint32_t len)
{
	println("KeyboardController claim this=", (uint32_t)this, HEX);

	// only claim at interface level
	if (type != 1) return false;
	if (len < 9+9+7) return false;
	print_hexbytes(descriptors, len);

	uint32_t numendpoint = descriptors[4];
	if (numendpoint < 1) return false;
	if (descriptors[5] != 3) return false; // bInterfaceClass, 3 = HID
	if (descriptors[6] != 1) return false; // bInterfaceSubClass, 1 = Boot Device
	if (descriptors[7] != 1) return false; // bInterfaceProtocol, 1 = Keyboard
	if (descriptors[9] != 9) return false;
	if (descriptors[10] != 33) return false; // HID descriptor (ignored, Boot Protocol)
	if (descriptors[18] != 7) return false;
	if (descriptors[19] != 5) return false; // endpoint descriptor
	uint32_t endpoint = descriptors[20];
	println("ep = ", endpoint, HEX);
	if ((endpoint & 0xF0) != 0x80) return false; // must be IN direction
	endpoint &= 0x0F;
	if (endpoint == 0) return false;
	if (descriptors[21] != 3) return false; // must be interrupt type
	uint32_t size = descriptors[22] | (descriptors[23] << 8);
	println("packet size = ", size);
	if ((size < 8) || (size > 64)) {
		return false; // Keyboard Boot Protocol is 8 bytes, but maybe others have longer... 
	}
...


- How can I do to pass through any data that doesn't come from a keyboard to the teensy USB device port? I know that I can use "Keyboard.Print()" to send the captured keystrokes to the USB device port. What if I just want to blindly pass the info to the USB device end?

The example sketch I mention does forwarding of the simple Keyboard data. It is setup to receive a couple of the HID extra data, but I don't think it is forwarding those yet.

One of the biggest issues on forwarding, is:
When you build your sketch, you define the USB Type, which then generates the USB Descriptor that the computer you plug into sees. And then the computer sets up to send and receive data to each of these specific interfaces. So in order to forward data to the computer from something plugged in to the host port you need to have some interface defined to forward to and then you may need to massage the data to match that interface. Sorry I know clear as mud!

Likewise if you plug in a mouse into your Host port and there is no code in place (like a MouseController object) to claim that device, then that device will not do anything... (Maybe slight lie) as the Mouse probably talks HID and if you have a HIDParser object defined it will probably claim that interface, but each time it receives data from the MOUSE it will see there is no device registered with it that handles that type of data, and then it will toss it on the ground.

Good Luck... And I hope some of this makes sense and helps.
 
Wow!
Thank you so much for all the effort on responding this thorough.

Things makes way more sense now that I've read your post along with the "KeyboardForeward.ino" code. I think that I actually should base my code into that example and not in the "Mouse.ino".

Note: I am not sure if we ever added in the translation code for this to figure out how to forward this data to the PC...

You did:
Code:
void OnHIDExtrasPress(uint32_t top, uint16_t key)
{
#ifdef KEYBOARD_INTERFACE
  if (top == 0xc0000) {
    Keyboard.press(0XE400 | key);
#ifndef KEYMEDIA_INTERFACE
#error "KEYMEDIA_INTERFACE is Not defined"
#endif
  }
#endif
#ifdef SHOW_KEYBOARD_DATA
  ShowHIDExtrasPress(top, key);
#endif
}

void OnHIDExtrasRelease(uint32_t top, uint16_t key)
{
#ifdef KEYBOARD_INTERFACE
  if (top == 0xc0000) {
    Keyboard.release(0XE400 | key);
  }
#endif
#ifdef SHOW_KEYBOARD_DATA
  Serial.print("HID (");
  Serial.print(top, HEX);
  Serial.print(") key release:");
  Serial.println(key, HEX);
#endif
}

Declaration of USBHub and USBHIDParser makes sense now after the explanation and I can understand why adding even one extra of each for weird keyboards.

Thank you again, I can't stress enough how much I appreciate your response.
 
Sorry I am not Paul, but ...

As mentioned by Paul, several of our examples were setup as big test cases to try out several different things, and the one named Mouse is/was the kitchen sink, especially for testing anything related to HID (Human Interface Device) type devices, like mice, joysticks, keyboards, etc..

Over time we have tried to add some more specific, so about 10 months ago I added the KeyboardForward sketch. Which is a simple I receive data from a wired keyboard and send most of the data to the host. In this one the only USBHost type devices I have included are:

Code:
USBHost myusb;
USBHub hub1(myusb);
KeyboardController keyboard1(myusb);
USBHIDParser hid1(myusb);
USBHIDParser hid2(myusb);

Why each one:
Hub - The keyboard might be connected through a hub. Either explicitly plugging into external hub and hub plugged into Teensy. Or some keyboards have built-in hub, so you can then plug other USB devices into the keyboard.

Keyboard controller - obviously because we are trying to process a keyboard.

USBHIDParser - Needed at least one if you are going to try to process some of the keys on most keyboards. More in a minute... I put in two as some of these devices have multiple things, so in case one gets eaten by something else...

But understand this sketch does not handle Bluetooth devices. There is another example under USBHost under Bluetooth called KeyboardBT.ino. Which still could use some code cleanup, as
it was setup to only handle one keyboard, but still has commented out other devices and code that should just be removed. This one can work for either wired or BT keyboards. Assuming our bluetooth code handles the specific keyboard.

But for now lets stick with the Forward example as I believe that is closest to your questions. Warning I am using USB terminology sort of loosely. And this not cover all of the other complications, like some keyboards don't by default produce standard boot keyboard data...

When you plug in a keyboard (to the host or your PC), the device is typically just do one Interface, but most do at least 2 interfaces some more.

Lets assume the first one has it's Interface Descriptor is marked as a (HID, BOOT device, KEYBOARD), which are keyboard1, will see and claim that Interface, and process the data that comes from the device directly, and does not use a HID parser. This part of the code handles most all of the standard normal keys like (A-Z, 0-9, Shift, control, Alt, ...) keys

However many/most keyboards have some other keys on them, like multi-media keys, (volume up, down...) Power buttons, Maybe other keys, like to launch a calculator or Web app, ...
This is where many of these keyboards, define a second HID interface, which we may wish to claim. This is where the hid1 or hid2 object come into play. If they see a standard HID interface that they can claim, they claim it. Then they ask the device for their descriptor that describes the HID data they generate, and the HID parser... parses it and looks for the collections that they generate and then ask any objects which are derived from USBHIDInput, if they are interested in this information. The keyboard object is derived this and as such it looks for a couple of specific pages (SYS_CONTROL, CONSUMER_CONTROL) and claims them... All others will not be claimed by the keyboard.

Note: Some keyboards might define their data marked vendor specific page, and we will not understand or process it. Also some keyboards may define another Interface in the descriptor, that again is marked something else vendor specific, where for example on PC maybe you install some software for the Keyboard, which adds a device driver that process that data. Again we will not understand or claim these interfaces.

I know the above is not overly clear, but I mention this to describe what/why some of the code in example:
Code:
void setup()
{
  myusb.begin();
#ifdef SHOW_KEYBOARD_DATA
  while (!Serial) ; // wait for Arduino Serial Monitor
  Serial.println("\n\nUSB Host Testing");
  Serial.println(sizeof(USBHub), DEC);

  // Only needed to display...
  keyboard1.attachPress(OnPress);
#endif
  keyboard1.attachRawPress(OnRawPress);
  keyboard1.attachRawRelease(OnRawRelease);
  keyboard1.attachExtrasPress(OnHIDExtrasPress);
  keyboard1.attachExtrasRelease(OnHIDExtrasRelease);
}
attachPress - is probably what is mostly used if you simply want to process keyboard coming in. That is if you typed: Input Line<cr>
You will get the keystrokes I n p .... <cr> one per callback...

However if you need to know more details like: when the first: 'i' was pressed and released and the state of the shift key... then you may need to connect up to:
attachRawPress, attachRawRelease: Where these functions will be called with an 8 bit: keycode.

And in the example to forward to PC, I needed to process this raw data, and then needed to translate the data back into the Teensy USB type containing Keyboard interface data and calls.

The calls for: attachExtrasPress, attachExtrasRelease - are the functions to call when we receive data on one of the Second HID interface for one of the two pages we understand.

Note: I am not sure if we ever added in the translation code for this to figure out how to forward this data to the PC...

Bluetooth: if you are wanting to support you need to include a bluetooth controller object. Currently code is setup where you either startup normal or in paring mode. At some point this code should be reworked to be cleaner and have API to enter pairing mode... But not there yet and not sure if/when anyone will.

BluetoothController bluet(myusb, true, "0000"); // Version does pairing to device
//BluetoothController bluet(myusb); // version assumes it already was paired

The Bluetooth controller code works sort of like a HIDParser code in that when it receives a connection request, they look for a class that supports the BTHIDInput interface and sees if they are interested in the connection... The KeyboardController does support this. And if they Bluetooth dongle and the keyboard are properly paired and we respond with the appropriate responses and handle the type of bluetooth Interfaces that they speak it should work. But there are limitations. For example we have not added the code that works with the BLE interface.

As for questions: hopefully some of the above gives you some insights.


Covered most of this above.

Note: Not all wireless keyboards/mice are Bluetooth. For example some of the Microsoft or Logitech keyboards, they have their own special USB dongles, which communicate with their keyboard and/or Mouse. With these, they will simply tell USB that they will look to USB like a normal Mouse and/or keyboard.




Lots above, but simply put, if the Bluetooth code is working right, it should just use the functions mentioned above.,


To fully understand you would need to read through the USB specifications and likewise the Bluetooth specifications.
Example when I talk about usage pages, they are defined: https://usb.org/sites/default/files/hut1_3_0.pdf and top level HID documentation page: (https://www.usb.org/hid)


That is what each of the USB device classes do with their claim method: like:
Code:
bool KeyboardController::claim(Device_t *dev, int type, const uint8_t *descriptors, uint32_t len)
{
	println("KeyboardController claim this=", (uint32_t)this, HEX);

	// only claim at interface level
	if (type != 1) return false;
	if (len < 9+9+7) return false;
	print_hexbytes(descriptors, len);

	uint32_t numendpoint = descriptors[4];
	if (numendpoint < 1) return false;
	if (descriptors[5] != 3) return false; // bInterfaceClass, 3 = HID
	if (descriptors[6] != 1) return false; // bInterfaceSubClass, 1 = Boot Device
	if (descriptors[7] != 1) return false; // bInterfaceProtocol, 1 = Keyboard
	if (descriptors[9] != 9) return false;
	if (descriptors[10] != 33) return false; // HID descriptor (ignored, Boot Protocol)
	if (descriptors[18] != 7) return false;
	if (descriptors[19] != 5) return false; // endpoint descriptor
	uint32_t endpoint = descriptors[20];
	println("ep = ", endpoint, HEX);
	if ((endpoint & 0xF0) != 0x80) return false; // must be IN direction
	endpoint &= 0x0F;
	if (endpoint == 0) return false;
	if (descriptors[21] != 3) return false; // must be interrupt type
	uint32_t size = descriptors[22] | (descriptors[23] << 8);
	println("packet size = ", size);
	if ((size < 8) || (size > 64)) {
		return false; // Keyboard Boot Protocol is 8 bytes, but maybe others have longer... 
	}
...




The example sketch I mention does forwarding of the simple Keyboard data. It is setup to receive a couple of the HID extra data, but I don't think it is forwarding those yet.

One of the biggest issues on forwarding, is:
When you build your sketch, you define the USB Type, which then generates the USB Descriptor that the computer you plug into sees. And then the computer sets up to send and receive data to each of these specific interfaces. So in order to forward data to the computer from something plugged in to the host port you need to have some interface defined to forward to and then you may need to massage the data to match that interface. Sorry I know clear as mud!

Likewise if you plug in a mouse into your Host port and there is no code in place (like a MouseController object) to claim that device, then that device will not do anything... (Maybe slight lie) as the Mouse probably talks HID and if you have a HIDParser object defined it will probably claim that interface, but each time it receives data from the MOUSE it will see there is no device registered with it that handles that type of data, and then it will toss it on the ground.

Good Luck... And I hope some of this makes sense and helps.

Could this go in the WiKi or be included with the example ino?
 
Back
Top