Teensy 4.1 Mouse.moveTo Mac Drag problem

masonb

Member
Hi all,

I have been running into a problem with the Mouse.moveTo when using this in conjunction with a mac.
I have set up a Teensy to work as a remote mouse when a device gets plugged in. When i plug in a windows machine i can use a separate touch screen device with input card to view and control the plugged in device from a single hdmi and usb cable, this works as expect and allows me to send mouse press and release as well was moveTo to click drag and select. However when a mac device is plugged in, if i press and drag, the fist point of contact releases the mouse left button with out mouse.release being invoked. After futher inspection i can see its the mouse.moveTo causing this problem when used with a mac. I need to use moveTo as i need absolute positioning onf the mouse to match up with the input visable on the touch screen device. Is this a known issue and if so is there a fix or work around?

Thanks
 
Last edited:
Sorry, I am not much of a MAC person. But it might help if you posted an example sketch that reproduces the issue.
Then potentially someone could try it out and/or have ideas on things for you to try.

Things like:
Do you set the buttons state, like when you click do you call something like: Mouse.set_buttons(1);
does it help or hurt?

Have you tried the the USB type that includes Touch Screen? Does this work better for your case?

Sorry again I personally do very little with MAC and have not played much with USB type of mouse, other than to verify it works and also on the USBHost side to handle the inputs...

Good luck
 
Thanks for your reply.

I tested using the Mouse.set_buttons, but i was using Mouse.press. Both played out the same

The project works on the principle that i send web commands to the teensy and on the teensy side the command is broken down to two parts, button state and mouse location.
I then have two methods, the mouse state command is used in the first method to set the state and the second command is used in the second method to set the location using Mouse.moveTo.

This currently works perfectly on windows to allow me to move the mouse, click and drag/highlight. However when used on mac, the moveTo seems to be triggering a click on the move causing the the drag to be broken by the press and release of the click. I ran a small test where i used Mouse.set_buttons(1,0,0) followed by moveTo(x,y). On windows this preformed a highlight on the desktop and on mac this just moved the mouse. I have tested the Mouse.set_buttons(1,0,0) and this held the mouse down allowing me to move the mouse with my trackpad to prove that the teensy can hold the mouse down and preform a highlight. This also showed me that the problem lies in the moveTo.

Methods

Code:
bool SendMousePosition(char* args) 
{
	// Args validity check
	for (byte i = 0; i < strlen(args); i++)
	{
		if (!isDigit(args[i]) && args[i] != ',')
		{
			Debug.LogLine("ERROR: Invalid mouse position");
			SendErrorResponse("ERROR: Invalid mouse position");
			return false;
		}
	}

	// Get X and Y
	char* save = NULL;
	int x = atoi(strtok_r(args, ",", &save));
	int y = atoi(strtok_r(NULL, ",", &save));

	// Check X Coordinate
	if (x < 0)
	{
		Debug.LogLine("ERROR: Invalid Mouse X");
		SendErrorResponse("Invalid Mouse X");
		return false;
	}

	// Check Y Coordinate
	if (y < 0)
	{
		Debug.LogLine("ERROR: Invalid Mouse Y");
		SendErrorResponse("Invalid Mouse Y");
		return false;
	}

	Debug.Log("Sending Mouse Position: ");
	Debug.Log(x);
	Debug.Log(",");
	Debug.LogLine(y);

	// Set mouse position
  Mouse.moveTo(x, y);
  
	return true;
}

bool SendMouseButton(char* args)
{
	char* save = NULL;

	// Get button and state
	int button = atoi(strtok_r(args, ",", &save));
	int state = atoi(strtok_r(NULL, ",", &save));

	Debug.Log("Sending Mouse Button: ");
	Debug.Log(button);
	Debug.Log(" & State: ");
	Debug.LogLine(state);

	// Check button is in range
	if (button < MOUSE_LEFT || button > MOUSE_MIDDLE)
	{
		Debug.LogLine("ERROR: Mouse button invalid");
		SendErrorResponse("Mouse button invalid");
		return false;
	}

	// Check state is in range
	if (state < 0 || state > 2)
	{
		Debug.LogLine("ERROR: Mouse state invalid");
		SendErrorResponse("Mouse state invalid");
		return false;
	}

	// Which state do we need?
	switch (state) 
	{
		case 0: // Release button
			Mouse.release(button);
			break;
		case 1: // Press button
			Mouse.press(button);
			break;
		case 2: // Click button
			Mouse.click(button);
			break;
	}

	return true;
}
 
Last edited:
In my research it looks like it may be an issue with the descriptors causing mac to lose its click when MoveTo is used one after another. We had this working in teensyduino version 1.18 before the Descriptor changes. I am interested in finding the HID descriptors from this version but am unable to locate them as this version is now very outdated. Would anyone be able to provide a copy of the descriptors and potentially the moveto class from this version?
 
We had this working in teensyduino version 1.18 before the Descriptor changes. I am interested in finding the HID descriptors from this version but am unable to locate them as this version is now very outdated. Would anyone be able to provide a copy of the descriptors and potentially the moveto class from this version?

Indeed 1.18 is from before I started really using github. At that time, Teensy 3.1 was the latest hardware. Latest Arduino IDE was version 1.0.5.

I found a copy of the 1.18 source in my old files archive. Attached to this message are the USB descriptor files.


I then have two methods, the mouse state command is used in the first method to set the state and the second command is used in the second method to set the location using Mouse.moveTo.

If this doesn't help, can I talk you into expanding this to a small program anyone with MacOS can copy into Arduino and upload to their board to reproduce the problem? Even if the setup() and loop() functions are "trivial", having it as a complete program which can be uploaded without any guesswork can really help.
 

Attachments

  • usb_desc.c
    43.6 KB · Views: 12
  • usb_desc.h
    11.8 KB · Views: 14
If this doesn't help, can I talk you into expanding this to a small program anyone with MacOS can copy into Arduino and upload to their board to reproduce the problem? Even if the setup() and loop() functions are "trivial", having it as a complete program which can be uploaded without any guesswork can really help.

I can put together a small test code sample with part of my code for others to test
 
Hi Paul,

This appears to still not be working on Mac, I have put together a small example of my code using the classes from my project with an adapted loop to not require an additional application and just run through a looped example. Below is the code.

Code:
#define BUFFER_SIZE   64
void setup() {

}

void loop() {
  char* buffer;

  buffer = (char*)"ss=2560,1600";
  ProcessQueryString(buffer);
  delay(1000);
  buffer = (char*)"mp=50,50";
  ProcessQueryString(buffer);
  delay(1000);  
  buffer = (char*)"mb=1,1";
  ProcessQueryString(buffer);
  delay(1000);
  buffer = (char*)"mp=400,400";
  ProcessQueryString(buffer);
  delay(1000);  
  buffer = (char*)"mb=1,0";
  ProcessQueryString(buffer);
  delay(5000);
}

bool SendMouseButton(char* args)
{
  char* save = NULL;

  // Get button and state
  int button = atoi(strtok_r(args, ",", &save));
  int state = atoi(strtok_r(NULL, ",", &save));

  // Check button is in range
  if (button < MOUSE_LEFT || button > MOUSE_MIDDLE)
  {
    return false;
  }

  // Check state is in range
  if (state < 0 || state > 2)
  {
    return false;
  }

  // Which state do we need?
  switch (state) 
  {
    case 0: // Release button
      Mouse.release(button);
      break;
    case 1: // Press button
      Mouse.press(button);
      break;
    case 2: // Click button
      Mouse.click(button);
      break;
  }

  return true;
}


/// <summary>
/// Send cursor position to the connected USB device.
/// </summary>
/// <param name="args">C-String value of 'mp=' query containg coordinates (X,Y) to move to.</param>
/// <returns>Success or failure.</returns>
bool SendMousePosition(char* args) 
{
  // Args validity check
  for (byte i = 0; i < strlen(args); i++)
  {
    if (!isDigit(args[i]) && args[i] != ',')
    {
      return false;
    }
  }

  // Get X and Y
  char* save = NULL;
  int x = atoi(strtok_r(args, ",", &save));
  int y = atoi(strtok_r(NULL, ",", &save));

  // Check X Coordinate
  if (x < 0)
  {
    return false;
  }

  // Check Y Coordinate
  if (y < 0)
  {
    return false;
  }

  // Set mouse position
  Mouse.moveTo(x, y);

  return true;
}
bool SetScreenSize(char* args)
{
  // Value validity check
  for (byte i = 0; i < strlen(args); i++)
  {
    if (!isDigit(args[i]) && args[i] != ',')
    {
      return false;
    }
  }

  // Get width and height
  char* save = NULL;
  int width = atoi(strtok_r(args, ",", &save));
  int height = atoi(strtok_r(NULL, ",", &save));

  // Check width
  if (width < 0)
  {
    return false;
  }

  // Check Y Coordinate
  if (height < 0)
  {
    return false;
  }

  // Set screen size
  Mouse.screenSize(width, height, false);

  return true;
}
/// <summary>
/// Process the existing query string.
/// </summary>
void ProcessQueryString(char * querystring) 
{   
  // Split queries by delimiter (&)
  char* qSave = NULL;
  char* query = strtok_r(querystring, "&", &qSave);

  while (query != NULL)
  {
    // Get query field name and value
    char* pSave = NULL;
    char* field = strtok_r(query, "=", &pSave);
    char* value = strtok_r(NULL, "=", &pSave);

    if (strcmp(field, "mb") == 0) {
      // Mouse button
      if (!SendMouseButton(value))
        return;
    }
    else if (strcmp(field, "mp") == 0) {
      // Mouse position
      if (!SendMousePosition(value))
        return;
    }
    else if (strcmp(field, "ss") == 0) {
      // Screen Size
      if (!SetScreenSize(value))
        return;
    }
    else
    {
      return;
    }

    // Get next query
    query = strtok_r(NULL, "&", &qSave);
  }
}
 
Back
Top