Touchscreen on Linux with Mouse.moveTo()

Status
Not open for further replies.
Yes, but they do not have relative movement like Mouse.move()

But the relative stuff could stay unchanged in the first mouse device.
Only the absolute movement goes to the additional touch screen device.
I don't see any conflict.

When I connect a real mouse and a real touch screen at the same time to the same computer they work both at the same time.

The Teensy user has to decide if he wants to control the mouse relatively with the mouse device or absolutely with the touch screen device.

did you try the absolute-only code in the old versions

Not yet.
 
I cannot.
The installer 1.18 has the "Next" button grayed out.
I can chose whatever folder but it is always disabled.
 
Download an old copy of Arduino. The versions it supports are shown on the intro screen, before the first time you click Next.
 
LOL, 1.18 was before the "?" Button was added to the installer..... back in the days when "Next button won't enable" was a FAQ. Nobody ever saw the versions supported info, I the installer or on the website, but the little "?" Button solves the problem!
 
Hello Paul

I'm software developer since 30 years and I never disable neither buttons nor menus in my software.
This is a severe design error.
Just for the reason that the user does not know WHY the F u c k is this button disabled ???

What I do in my software is letting all buttons and menus enabled always.
And when the user clicks a function that is currently not available he receives a message box with an explanation.
In this case the installer should tell him that he first has to chose the folder where Arduino 1.05 is installed before he can advance.
This saves a lot of headaches from the user and eliminates the support questions.


But if the TeensyDuino installer would be programmed more intelligently it would not EVEN ask the user where Arduino is installed.
Because TeensyDuino could simply read the installation folder from the Registry:
HKEY_LOCAL_MACHINE\SOFTWARE\Arduino\Install_Dir

The installer could check if it finds the Arduino.exe in this folder (to verify that it is still installed there)
and go on installing immediately without bothering the user for a folder location.

______________________________________________________________________


OK.
Now back to the problem.
I compiled with TeensyDuino 1.18.
The advantage of this code is that now I have a version that works on Linux and on Windows at the same time.
The second mouse device in Windows control panel with the yellow exclamation mark has disappeared and it works fine on Windows.
It also works on Linux on my testing machine. (but there it worked before already)

We tested also on an Ubunto machine where it works.

But on the machine that causes us the problems it stays the same: It does not work correctly.
Now I have more detailed information about the problem:

What happens is that this Linux (SUSE) does not transform the coordinates correctly.
I saw in your Teensy code that you send coordinates between 0 and 32767 via USB.
But this Linux does not convert these coordinates correctly into screen coordinates.
When Teensy sends the coordinate for the middle of the screen (something about 16384) this Linux has the mouse already at the bottom right corner of the screen. The difference increments linearly from the top left corner (where it is correct) to the bottom right corner (where it wants to go out of the screen).

This has been tested with screen resolutions 800x600 and 1024x768.

So this is another bug in Linux apart from the bug with the relative mouse device.
(Now I have only an absolute mouse device)

Once again I see that:
1.) Linux is full of bugs
2.) Linux behaves different on each machine and it is extremely difficult to reproduce a problem of one machine on another machine.

Currently we have to use the workaround that we multiply the X and Y coordinate with a factor to get the correct mouse position.
 
Last edited:
Sounds like you've got a workaround.

If you or anyone else finds a better solution, I'll almost certainly implementing it in future Teensyduino. But I really do not believe I can continue putting development time into this.
 
Yes.
Currently we use this workaround.

You don't have time for that and I currently don't have time for that, neither.
But maybe some day I will find some time to implement a touch screen into Teensy.

I saw your code in usb_mouse_position() and it is very simple.
You transform the coordinates into values between 0 and 32767 and send them in a little packet via USB.
This is nothing complicated.

Only the USB report would have to be changed.

Can you tell me which documents do I have to study which explain these cryptic HID numbers that you have sent in your previous post #2 ?

What else in the code do I have to change to convert a mouse device into a touch screen device ?

What I want is this:
[This forum's image upload is buggy. I cannot include an image into this post]
 
Oh, you also need the usage tables document, which is mentioned on page 36 and 59, and in the revision history on page viii. You can find it at www.usb.org.

I'm pretty sure you'll also be disappointed by that document. If you don't like these documents, maybe take that up with the USB-IF. They are the official USB HID specs. You asked for the documentation on the report descriptor format. Well, this is it.
 
Hi Paul

Today I received the worst news:
Our client, for whom my company is developing, changed anything in that Linux and now the Teensy mouse commands are completely ignored while keyboard still works.

This Linux machine has a USB touch screen from IBM connected which is working.
But the Teensy mouse device is dead now.
Not even clicks are working anymore.

And if we connect a real mouse it does not work neither.
It seems they block mouse support by purpose to force the usage of touchscreen ??
Or it is a side effect of a new touch screen driver with a bug ??

While before we could use the workaround with a multiplication factor, now we are really stuck.

So we need someone who programs a touch screen HID device to run in Teensy.

1.) Do you know someone who could do that for a payment ?
2.) Would you yourself do that for a payment ?
 
But the Teensy mouse device is dead now.
Not even clicks are working anymore.

And if we connect a real mouse it does not work neither.
It seems they block mouse support by purpose to force the usage of touchscreen ??

Yes, it does indeed seem to be a limitation in X11, where you can have pointer control by absolute or relative coordinates, but not both. It's not anything to do with Teensy... just a limitation in the design of X11.

That's why I "ignored" the stuff in reply #17. It's a touch screen with absolute coordinates, but not also containing relative coordinates.


While before we could use the workaround with a multiplication factor, now we are really stuck.

So we need someone who programs a touch screen HID device to run in Teensy.

Did you trying the 0018 absolute-only code with this Linux machine using the touch screen?

My guess is that may work. Maybe? If anything can, it's probably that.

1.) Do you know someone who could do that for a payment ?
2.) Would you yourself do that for a payment ?

I certainly can not get directly involved, even on a paid consulting basis. I have way too much to do on Teensy!!!

I do know a couple consultants who take on projects, but their expertise is more with PCB design and "normal" firmware. Still, you could ask if they're willing to take HID device stuff.

But before you do, I'd definitely try the 0018 code. If that doesn't work on the same machine that's using the touch screen, odds are slim any HID approach will.

As a last resort, you'd probably need to go to a non-HID (or custom, non-pointer HID) communication with a custom-written daemon that can inject X events. Or update the Linux machine to the bleeding edge libinput stuff which Koromix mentioned in reply #19.
 
Hello Paul

I wrote in #31 that I am already using the code of version 1.18.
And it worked before.

But now they changed anything on that Linux so that NOT EVEN a real mouse is working anymore.

So any change in the Teensy MOUSE device (with relative or absolute coordinates does not matter) will not solve anything.

Mouse is dead.
I need a real HID touch screen device.



> As a last resort, you'd probably ..... daemon that can inject X events.
> Or update the Linux machine to the bleeding edge libinput stuff which Koromix mentioned in reply #19.

I cannot modify this Linux.
We neither have root access to that machine nor permission to change anything on that Linux.



> That's why I "ignored" the stuff in reply #17. It's a touch screen with absolute coordinates, but not also containing relative coordinates.

Obviously touch screens always send absolute coordinates.

For that reason I wrote that the touch screen should be an ADDITIONAL device beside the relative mouse device.
Not a replacement.
I have the impression that you did not read what I wrote before.
 
Last edited:
The code in 1.18 is a touch screen (only abs coordinates), even though it's called "Mouse" in the software.

Maybe you could edit the HID info to make it a "real" touch screen, but I have no idea what that would be. It's already only abs x and y.
 
> The code in 1.18 is a touch screen


Now I understand that the entire discussion of 40 postings was completely in vain
because at the end you did not understand nothing of what I was talking about all the time !


How can you tell me that

0x05, 0x01, // Usage Page (Generic Desktop)
0x09, 0x02, // Usage (Mouse)

is the same as

0x05, 0x0D, // USAGE_PAGE (Digitizers)
0x09, 0x04, // USAGE (TouchScreen)

??

I thought you were the USB expert ?

Pens and Touch screens are an own category among the HID devices.
They have their own usage's like "Finger", "Stylus", "Tip Switch" etc.

Nothing to do with mice.
 
Last edited:
Wow! Thanks for answering, you are the only one ever to do so. I read the link you gave me, and (bear with me) I believe that your code requires the user to change some settings (like Control Panel, etc.). This will not work for my purpose because the Teensy (disguised as a mouse) is being plugged into an NVR security system, and so I can't access the operating system. So, I just re-read the link to the "Ugly Hack" I gave in my above post. I wrote that the hack, "has one user very happy with the results", well I see that is you! Since I haven't tried to program any computer since my Commodore VIC-20 back in the mid 80's, I'm woefully ignorant of all the advancements made since then. Would you perhaps be so kind in telling me how (or getting me started) to get the "Ugly Hack" working? Anything at all would be greatly appreciated!

Beavis
 
It seems you did not read my article.
I cannot explain otherwise where you found that my solution reuqires any change in control panel ????
Please read my article THOROUGHLY !!!

The only thing you do is program the Teensy, then you plug the USB cable into a computer running Windows, Linux, Mac, or even Android and it simulates a real mouse, keyboard or touchscreen.
The computer does NOT notice that this is not a "real" mouse, keyboard or touchscreen.
There is no configuration required.

Maybe you try it before discarding the idea and looking for other solutions.
The work is aready done.
Compile my code and you are ready.
You do NOT need access to the operating system.
I developed this code because I also need to control a system that I cannot access. (The company does not give me the admin password)
You don't need any "ugly hack".
 
Hello Paul,

I'm very interested by this mod as I try to use Mouse.MoveTo command on a linux computer. I have a Teensy 3.2 and I've tried to modify the usb_desc.c file as the text to tweek seems to be situated there.

I did compile my arduino sketch having modified the usb_des.c file but it doesn't seem to work. Is there something special I need to do in order for the mod to be taken into account ?

Sorry for my stupid questions.

Marc,
 
But since Teensy can't see your screen to know where the pointer is located, Mouse.move() is only useful in some limited situations. I had thought nobody ever used it, but that turned out to be a wrong assumption. Some people do indeed use Mouse.move() in real projects, especially to make custom controllers with certain games.

I'd actually go as far as saying, relative positioning with Mouse.move() is usually much more useful than absolute positioning for most applications.
If you aren't building a touchscreen (and most commercial screens already have a built-in controller) or writing yet another pointless device to hack PCs, you will most likely want to read and relay relative input from your sensors anyway. For instance, when using joysticks or buttons to control the mouse, the absolute positioning would not be helpful at all, since only the relative movement is known and there would not be any benefit using the absolute positioning in relative mode. In fact, if the configured screen resolution didn't match the target system, the relative emulation would fail and the cursor would move skewed.
Apart from that, almost all operating systems have a configurable mouse speed, so having the OS handle the mouse speed is another plus of relative positioning.
 
Last edited:
Status
Not open for further replies.
Back
Top