TouchscreenUSB.release(finger), Touch freezes, bug with Linux??

epikao

Active member
Hello,
on teensy 3.2 there are two USB HID Touchscreen Emulator functions:
TouchscreenUSB.press(finger, x, y);
TouchscreenUSB.release(finger);

see also:
https://forum.pjrc.com/threads/32331-USB-HID-Touchscreen-support-needed

Unfortunately that works for me only on Windows, but I need it for the Raspberry, Raspbian Jessie (Linux).
On Raspbian Jessie (Linux) as long no Release happen the touch works - but as soon as I release the touch freezes...

Is there something on the Descriptor/Teensy Code I could adapt to make it work on Linux?
I don't need it for Windows...

thank you for any help or advise.
 
You must post a complete program. Even if it's trivial, this is an absolute requirement. I will not investigate this problem, I won't even put it on my list of possible issues, until a complete program is posted.

Please double check that the program you post really does reproduce the problem. Yeah, obvious as that sounds, often we get cases like this were someone quickly writes something that "should" work, but then doesn't for some reason.

Also be specific about the PC side setup. So far you've only said "Linux", not even which distro, not to mention which version. If you want this problem investigated and fixed, the best way to get that to happen is posting detailed info so I can reproduce the issue.
 
Also, if you are not using 1.41-beta3, please do so *BEFORE* you post the program. Any testing and any changes will be applied with the latest code, so please make sure you are not running an older version when it comes to reporting a problem.
 
Also, if you are not using 1.41-beta3, please do so *BEFORE* you post the program. Any testing and any changes will be applied with the latest code, so please make sure you are not running an older version when it comes to reporting a problem.

oh :) now I am very happy to get a feedback. I use Arduino 1.8.2 Software / Teensyduino 1.36
My Code/your USB Touchscreen Functions works properly with Windows 10. With Ubuntu 17.04 on a x86, 32bit System the touch/mouse symbol freezes after the first release.
With current Raspian Jessie (with Desktop) on a Raspberry Pi 3 the Touch freeze as soon as I release but interesting: If I start the browser chromium and a javascript drawing program (for multitouch) touches freezes only after several releases (so not after the first).


And please find below my Code. With that Code I can read every Focaltech Controller per I2C and transfer the X, Y Data to your USB HID Touchscreen Functions.
So it's a simple I2C Touchscreen Interface to USB HID Touchscreen Interface Converter:

//Code deleted, please see more simple code next reply
 
Last edited:
If the problem is present with 1.41-beta3, I will also need a link to the website with this javascript, and specific info about where to get the "Focaltech Controller per I2C" hardware.

Please try to imagine if you had to read this message and needed to duplicate the test. Would you know which hardware to buy? What would you do for the javascript code?

Can you understand how leaving out these details means I can not recreate the test here?

If you want this problem properly investigated, please try to imagine any other details, any other specific info or steps needed to recreate the problem. Remember, I can not see your screen. I do not know anything about this javascript program. I can not know what you actually do with it. Try to imagine someone who knows the USB code but nothing about what you are actually doing, will try to fill in missing info and very likely do something different (perhaps not reproducing the problem). Do not leave these details to chance. Be specific. Include all info needed for anyone not familiar with your project to precisely follow the same steps and be sure to recreate the problem.

I do not wish to be difficult about this, but you really must understand we get these sorts of incomplete problem reports regularly. When details are missing, I almost never manage to reproduce the problem (even after wasting quite a lot of time). I know it requires extra effort on your part to provide this info, but it really is in your interest. I can only fix problems that I'm able to reproduce.
 
Please also understand your program's requirement for a specific piece of hardware means I will need to buy that hardware. This is (probably) not a problem. I do have a small budget for buying hardware to test bug reports. I only need you to show me exactly which hardware to buy. Again, please be specific... recently someone asked about a I2C LCD and gave the wrong display in their link, causing me to buy a different model than they were actually using. Don't do that!

But buying the correct Focaltech Controller will require time for the hardware to arrive. If it is a display ordered from a Chinese vendor, I will have to wait for the package to arrive by international mail.

In your other message, you wrote "It's URGENT". One thing you can do to expedite this process is to write a different program that reproduces the problem using only simple hardware, like a pushbutton connected to a digital pin or a pot connected to an analog pin. Or use of any of the hardware PJRC sells would be quick, since that hardware is already here.

Likewise, if there is a simpler way to recreate the lockup (not using specific javascript), the simpler and faster I can recreate the problem, the sooner I can work on a solution.
 
so now I have updated to 1.8.5, 1.41-beta3.

Still the same, and you don't need a Hardware / Touchscreen to test it, with follow code it's the same (just Touch Emulating):

const int LED = 13;
float mtmax = 32767.0;
float screenW = 2560.0;
float screenH = 1440.0;
int i=0;

void setup() {
// put your setup code here, to run once:
digitalWrite(LED,LOW);
TouchscreenUSB.begin();


}

void loop() {

for (i=0; i<40; i++) {
//Mouse.move(2, -1);
TouchscreenUSB.press(0, mtmax/2 + mtmax/screenW * i * 2, mtmax/2 + mtmax/screenH * i * -1);
TouchscreenUSB.press(1, 1000 + mtmax/2 + mtmax/screenW * i * 2, -500 + mtmax/2 + mtmax/screenH * i * -1);
delay(25);
}
TouchscreenUSB.release(0);
//TouchscreenUSB.release(1);

// put your main code here, to run repeatedly:
delay(3000);

}

on windows 10 it works perfectly, on linux ubuntu 17.04 first touch than it freezes
You can also test it with http://www.repaa.net/draw.html

I use teensy 3.2 (I have bought 15pcs of them for the raspberry)

Additional: with Windows IOT 10 on Raspberry it also works perfect - so it has something to do with linux!
Maybe Linux need only Touchscreen data (without Keyboard, Mouse, Serial or something... I don't know??)
 
Last edited:
Ok, I will look into this, using the code from message #8.

Ok. hope you will find a solution because I think with that I can make every capacitive Touchscreen working on every Platform without the need of build kernel and adapt driver etc.
Please let me know if you need more infos or if I can test something more?

thanks.
 
I'm running it now on a Teensy 3.6 and watching the USB communication with a protocol analyzer.

I can see the Teensy keeps communicating. But for some reason, Linux stops moving the cursor after the first run. Windows keeps responding by moving the cursor. I do not know why the 2 systems respond differently.
 
I'm running it now on a Teensy 3.6 and watching the USB communication with a protocol analyzer.

I can see the Teensy keeps communicating. But for some reason, Linux stops moving the cursor after the first run. Windows keeps responding by moving the cursor. I do not know why the 2 systems respond differently.


hmm thankyou, in May 2016 in the other USB Touch Thread Member "Ivartaim" reports follow:

Hi!
I have been playing around with the following code using the new functions. Using a teensy 3.0 and samsung galaxy s6 with android 6
I have one possible bug report and two possible suggestions. it could be that there is something wrong in my code also.

1) BUG
The release function doesn't release the finger properly in my case.

After calling "TouchscreenUSB.release(1);"

I cant call TouchscreenUSB.press again before i have restarted my phone. Even the disconnection of the usb doesn't help.

2) SUGGESTION
Instead of keyboard+touchscreen I would like to have touchscreen+serial. Because when teensy identifies itself as a keybord, then there is no way to use the androids touchscreens keyboard. And serial would be useful for printing out info.

3) SUGGESTION
Have some kind of aspect ratio input for coordinate remapping in the API. Of course this can be done before sending in commands to the API, but maybe through the API it would be cleaner?



Is there maybe a way to only select touchscreen? without keyboard, without serial and without mouse?

Elmue use another touch code which is working with all systems according codeproject page, but unfortunately only single-touch:
https://www.codeproject.com/Articles/1001891/A-USB-HID-Keyboard-Mouse-Touchscreen-emulator-with

Anyway - maybe he can help?
 
Last edited:
Hello Paul

epikao asks if I can help with this.
Any advances ?

Could you post the analyzer files ?
(One from Linux and one from Windows would be perfect)

I suppose you use DataCenter, so it would be the *.tdc files.
I also have a Beagle 480 now.

As you are MAC user: Does it work on MAC ?
 
any update? Please note: Your Hardware looks the only one with easly configurable Multitouch in the market. I think that is a very advantage and multitouch are more and more requested...! So it would be really appreciate if that will work on linux.... let me know if you need other support? I can also provide you a USB touchscreen which are working with all OS, so you can copy the descriptor?
 
Hello epikao

It is very tipical that Paul appears and then you don't hear from him anymore.
It seems that this problem is not easy to solve and he is very busy and currently working on other projects.

Most probably what you discovered is one of the plenty bugs in Linux.
May be there is no quick solution.

You are asking here for free support for your problem.
But this is work and costs time and if it is not payed, the motivation will not be high to help you.

So me too.
I'm also working on another big project now that I will first finish.
But maybe some day I come back to work on the touchscreen stuff.

You say you have a touchscreen that is working on all OS.
What does that mean: All OS ?

Which one did you test ?
- Windows XP
- Windows 7
- Windows 10
- MAC OS
- Linux (which ditributions)
- Android
- Raspberry

You could post the descriptor here.
If you don't know how to get it:
Install USBlyzer (the trial version works for a month, if I remember well)
Connect the touchscreen to Windows and export the descriptor (right click into the descriptor window)

However I think the quickest solution for you would be to solve the problem on your own.
You said that my Multitouch descriptor on Codeproject is working well for you and the only problem is that you cannot use two finger at once.

Try to duplicate the

USAGE (Finger)
COLLECTION (Logical)
.....
END_COLLECTION

which represents one finger so you have two fingers.

Then change

USAGE(Contact Count Maximum)
LOGICAL_MAXIMUM (1)

from 1 into 2

Then modify the function
usb_mouse_send_packet()
to send 13 instead of 7 bytes.

This is very easy and may be it already does the trick.

I started the same way as you:
I asked Paul to implement a touchscreen into TeensyDuino a few years ago.
He did not do it and so I had to learn USB on my own.
It took me some weeks but at the end I wrote the article on Codeproject.
Then, years later, he implemented the touchscreen that you are using now, but it does not work correctly.


Read this excellent manual ENTIRELY and you will learn how USB works:
http://www.usbmadesimple.co.uk

And use USBlyzer which does not only show the descriptors but also the data packets that are transmitted.

IMPORTANT:
If you modify the descriptor you MUST also modify the ID of the USB device:
PRODUCT_ID_SKMT in usb_desc.h
because Windows stores the descriptors in the registry and does not read them each time anew.
Increment the 0x048B each time you modify the descriptor !
 
Last edited:
I still have this on my issue list, but it got pushed to a lower priority by the work to wrap up the 1.41 release. Since then, it's also been displaced by investigating & fixing several other minor issues, like a couple long-standing but minor bugs in MotionCal (for the sensors on the Prop Shield - and apparently Adafruit now sells products with those sensors and sends their customers to get MotionCal...).

Yesterday I also spent most of my dev cycles on an urgent repair and update to some of the test fixtures we use here. It takes a lot of unseen work to keep PJRC going and Teensy continuously available. I do try to keep up with the forum and emails and investigating & fixing bugs, but there's only so many hours in every day. I have to prioritize, and often that means some issues get neglected.

Part of the difficulty with debugging this particular problem is I don't really have good ways to test, other than watching the on-screen results of manipulating ordinary GUI programs.

Does anyone have a suggestion for software I can run on the PC side. The ideal would be some sort of touch diagnostic which shows (perhaps as X,Y,pressure) what the OS is really seeing for each of the 10 fingers.
 
I have a Dell P2314T monitor on my desk right now. I'm watching how it communicates with my USB protocol analyzer. I'm also re-reading Microsoft's documentation on the expected usages. Also reviewing their docs on multi-touch reporting modes. This Dell display is using parallel mode. So far, I'm doing serial mode for Teensy.

My best guess at this moment is I need to send one extra report with the "Tip Switch" usage (0x42) zeroed. Here's a screenshot of some of the Dell's USB packets during a test where I touched briefly in the middle of the screen.

sc.png

I hope to have a first attempt at a solution soon. I must admit, I don't have a really great way to test this. Going to need feedback....
 
Hmmm... my use of the Contact Count usage (0x54) might also be incorrect for multi-touch serial mode. I'm going to try taking that out for now.
 
Ok, here's new code to try, pretty much a complete rewrite of TouchscreenUSB. I'm pretty sure this will fix the release(finger) problem. Not sure if it really fixes everything, as I don't have a great way to test. But it does seem to be working much better and seems to fully solve the pointer click getting "stuck".

These 4 files go into hardware/teensy/avr/cores/teensy3. I've also committed this to github, so you can get it there if that's more convenient.

Please let me know how this works for you?



EDIT: something went wrong with these filenames. They should be:

usb_desc.c
usb_desc.h
usb_dev.c
usb_touch.c

Please make sure you have the filename correct before copying to hardware/teensy/avr/cores/teensy3. Or get the files from github:

https://github.com/PaulStoffregen/cores/tree/master/teensy3
 

Attachments

  • usb_de.c
    74.3 KB · Views: 1,279
  • usb_desc.h
    33.8 KB · Views: 180
  • usb_dev.c
    30.2 KB · Views: 119
  • usb_to.c
    4.3 KB · Views: 123
Hello Paul

I use USBlyzer on the Windows side which shows the data packets decoded.
But DataCenter shows you the same.
So I think you will not find a better way to see what is sent by the Teensy.

The link from epikao may be helpfull for testing:
http://www.repaa.net/draw.html
Did you try it ?

I saw the MSDN links in the comment of the Multitouch descriptor in usb_desc.c.
(By the way: An applause for your work in usb_desc.c in the last years! You have added a lot of USB stuff there)

It seems that you have constructed the multi touch descriptor based on the MSDN ?
But it seems also that you have never tested it on Linux ?
Did you test on MAC ?

My experience as a Windows programmer is that there is a lot of wrong and uncomplete information in the MSDN.
So don't trust 100% what you read there to be always correct.
Especially the examples in the MSDN are mostly COMPLTELY wrong.
It seems that Microsoft lets the student apprentices write their examples.

You implemented for example
Usage (Scan Time)
I doubt that it is really neccessary although the MSDN says that it is.
At least my descriptor works without that.

And you did not implement
Usage (In Range)
although the MSDN says that it is necessary.


But at the end the problem is that Linux freezes.

My Multitouch descriptor on Codeproject works on Linux.
So I would start the investigation what is in your descriptor what Linux does not like ?
May be 10 fingers is too much for Linux ?
Who needs that ? I would set the default as
#define MULTITOUCH_FINGERS 2
Did you try if it also freezes with 2 fingers in the descriptor ?

I currently don't have time to test this.
But I would start with a working descriptor:
https://www.codeproject.com/Articles/1001891/A-USB-HID-Keyboard-Mouse-Touchscreen-emulator-with
See chapter Touchscreen (version 'Multi Touch')
I tested this on Windows XP, 7, 8, 10 and on Linux (Suse, Ubuntu, Knoppix) where it works seamlessly.

It is a multi touch descriptor but defines only one finger.
So it must be modified to use multiple fingers.

After each modification in the descriptor you must check if Linux starts making problems.
And obviously if it still works on Windows and MAC.

So my recommendation is to use a proved and working descriptor (mine on Codeproject) and modify it to use multiple fingers.
And finally: Testing is all !
At least on Windows 7 and Windows 10 and on Linux and MAC.
(If I'm not wrong Windows XP does not support multitouch.)

And at the end epikao can test it on Raspberry.
___________________________________________

And another point:
You let the touch screen send coordinates in the range from zero to
Logical Maximum (32767)

I think that this is not clever.
The Teensy users will have to make complicated calculations to get these coordinates.

In my project I use coordinates from 0 to 10000.
This allows to easily use a coordinate system from 0.00% to 100.00% independent from the screen size !!.
You simply calculate the finger position in percent of the screen size as a double and multiply the value with 100.
 
I mostly tried to emulate what this Dell monitor actually does. It doesn't use In Range, but it does implement Scan Time.
 
So my recommendation is to use a proved and working descriptor (mine on Codeproject) and modify it to use multiple fingers.

That site is responding *very* slowly, and when it finally does anything, I get "You must logon to download zip files."

If you've never even tried more than 1 finger, how do you know your descriptors would work?

It seems that you have constructed the multi touch descriptor based on the MSDN ?
But it seems also that you have never tested it on Linux ?

I am indeed testing with Linux. Maybe you're assuming I don't test Linux because I read the only available documentation? Please point me to any solid documentation for Linux or Mac and I will use it.

I currently don't have time to test this.

This is not the sort of feedback I'm interested to hear.
 
The link from epikao may be helpfull for testing:
http://www.repaa.net/draw.html
Did you try it ?

Yes.

sc.jpg

Code:
#include <Bounce.h>

Bounce mybutton = Bounce(14, 10);
int yoffset = 4000;

void setup() {
  pinMode(14, INPUT_PULLUP);
  digitalWrite(13, LOW);
  TouchscreenUSB.begin();
}

void drawline(int x, int y) {
 for (int i=0; i < 6000; i += 100) {
   TouchscreenUSB.press(0, x + i, y + i/13);
   delay(10);
 }
 TouchscreenUSB.release(0);
}

void loop() {
  mybutton.update();
  if (mybutton.fallingEdge()) {
    Serial.println("press");
    drawline(16000, yoffset);
    yoffset += 1200;
    if (yoffset > 24000) yoffset = 4000;
  }
}
 
Last edited:
10 simultaneous touch drawing confirmed, on Linux.

sc.jpg

Code:
#include <Bounce.h>

Bounce mybutton = Bounce(14, 10);
int yoffset = 6000;

void setup() {
  pinMode(14, INPUT_PULLUP);
  digitalWrite(13, LOW);
  TouchscreenUSB.begin();
}

void drawcircle(int x, int y) {
  float arc = 2.0 * PI / 10.0;
  float r = 3000.0;
  for (float angle=0; angle < arc; angle += 0.01) {
    for (int i=0; i < 10; i++) {
      float ph = arc * (float)i;
      TouchscreenUSB.press(i, r * cosf(angle+ph) + x, r * sinf(angle+ph) + y);
    }
    delay(10);
  }
  for (int i=0; i < 10; i++) {
    TouchscreenUSB.release(i);
  }
}

void loop() {
  mybutton.update();
  if (mybutton.fallingEdge()) {
    Serial.println("press");
    drawcircle(16000, yoffset);
    yoffset += 4200;
    if (yoffset > 24000) yoffset = 4000;
  }
}
 
Back
Top