Advice needed on capturing mouse x,y values

Status
Not open for further replies.
R

rfresh737

Well-known member
I have the new Teensy 3.5 board. I looked at the mouse examples that came with the Teensy inside the Ardunio IDE, but did not see any examples of what I'm trying to do.

My project is going to simulate mouse clicking on the PC screen to operate a windows application. I can pretty much do that as I've worked on the code to make that part work.

However, I need to know the X,Y position of the upper left corner of the app window and the X,Y of it's lower right corner. What I thought I'd do in my sketch, is to use the Teensy mouse lib and move my PC mouse to the upper left corner and click and record the X,Y values and then do the same for the lower right corner. Then my sketch can calculate the correct offsets to operate the various buttons in the windows application.

I'm stuck at figuring out how I can use my sketch to receive the X,Y values as I move my PC mouse around on the screen. Are there any examples of this someone can point me to?

Thank you for any help...
 
A couple of things in your post echo some of my own efforts in the project I'm working on now. I hope I'm not out of line butting in on your post, with me being quite noobish etc.

It sounds like you want to use some apps as any user typically would (no mods or add ons), but the mouse controls are like, scripted, so once you do all the clicks, drags, selects etc on whatever buttons, boxes, menus, that Teensy could then do those things as an automated preset? Depending on how close my guess is, PYAUTOGUI could be quite useful:

https://pyautogui.readthedocs.io/en/latest/

if not in whole form, then at least to get you the x,y of the mouse. It seems like a great tool to automate keyboard & mouse control of a PC, but I have only used it to get mouse x,y without any problems. I just use like this:

x, y = pyautogui.position()

I tried other programs with this type of feature, but they were limited to their own window, or were too complicated to use full screen. This one covers the whole screen.

The other thing you mentioned was defining a window by clicking on two corners to define two coordinate sets as reference points. I tried this, but had some problems with consistency. On a WIN7 PC and a WIN8.1 PC it worked great. Accurate AND consistent. But the intended security DVR's were unusable. The pointer would not track linearly. Like it would be kinda close, but then in some areas it would start to drift off target and then even reverse direction or simply vanish. Also just getting the corners to plot was tricky because the window wouldn't fit exactly so the true 0,0 was just outside the border. Using points set in from the edges helped. Currently I still am using this method but with great results (98% to 99%) entirely usable. Not sure why the difference in results, except that I was using earlier versions of Arduino and Teensyduino (from 1 year back). I don't know what the problem was, but I don't care as NOW it's working great. I'm just finishing up with all the features like dragging and mouse wheel and a macro which will log on via onscreen keyboard.

I hope I've been at least a little bit helpful and haven't wasted your time. I'd be interested to hear about your solution. Good luck!
 
Status
Not open for further replies.
Back
Top