PDA

View Full Version : Assistive device- chord EMG input for virtual keyboard



PaulH
03-16-2013, 01:49 AM
Purpose (started 30 years ago!) was to design a device that would allow people with amputations, arthritis,etc. to use EMG as an input for a virtual keyboard.

Just over a year ago I learned about arduino and then discovered the teensy.

First step was to use a teensy++2 along with 5 buttons to generate the alphabet plus a few simple commands to be able to type a document. Pressing button #1 would type "a", pressing button #2 would type "b", pressing both would type "c", etc. With the help of several very generous and smart people I was able to make this work.

The next step was to find an affordable EMG sensor board and I found one at Advancertechnologies.

Now I am struggling to write a simple sketch to replace the digital buttons with the analog EMG sensors. To make things simple I am using 5 variable resistors to simulate the EMG sensors. I wrote a quick sketch to verify that this worked the same as the sensors.

Now to the struggle. I have no programing knowledge. Below is the start of a new sketch that I am trying to write. As you will quickly see I really need some help.

At this point:

char to_send = keymap[byte];


I am getting this error:

sketch_mar14anewkeyboardrevtidy1.ino: In function 'void loop()':
sketch_mar14anewkeyboardrevtidy1:44: error: expected primary-expression before ']' token

At the moment I am using serial print just to keep things simple.

Any help would be most appreciated.

Thank you,

Paul




int keymap[32] = {-1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21, 22,23,24,25,26,27,28,29,40,42,44,57,0};


int sv0, sv1, sv2, sv3, sv4;
byte key;

void setup() {
// initialize serial communication at 9600 bits per second:
Serial.begin(9600);
}

// the loop routine runs over and over again forever:
void loop() {

key = 0;
sv0 = analogRead(A0);
sv1 = analogRead(A1);
sv2 = analogRead(A2);
sv3 = analogRead(A3);
sv4 = analogRead(A4);

if (sv0 > 250)
{
key = key | B00000001;
}
if (sv1 > 250);
{
key = key | B00000010;
}
if (sv2 > 250);
{
key = key | B00000100;
}

if (sv3 > 250);
{
key = key | B00001000;
}
if (sv4 > 250);
{
key = key | B00010000;
}
{
char to_send = keymap[byte];
Serial.print(to_send);
}
}

MichaelMeissner
03-16-2013, 03:19 AM
Your main problem is the line:



char to_send = keymap[byte];


I think you meant:



char to_send = keymap[key];


Note, you want the binary constants to be of the form 0B0001 with the leading 0. I tend to prefer hex myself, as you have less digits, and it is easy to accidently add or delete a 0.

Give you are doing the same processing on each of the 5 inputs, I probably would have put the pin number into an array, and used shifting to create the mask. I also find it a good exercise to use pinMode in setup to properly setup each pin I am using to the appropriate value (INPUT or OUTPUT).

Here is how I would have written it:



int keymap[32] = {-1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21, 22,23,24,25,26,27,28,29,40,42,44,57,0};

const int num_pins = 5;
int input_pins[num_pins] = { A0, A1, A2, A3, A4, A5 };

void setup()
{
int i;

for (i = 0; i < num_pins; i++)
{
pinMode (input_pins[i], INPUT);
}

// initialize serial communication at 9600 bits per second:
Serial.begin(9600);
}

// the loop routine runs over and over again forever:
void loop()
{
int key = 0;
int i;
int value;
char to_send;

for (i = 0; i < num_pins; i++)
{
value = analogRead (input_pins[i]);
if (value > 250)
{
key |= (1 << i);
}
}

to_send = keymap[key];
Serial.print (to_send);
}


One issue is the code as above probably isn't what you want, since you probably want to only do the action if any of the pins change in value. Otherwise, it will do many thousands/millions of reads per second.

Nantonos
03-16-2013, 11:44 AM
And for anyone else who was unfamiliar with the term:

Electromyography (EMG) (http://en.wikipedia.org/wiki/Electromyography) is a technique for evaluating and recording the electrical activity produced by skeletal muscles (http://en.wikipedia.org/wiki/Skeletal_muscles).

PaulH
03-16-2013, 02:29 PM
Thank you for your help Michael.
I understand that the way that I am doing this may not be the best but for me it is the easiest way for me to visualize what is going on as I am completely ignorant about the various function statements that are available. I plugged in a "delay(500)" at the end to slow the output.
What I am trying to write is a simple code that when a threshold is over 250 on A0 then an"a" is generated, when the threshold is over 250 on A1 then an "b" will be generated. With binary I believe that I will be able to figure out how to get a "c" when the threshold is over 250 on A0 and A1.
I changed the code to keymap[key] but the characters generated are not what is in the binary.

I know I am trying to run before I walk with this program but I have managed in the past and I am pretty determined to plug my way through it.

Please feel free to comment.

Paul

PaulStoffregen
03-17-2013, 07:48 AM
What I am trying to write is a simple code that when a threshold is over 250 on A0 then an"a" is generated...

Perhaps this will involve remember the previous reading, so you can detect an increase? Maybe it could look something like this:



int previousA0 = 0;

void loop() {
int nowA0;

nowA0 = analogRead(A0);
if (nowA0 > 250 && previousA0 <= 250) {
Keyboard.print("a");
}
previousA0 = nowA0;
}

PaulH
03-17-2013, 05:07 PM
Thank you Paul, I will play around with that and see how it goes.

Take care!

Paul

PaulH
03-17-2013, 10:53 PM
The following code gave the "a" as a continual output. I changed to serial print in case I made a mess of things (which is quite often!).
I added the delay to slow things down.

Teensy++2
Is it possible to do the following?
If threshold is under 250 then do nothing.
If over 250 then Serial.print("a")
If new threshold is over 250 then do nothing
If new threshold is under 250 then do nothing

Here is the code I have so far.


void setup(){
Serial.begin(115200);

int previousA0 = 0;

}
void loop (){
int nowA0;
int previousA0;

nowA0 = analogRead(A0);
if (nowA0 > 250 && previousA0 <= 250) {
Serial.print("a");
}
previousA0 = nowA0;
delay(500);
}


Thank you,

Paul H

MichaelMeissner
03-17-2013, 11:05 PM
The following code gave the "a" as a continual output. I changed to serial print in case I made a mess of things (which is quite often!).
I added the delay to slow things down.

Teensy++2
Is it possible to do the following?
If threshold is under 250 then do nothing.
If over 250 then Serial.print("a")
If new threshold is over 250 then do nothing
If new threshold is under 250 then do nothing


The problem with your code is the variable 'previousA0' goes out of scope when the loop function ends. In this case, you want to remember the variable from one call of loop to the next. So you want it to be a static or global variable (if you have only one source file, there isn't a difference between the two):

For example:



static int previousA0 = 0;

void setup(){
Serial.begin(115200);
}

void loop (){
int nowA0;

nowA0 = analogRead(A0);
if (nowA0 > 250 && previousA0 <= 250) {
Serial.print("a");
}
previousA0 = nowA0;
delay(500);
}

:cool:

PaulH
03-18-2013, 12:00 AM
Brilliant!!!

That was what I was looking for. Thank you very much.
I managed to increase the inputs to 5 and to generate a different letter for each input. Wow, that was the most progress that I have made in a long time.

Now for the challenging part. I need to change the output from "Serial.print("a")" to something that will be able to select a character from the following character set:


int keymap[32] = {-1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21, 22,23,24,25,26,27,28,29,40,42,44,57,0};


I like the direction the program has been going on and I would like to continue down this path so that the output of a threshold will generate a character from the character set. This way I can start looking at how to have the thresholds "chorded" as I was attempting to do in the original sketch.

Again, thank you all so much!


Paul H

PaulStoffregen
03-18-2013, 08:24 AM
The challenging part of detecting chords is dealing with imprecise human timing.

For the moment, let's forget about the keymap and talk only about a simpler case you mentioned earlier: A0 alone outputs 'a', and A1 alone outputs 'b', but together they output 'c'.

The trouble is the odds of human motion causing A0 and A1 crossing the threshold at precisely the same instant is pretty much zero. One or the other will almost always occur first. So you can't simply look for the combination of both. You need to delay sending any output, to allow time for the other the also change.

This is probably a job for elapsedMillis, which is a special type of variable that automatically increments 1000 times per second. You can learn more about it here:

http://www.pjrc.com/teensy/td_timing.html

One approach might use 6 variables, like this:



int previousA0 = 0;
int previousA1 = 0;
int anyChange = 0;
int changeA0 = 0;
int changeA1 = 0;
elapsedMillis sinceChange = 0;


The first two are the same as before, so you can detect the change. The next 3 are flags that are either 0 or 1. The first becomes a 1 when any input has crossed the threshold, and the others when a specific input crosses the threshold. The last counts how much time has elapsed since "anyChange" became 1.

First, you'll take the Keyboard.print() out of the code which detects the changes. Instead, you'll just set the variables, and if this is the first change (when anyChange is zero), you'll also put the elapsed time at zero.



nowA0 = analogRead(A0);
if (nowA0 > 250 && previousA0 <= 250) {
changeA0 = 1;
if (anyChange == 0) {
anyChange = 1;
sinceChange = 0;
}
}
previousA0 = nowA0;


You'll need to duplicate this code for each input. Then you'll need to add code which looks at these variables and decides what to send. Obviously, it should do nothing until anyChange is set, and the elapsed time is long enough to have allowed for imprecise human motion.

It might look something like this:



if (anyChange == 1) {
if (sinceChange > 50) {
// after 50 milliseconds, decide what to send
if (changeA0 == 1 && changeA1 == 1) {
Keyboard.print('c');
} else
if (changeA0 == 1) {
Keyboard.print('a');
} else
if (changeA1 == 1) {
Keyboard.print('b');
}
// always clear all flags after sending
anyChange = 0;
changeA0 = 0;
changeA1 = 1;
}
}


The main point here is the chained if else if else if else section checks for one of the possible cases, giving priority to the chord first. If both aren't set, then each individual one is checked. As you expand this to more keys, that if-else chain will grown long. Don't worry, the compiler is very good at dealing with such things. Of course, no matter which output you send, when you've decided to send something, the flags all need to be reset back to zero so you can respond to the next input.

For this to work, you can't have a delay(500) anywhere. The elapsedMillis variable establishes your timing.

PaulH
03-18-2013, 11:21 AM
Thank you Paul,
When I get home from work I will start with two inputs and gradually increase to 5. I truly appreciate your explanation of the process as I know what needs to be done, just not how to do it. With your clear explanations I can begin to understand what the functions are doing.
I will remove the delay.

Take care and thanks again!

Paul

PaulH
04-06-2013, 11:59 PM
Well, after struggling to implement Paul's approach I went and got a tutor to help me out. My tutor took a slightly different approach so the next session I will ask him to help me understand and implement Paul's approach. In the mean time the following code works. I used variable resistors to simulate the EMG sensor boards. This will do for the first step of the code. Now I am waiting for the remainder of the EMG sensors to arrive so that I can complete the device. Trials will be done with cerebral palsy patients. This entire endeavor may turn out to be a dud but at least I will know once and for all if it would have actually worked. I do not have the means to switch to multiple character sets yet as I have to determine the number of inputs that the people using it are comfortable with.

Here is the code and feel free to comment and make suggestions.


#define maskC B00011111
#define unpress_time 200
int sv0, sv1, sv2, sv3, sv4;
int keymap[32] = {-1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21, 22,23,24,25,26,27,28,29,40,42,44,57,0};

//enter=40,backspace=42, tab=43, space=44, caps lock 57
void setup() {
Serial.begin(115200);
DDRC ^= maskC;
PORTC ^= maskC;
}

void loop() {
static byte keysdown = 0;
static byte prevFull = 0;
static byte prevFull_release = 0;

static unsigned long tlpress = 0;
static unsigned long tlunpress = 0;

sv0 = analogRead(A0);
sv1 = analogRead(A1);
sv2 = analogRead(A2);
sv3 = analogRead(A3);
sv4 = analogRead(A4);

byte full = 0;/*~(PINC) & maskC;*/

if (sv0 > 250)
{
full = full | B00000001;
}
if (sv1 > 250)
{
full = full | B00000010;

}
if (sv2 > 250)
{
full = full | B00000100;
}

if (sv3 > 250)
{
full = full | B00001000;

}
if (sv4 > 250)
{
full = full | B00010000;
}

if(full && !keysdown) {
keysdown = 1;
tlpress = micros();
tlunpress = 0;
}
else if(!full && keysdown) {
keysdown = 0;

//Serial.print(keymap[prevFull_release]);
Keyboard.set_key1(keymap[prevFull_release]);
Keyboard.send_now();
Keyboard.set_key1(0);
Keyboard.send_now();

tlunpress = 0;
prevFull = 0;
prevFull_release = 0;
}

if(keysdown) {
if(full < prevFull) {
tlunpress = millis();
if((millis() - tlunpress) > unpress_time) {
prevFull_release = prevFull;
}
prevFull = full;
}
else if(full > prevFull) {
prevFull = full;
prevFull_release = prevFull;
tlunpress = 0;
tlpress = millis();
}
else {
//full == prevFull
if(tlunpress != 0 && (millis() - tlunpress) > unpress_time) {
prevFull_release = prevFull;
tlunpress = 0;
}
}
}
delay(50);
}

PaulH
04-13-2013, 08:33 PM
Just a quick update. Figured out how to toggle different character sets as well as a mouse function. Sensors arrived so I have to turn my attention to putting together a suite of assessment tools to see what type of signals I can get from my volunteer. I have used the oscilloscope program in the "processor" and it works as a good visualization for trying out the EMG sensors for the first time. On the next session I will test the volunteer's coordination to see how many inputs will be practical. The fewer the inputs the greater number of character sets. I may have to reduce the inputs to four plus a toggle. This would reduce the number of characters in the set to 15 from 32 and would increase the amount of times the toggle would have to be used but I may not have any choice. On the positive side four inputs are enough for the mouse control including right a left button capabilities.

Paul

PaulH
04-20-2013, 01:07 PM
Sensors are in,board is finished, and code has been modified to include three character sets plus mouse control (not included for testing) and toggling through character sets. I am going to look at using conductive fabric sewn into a cut down glove rather than the sticky electrodes to make the testing a little bit more enjoyable. If testing goes well I may write the mouse control with variable speeds (higher EMG the faster the mouse moves).
Paul

PaulStoffregen
04-20-2013, 08:46 PM
Glad it's working out. :)

PaulH
04-20-2013, 09:05 PM
Thanks for the encouragement Paul, I am having a blast learning how to do this.

Quick question, is there a way to use something like a "mousemap" function that would work the same way the "keymap" funtion? I can see how I can modify an arduino mouse sketch designed for digital buttons but I would really like to see if I could use the same approach as I have with the keys.

Something like this (I know that it is wrong on so many levels but it visualizes what I would like to do):

int keymap3[6] = {-1,Mouse.move(-5,0,0),Mouse.move(5,0,0),0,Mouse.move(0,-5,0),Mouse.move(0,5,0),0};



Currently I have the code set up so that after toggling through the keymap it then goes to "mouseon". Here is the code that I have so far.


// three character sets of 16 plus mouse

#define unpress_time 200
int sv0, sv1, sv2, sv3, sv4;
int keymap1[16] = {-1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18};
int keymap2[16] = {-1,19,20,21,22,23,24,25,26,27,28,29,44,42,40,0};
int keymap3[16] = {-1,30,31,32,33,34,35,36,37,38,39,57,55,54,41,0};
//enter=40,backspace=42, tab=43, space=44, caps lock 57

volatile int * mapPointer = keymap1;

int mouseOn = 0;

void swap()
{
if (mapPointer == keymap1)
mapPointer = keymap2;
else if (mapPointer == keymap2)
mapPointer = keymap3;
else if (mapPointer == keymap3 && mouseOn == 0)
mouseOn = 1;
else if (mouseOn)
{
mouseOn = 0;
mapPointer = keymap1;
}

}

void setup() {
Serial.begin(115200);


attachInterrupt(5, swap, RISING);

}
void loop() {
static byte keysdown = 0;
static byte prevFull = 0;
static byte prevFull_release = 0;
static unsigned long tlpress = 0;
static unsigned long tlunpress = 0;
sv0 = analogRead(A0);
sv1 = analogRead(A1);
sv2 = analogRead(A2);
sv3 = analogRead(A3);
sv4 = analogRead(A4);
byte full = 0;/*~(PINC) & maskC;*/
if (sv0 > 250)
full = full | B00000001;
if (sv1 > 250)
full = full | B00000010;
if (sv2 > 250)
full = full | B00000100;
if (sv3 > 250)
full = full | B00001000;


if (mouseOn == 0)
{
if(full && !keysdown) {
keysdown = 1;
tlpress = micros();
tlunpress = 0;
}
else if(!full && keysdown) {
keysdown = 0;
//Serial.print(mapPointer[prevFull_release]);
Keyboard.set_key1(mapPointer[prevFull_release]);
Keyboard.send_now();
Keyboard.set_key1(0);
Keyboard.send_now();
tlunpress = 0;
prevFull = 0;
prevFull_release = 0;
}
if(keysdown) {
if(full < prevFull) {
tlunpress = millis();
if((millis() - tlunpress) > unpress_time) {
prevFull_release = prevFull;
}
prevFull = full;
}
else if(full > prevFull) {
prevFull = full;
prevFull_release = prevFull;
tlunpress = 0;
tlpress = millis();
}
else {
//full == prevFull
if(tlunpress != 0 && (millis() - tlunpress) > unpress_time) {
prevFull_release = prevFull;
tlunpress = 0;
}
}
}
}
else
{
// do mouse stuff
}


delay(50);
}

PaulStoffregen
04-20-2013, 10:04 PM
Maybe 2 arrays, one for each axis?



const int mousemapXaxis[6] = {0, -5, 5, 0, 0, 0};
const int mousemapYaxis[6] = {0, 0, 0, -5, 5, 0};


There are other complex ways to do this with structs and pointers, but why make it really complex if a couple simple arrays will do?

Using "const" on arrays that (should) never get written is a good idea.

PaulH
04-20-2013, 10:49 PM
Okay, I started learning about arrays two days ago and the gears are slowly starting to turn. Would an array be able to let me do the following:

if sv0 state changes then move mouse left
if sv1 state changes then move mouse right
if sv2 state changes then move mouse up
if sv3 state changes then move mouse down
if svo and sv1 state changes then left click
if sv2 and sv3 state changes then right click

Or am I opening up a whole new can of worms?

Thank you for you patience!

Paul

PaulH
05-01-2013, 03:30 AM
Well, with the help of my tutor I now have the mouse control. I am still tossing around ideas that will wok best but for now this is what I have (using Paul's timing method). Left thumb flex then mouse moves left. Flex again mouse stops. Flex baby finger and mouse moves right. Flex again and mouse stops. Flex thumb and baby finger and you get "left click". Right hand is up, down, and "right click". Later on in the week (work permitting) I will plug this into my main code to allow for text, functions, and mouse control. I am sure that it needs cleaning up but I need to get some sleep.

Paul


int previousA0 = 0 ;
int previousA1 = 0;
int previousA2 = 0 ;
int previousA3 = 0;
int anyChange = 0;
int changeA0 = 0;
int changeA1 = 0;
int changeA2 = 0;
int changeA3 = 0;
elapsedMillis sinceChange = 0;
elapsedMillis mouseChange = 0;

int movingLeft = 0;
int movingRight = 0;
int movingUp = 0;
int movingDown = 0;

void setup()
{
Mouse.begin();
}

int nowA0, nowA1, nowA2, nowA3;

void loop()
{
// input 1
nowA0 = analogRead(A0);
if (nowA0 > 250 && previousA0 <= 250)
{
changeA0 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA0 = nowA0;

// input 2
nowA1 = analogRead(A1);
if (nowA1 > 250 && previousA1 <= 250)
{
changeA1 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA1 = nowA1;

// input 2
nowA2 = analogRead(A2);
if (nowA2 > 250 && previousA2 <= 250)
{
changeA2 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA2 = nowA2;

// input 4
nowA3 = analogRead(A3);
if (nowA3 > 250 && previousA3 <= 250)
{
changeA3 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA3 = nowA3;


if (anyChange == 1)
{
if (sinceChange > 50)
{
{
if (changeA0 == 1 && changeA1 == 1)
Mouse.click(MOUSE_LEFT);
else if (changeA0 == 1)
{
if (movingLeft == 1)
movingLeft = 0;
else
movingLeft = 1;
}
else if (changeA1 == 1)
{
if (movingRight == 1)
movingRight = 0;
else
movingRight = 1;

}
anyChange = 0;
changeA0 = 0;
changeA1 = 0;

}
//right
{
if (changeA2 == 1 && changeA3 == 1)
Mouse.click(MOUSE_RIGHT);
else if (changeA2 == 1)
{
if (movingUp == 1)
movingUp = 0;
else
movingUp = 1;
}
else if (changeA3 == 1)
{
if (movingDown == 1)
movingDown = 0;
else
movingDown = 1;

}
anyChange = 0;
changeA2 = 0;
changeA3 = 0;

}
}
}
if (mouseChange > 50)
mouseChange = 0;

if (movingLeft && !movingRight && mouseChange == 0)
Mouse.move(1,0);
if (movingRight && !movingLeft && mouseChange == 0)
Mouse.move(-1,0);
if (movingUp && !movingDown && mouseChange == 0)
Mouse.move(0,1);
if (movingDown && !movingUp && mouseChange == 0)
Mouse.move(0,-1);

}

PaulH
05-04-2013, 02:49 AM
I managed to merge the mouse control as well as the text generating codes together but I overlooked the need for a few changes. While trying it with the EMG sensors I noticed that I really had to flex in order for the interrupt to change the character maps. At the moment the interrupt is digital and I am using an analog input. I am going to have to have to figure out how to write an analog interrupt. I modified the timing a little as well to speed up the mouse and to extend the time allowed for multiple signals to be accepted as one input. I think that I have far too many brackets in there as well.
Still lots of work to do.


//three character sets of 16 plus mouse
int previousA0 = 0 ;
int previousA1 = 0;
int previousA2 = 0 ;
int previousA3 = 0;
int anyChange = 0;
int changeA0 = 0;
int changeA1 = 0;
int changeA2 = 0;
int changeA3 = 0;
elapsedMillis sinceChange = 0;
elapsedMillis mouseChange = 0;
int movingLeft = 0;
int movingRight = 0;
int movingUp = 0;
int movingDown = 0;


#define unpress_time 250//increase for text input timing
int sv0, sv1, sv2, sv3, sv4;
int keymap1[16] = {-1,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18};
int keymap2[16] = {-1,19,20,21,22,23,24,25,26,27,28,29,44,42,40,42};
int keymap3[16] = {-1,30,31,32,33,34,35,36,37,38,39,57,55,54,73,76};
//enter=40,backspace=42, tab=43, space=44, caps lock 57, etc.

volatile int * mapPointer = keymap1;

int mouseOn = 0;

void swap()
{
if (mapPointer == keymap1)
mapPointer = keymap2;
else if (mapPointer == keymap2)
mapPointer = keymap3;
else if (mapPointer == keymap3 && mouseOn == 0)
mouseOn = 1;
else if (mouseOn)
{
mouseOn = 0;
mapPointer = keymap1;
}

}

void setup() {
Serial.begin(115200);


attachInterrupt(5, swap, RISING);

}
void loop() {
static byte keysdown = 0;
static byte prevFull = 0;
static byte prevFull_release = 0;
static unsigned long tlpress = 0;
static unsigned long tlunpress = 0;
sv0 = analogRead(A0);
sv1 = analogRead(A1);
sv2 = analogRead(A2);
sv3 = analogRead(A3);

byte full = 0;/*~(PINC) & maskC;*/
if (sv0 > 250)
full = full | B00000001;
if (sv1 > 250)
full = full | B00000010;
if (sv2 > 250)
full = full | B00000100;
if (sv3 > 250)
full = full | B00001000;


if (mouseOn == 0)
{
if(full && !keysdown) {
keysdown = 1;
tlpress = micros();
tlunpress = 0;
}
else if(!full && keysdown) {
keysdown = 0;
//Serial.print(mapPointer[prevFull_release]);
Keyboard.set_key1(mapPointer[prevFull_release]);
Keyboard.send_now();
Keyboard.set_key1(0);
Keyboard.send_now();
tlunpress = 0;
prevFull = 0;
prevFull_release = 0;
}
if(keysdown) {
if(full < prevFull) {
tlunpress = millis();
if((millis() - tlunpress) > unpress_time) {
prevFull_release = prevFull;
}
prevFull = full;
}
else if(full > prevFull) {
prevFull = full;
prevFull_release = prevFull;
tlunpress = 0;
tlpress = millis();
}
else {
//full == prevFull
if(tlunpress != 0 && (millis() - tlunpress) > unpress_time) {
prevFull_release = prevFull;
tlunpress = 0;
}
}
}
}
else
{
// do mouse stuff
int nowA0, nowA1, nowA2, nowA3;

// input 1
nowA0 = analogRead(A0);
if (nowA0 > 250 && previousA0 <= 250)
{
changeA0 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA0 = nowA0;

// input 2
nowA1 = analogRead(A1);
if (nowA1 > 250 && previousA1 <= 250)
{
changeA1 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA1 = nowA1;

// input 3
nowA2 = analogRead(A2);
if (nowA2 > 250 && previousA2 <= 250)
{
changeA2 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA2 = nowA2;

// input 4
nowA3 = analogRead(A3);
if (nowA3 > 250 && previousA3 <= 250)
{
changeA3 = 1;
if (anyChange == 0)
{
anyChange = 1;
sinceChange = 0;
}
}
previousA3 = nowA3;


if (anyChange == 1)
{
if (sinceChange > 50) // this is basically like unpress time
{

//left hand
if (changeA0 == 1 && changeA1 == 1)
Mouse.click(MOUSE_LEFT);
else if (changeA0 == 1)
{
if (movingLeft == 1)
movingLeft = 0;
else
movingLeft = 1;
}
else if (changeA1 == 1)
{
if (movingRight == 1)
movingRight = 0;
else
movingRight = 1;

}
anyChange = 0;
changeA0 = 0;
changeA1 = 0;


//right hand

if (changeA2 == 1 && changeA3 == 1)
Mouse.click(MOUSE_RIGHT);
else if (changeA2 == 1)
{
if (movingUp == 1)
movingUp = 0;
else
movingUp = 1;
}
else if (changeA3 == 1)
{
if (movingDown == 1)
movingDown = 0;
else
movingDown = 1;

}
anyChange = 0;
changeA2 = 0;
changeA3 = 0;


} // since change > 50
} // anychange
if (mouseChange > 30)//this should control mouse speed
mouseChange = 0;

if (movingLeft && !movingRight && mouseChange == 0)
Mouse.move(2,0);
if (movingRight && !movingLeft && mouseChange == 0)
Mouse.move(-2,0);
if (movingUp && !movingDown && mouseChange == 0)
Mouse.move(0,2);
if (movingDown && !movingUp && mouseChange == 0)
Mouse.move(0,-2);



} // mouseOn
delay(5);
}