Android Multi Touch


#1

So, my multi-touch control is looking good… until i discover multi-touch is missing from JUCE Android !!

so here you go,

JuceAppActivity.java

[code] private native void handleMouseDown (int id, float x, float y, long time);
private native void handleMouseDrag (int id, float x, float y, long time);
private native void handleMouseUp (int id, float x, float y, long time);

    @Override
    public boolean onTouchEvent (MotionEvent event)
    {
        int action = event.getAction();
        switch (action & MotionEvent.ACTION_MASK)
        {
            case MotionEvent.ACTION_DOWN:
            {
                int id = event.getPointerId(0);
                handleMouseDown (id, event.getX(), event.getY(), event.getEventTime());
                return true;
            }
            case MotionEvent.ACTION_MOVE:
            { 
                int n = event.getPointerCount();
                for (int i = 0; i < n; ++i)
                {
                    int id = event.getPointerId(i);
                    handleMouseDrag (id, event.getX(i), event.getY(i), event.getEventTime());
                }
                return true;
            }
            case MotionEvent.ACTION_CANCEL:
            case MotionEvent.ACTION_UP:
            {
                int id = event.getPointerId(0);
                handleMouseUp (id, event.getX(), event.getY(), event.getEventTime());
                return true;
            }
            case MotionEvent.ACTION_POINTER_UP:
            {
                int i = (action & MotionEvent.ACTION_POINTER_INDEX_MASK) 
                    >> MotionEvent.ACTION_POINTER_INDEX_SHIFT;
                
                int id = event.getPointerId(i);
                handleMouseUp (id, event.getX(i), event.getY(i), event.getEventTime());
                return true;

            }
            case MotionEvent.ACTION_POINTER_DOWN:
            {
               int i = (action & MotionEvent.ACTION_POINTER_INDEX_MASK) 
                    >> MotionEvent.ACTION_POINTER_INDEX_SHIFT;
                
                int id = event.getPointerId(i);
                handleMouseDown (id, event.getX(i), event.getY(i), event.getEventTime());
                return true;
            }
            default: break;
        }

        return false;
    }[/code]

modules\juce_gui_basics\native\juce_android_Windowing.cpp

[code] static const int maxTouch = 4;
void handleMouseDownCallback (int id, float x, float y, int64 time)
{
lastMousePos.setXY ((int) x, (int) y);
currentModifiers = currentModifiers.withoutMouseButtons();
handleMouseEvent (id, lastMousePos, currentModifiers, time);
currentModifiers = currentModifiers.withoutMouseButtons().withFlags (ModifierKeys::leftButtonModifier);
handleMouseEvent (id, lastMousePos, currentModifiers, time);
}

void handleMouseDragCallback (int id, float x, float y, int64 time)
{
    lastMousePos.setXY ((int) x, (int) y);
    handleMouseEvent (id, lastMousePos, currentModifiers, time);
}

void handleMouseUpCallback (int id, float x, float y, int64 time)
{
    lastMousePos.setXY ((int) x, (int) y);
    currentModifiers = currentModifiers.withoutMouseButtons();
    handleMouseEvent (id, lastMousePos, currentModifiers, time);
}

JUCE_VIEW_CALLBACK (void, handleMouseDown, (JNIEnv* env, jobject view, jint id, jfloat x, jfloat y, jlong time), handleMouseDownCallback (id, (float) x, (float) y, (int64) time))
JUCE_VIEW_CALLBACK (void, handleMouseDrag, (JNIEnv* env, jobject view, jint id, jfloat x, jfloat y, jlong time), handleMouseDragCallback (id, (float) x, (float) y, (int64) time))
JUCE_VIEW_CALLBACK (void, handleMouseUp, (JNIEnv* env, jobject view, jint id, jfloat x, jfloat y, jlong time), handleMouseUpCallback (id, (float) x, (float) y, (int64) time))

[/code]

– hugh.


#2

I definitely owe you a beer :slight_smile:


#3

Jules, what do you think of adding a multitouch demo to the JuceDemo app ? It would be also very useful for testing multitouch. Here is a very small one that I wrote when I was also looking at android multitouch support and trying to get it to work (and failed) :

[code]class MultitouchDemo : public Component {
Array<Point > mtouch;

Point &touch(int id) {
while (id >= mtouch.size()) mtouch.add(Point(-1,-1));
return mtouch.getReference(id);
}

void mouseDown(const MouseEvent &e) {
int id = e.source.getIndex();
DBG(“mouseDown " << id);
touch(id) = Point(e.x, e.y);
repaint();
}
void mouseUp(const MouseEvent &e) {
int id = e.source.getIndex();
DBG(“mouseUp " << id);
touch(id) = Point(-1, -1);
repaint();
}
void mouseDrag(const MouseEvent &e) {
int id = e.source.getIndex();
//DBG(“mouseDrag: source=” << id << " x=” << e.x << " y=” << e.y);
touch(id) = Point(e.x, e.y);
repaint();
}
void paint(Graphics &g) {
for (int i=0; i < mtouch.size(); ++i) {
if (i == 0) g.setColour(Colours::red);
if (i == 1) g.setColour(Colours::blue);
if (i == 2) g.setColour(Colours::green);
if (i >= 3) g.setColour(Colours::yellow);
int x = mtouch[i].getX(), y = mtouch[i].getY();
if (x != -1 || y != -1)
g.fillEllipse(x-60, y-60, 120, 120);
}
}

};
[/code]

I think it shows there is an issue in the code you have commited , for example if you press two fingers, and then raise one of the fingers, JUCE considers that both fingers have been raised


#4

if you press two fingers, and then raise one of the fingers, JUCE considers that both fingers have been raised[quote][/quote]

That could be a problem with the code i posted. i couldnt figure out why this was. basically, if you lift one finger, nothing worked until the other finger is lifted, after which it worked as expected.

– hugh.


#5

I'm indeed experiencing this in my Juce android app.

Was it updated in Juce since ?

 


#6

 

hmm. maybe it's time i fixed this one. I'll add it to my list....


#7

Btw, I've put 16 as the maximum number of touches but obviously a higher value is more future-proof.

 

 

 


 


#8

I think I've fixed the android multitouch bug :

 

In the handleMouseDownCallback and handleMouseUpCallback, you use a single ModifierKeys member, while you should have one for each touch index.

So I added in juce_Android_Windowing.cpp:

ModifierKeys currentModifiersList[16];

to the private members of AndroidComponentPeer, then initialize this list in the constructor:

for(int i=0;i<16;i++)
            currentModifiersList[i] = 0;

And then I reimplemented the callbacks like this:

void handleMouseDownCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
        
        currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons().withFlags (ModifierKeys::leftButtonModifier);
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
       
    }

    void handleMouseDragCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

    void handleMouseUpCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
       currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons();
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

 

This fixes it for me.

 


#9

Hmm.. I don't think that's the right approach to fixing it - the library has always assumed that there's a single set of modifier keys, but each mouse-source behaves independently. That works for multi-touch on windows and iOS, not sure why it's not working here...


#10

This is indeed causing some problems with combobox's popups for instance, so I modified the methods like this:

 

void handleMouseDownCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
        currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons().withFlags (ModifierKeys::leftButtonModifier);
        currentModifiers = currentModifiersList[index];
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);      
    }

    void handleMouseDragCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

    void handleMouseUpCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
       currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons();
       currentModifiers = currentModifiersList[index];
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

And now everything seems to work as intended.

 

I know this is not the most elegant fix ever made, but at least it should give you some indications on what's wrong with the current code ?

 


#11

Ok, I've just had a bit of a tinker-around with this. My system's in a bit of a mess and I can't test android builds at the moment, but I've changed the code so it should work more like the iOS and Windows versions.