Android Multi Touch

hugh · November 13, 2012, 1:01pm

So, my multi-touch control is looking good… until i discover multi-touch is missing from JUCE Android !!

so here you go,

JuceAppActivity.java

[code] private native void handleMouseDown (int id, float x, float y, long time);
private native void handleMouseDrag (int id, float x, float y, long time);
private native void handleMouseUp (int id, float x, float y, long time);

    @Override
    public boolean onTouchEvent (MotionEvent event)
    {
        int action = event.getAction();
        switch (action & MotionEvent.ACTION_MASK)
        {
            case MotionEvent.ACTION_DOWN:
            {
                int id = event.getPointerId(0);
                handleMouseDown (id, event.getX(), event.getY(), event.getEventTime());
                return true;
            }
            case MotionEvent.ACTION_MOVE:
            { 
                int n = event.getPointerCount();
                for (int i = 0; i < n; ++i)
                {
                    int id = event.getPointerId(i);
                    handleMouseDrag (id, event.getX(i), event.getY(i), event.getEventTime());
                }
                return true;
            }
            case MotionEvent.ACTION_CANCEL:
            case MotionEvent.ACTION_UP:
            {
                int id = event.getPointerId(0);
                handleMouseUp (id, event.getX(), event.getY(), event.getEventTime());
                return true;
            }
            case MotionEvent.ACTION_POINTER_UP:
            {
                int i = (action & MotionEvent.ACTION_POINTER_INDEX_MASK) 
                    >> MotionEvent.ACTION_POINTER_INDEX_SHIFT;
                
                int id = event.getPointerId(i);
                handleMouseUp (id, event.getX(i), event.getY(i), event.getEventTime());
                return true;

            }
            case MotionEvent.ACTION_POINTER_DOWN:
            {
               int i = (action & MotionEvent.ACTION_POINTER_INDEX_MASK) 
                    >> MotionEvent.ACTION_POINTER_INDEX_SHIFT;
                
                int id = event.getPointerId(i);
                handleMouseDown (id, event.getX(i), event.getY(i), event.getEventTime());
                return true;
            }
            default: break;
        }

        return false;
    }[/code]

modules\juce_gui_basics\native\juce_android_Windowing.cpp

[code] static const int maxTouch = 4;
void handleMouseDownCallback (int id, float x, float y, int64 time)
{
lastMousePos.setXY ((int) x, (int) y);
currentModifiers = currentModifiers.withoutMouseButtons();
handleMouseEvent (id, lastMousePos, currentModifiers, time);
currentModifiers = currentModifiers.withoutMouseButtons().withFlags (ModifierKeys::leftButtonModifier);
handleMouseEvent (id, lastMousePos, currentModifiers, time);
}

void handleMouseDragCallback (int id, float x, float y, int64 time)
{
    lastMousePos.setXY ((int) x, (int) y);
    handleMouseEvent (id, lastMousePos, currentModifiers, time);
}

void handleMouseUpCallback (int id, float x, float y, int64 time)
{
    lastMousePos.setXY ((int) x, (int) y);
    currentModifiers = currentModifiers.withoutMouseButtons();
    handleMouseEvent (id, lastMousePos, currentModifiers, time);
}

…

JUCE_VIEW_CALLBACK (void, handleMouseDown, (JNIEnv* env, jobject view, jint id, jfloat x, jfloat y, jlong time), handleMouseDownCallback (id, (float) x, (float) y, (int64) time))
JUCE_VIEW_CALLBACK (void, handleMouseDrag, (JNIEnv* env, jobject view, jint id, jfloat x, jfloat y, jlong time), handleMouseDragCallback (id, (float) x, (float) y, (int64) time))
JUCE_VIEW_CALLBACK (void, handleMouseUp, (JNIEnv* env, jobject view, jint id, jfloat x, jfloat y, jlong time), handleMouseUpCallback (id, (float) x, (float) y, (int64) time))

[/code]

– hugh.

jules · November 13, 2012, 5:14pm

I definitely owe you a beer

jpo · November 14, 2012, 11:56pm

Jules, what do you think of adding a multitouch demo to the JuceDemo app ? It would be also very useful for testing multitouch. Here is a very small one that I wrote when I was also looking at android multitouch support and trying to get it to work (and failed) :

[code]class MultitouchDemo : public Component {
Array<Point > mtouch;

Point &touch(int id) {
while (id >= mtouch.size()) mtouch.add(Point(-1,-1));
return mtouch.getReference(id);
}

void mouseDown(const MouseEvent &e) {
int id = e.source.getIndex();
DBG(“mouseDown " << id);
touch(id) = Point(e.x, e.y);
repaint();
}
void mouseUp(const MouseEvent &e) {
int id = e.source.getIndex();
DBG(“mouseUp " << id);
touch(id) = Point(-1, -1);
repaint();
}
void mouseDrag(const MouseEvent &e) {
int id = e.source.getIndex();
//DBG(“mouseDrag: source=” << id << " x=” << e.x << " y=” << e.y);
touch(id) = Point(e.x, e.y);
repaint();
}
void paint(Graphics &g) {
for (int i=0; i < mtouch.size(); ++i) {
if (i == 0) g.setColour(Colours::red);
if (i == 1) g.setColour(Colours::blue);
if (i == 2) g.setColour(Colours::green);
if (i >= 3) g.setColour(Colours::yellow);
int x = mtouch[i].getX(), y = mtouch[i].getY();
if (x != -1 || y != -1)
g.fillEllipse(x-60, y-60, 120, 120);
}
}

};
[/code]

I think it shows there is an issue in the code you have commited , for example if you press two fingers, and then raise one of the fingers, JUCE considers that both fingers have been raised

hugh · November 20, 2012, 11:28pm

if you press two fingers, and then raise one of the fingers, JUCE considers that both fingers have been raised[quote][/quote]

That could be a problem with the code i posted. i couldnt figure out why this was. basically, if you lift one finger, nothing worked until the other finger is lifted, after which it worked as expected.

– hugh.

shini · October 16, 2013, 3:54pm

I'm indeed experiencing this in my Juce android app.

Was it updated in Juce since ?

hugh · October 17, 2013, 10:45am

hmm. maybe it's time i fixed this one. I'll add it to my list....

shini · October 21, 2013, 1:31pm

Btw, I've put 16 as the maximum number of touches but obviously a higher value is more future-proof.

shini · October 22, 2013, 4:51pm

I think I've fixed the android multitouch bug :

In the handleMouseDownCallback and handleMouseUpCallback, you use a single ModifierKeys member, while you should have one for each touch index.

So I added in juce_Android_Windowing.cpp:

ModifierKeys currentModifiersList[16];

to the private members of AndroidComponentPeer, then initialize this list in the constructor:

for(int i=0;i<16;i++)
currentModifiersList[i] = 0;

And then I reimplemented the callbacks like this:

void handleMouseDownCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);

        currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons().withFlags (ModifierKeys::leftButtonModifier);
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);

    }

    void handleMouseDragCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

    void handleMouseUpCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
       currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons();
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

This fixes it for me.

jules · October 23, 2013, 12:33pm

Hmm.. I don't think that's the right approach to fixing it - the library has always assumed that there's a single set of modifier keys, but each mouse-source behaves independently. That works for multi-touch on windows and iOS, not sure why it's not working here...

shini · October 24, 2013, 10:57am

This is indeed causing some problems with combobox's popups for instance, so I modified the methods like this:

void handleMouseDownCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
        currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons().withFlags (ModifierKeys::leftButtonModifier);
       currentModifiers = currentModifiersList[index];
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

    void handleMouseDragCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

    void handleMouseUpCallback (int index, float x, float y, int64 time)
    {
        lastMousePos.setXY ((int) x, (int) y);
       currentModifiersList[index] = currentModifiersList[index].withoutMouseButtons();
       currentModifiers = currentModifiersList[index];
        handleMouseEvent (index, lastMousePos, currentModifiersList[index], time);
    }

And now everything seems to work as intended.

I know this is not the most elegant fix ever made, but at least it should give you some indications on what's wrong with the current code ?

jules · October 28, 2013, 10:27am

Ok, I've just had a bit of a tinker-around with this. My system's in a bit of a mess and I can't test android builds at the moment, but I've changed the code so it should work more like the iOS and Windows versions.

Topic		Replies	Views
Touch handling is extremely laggy on Android Android	6	513	August 11, 2022
iOS native gestures MacOSX and iOS	1	744	December 4, 2012
Touch interface questions General JUCE discussion	5	533	October 19, 2011
Multitouch on iPad MacOSX and iOS	16	1112	July 15, 2011
How to do iOS-Style Multitouch handling? General JUCE discussion	2	1119	August 30, 2017

Android Multi Touch

Purchase

Discover

Learn

Support

About

Events

Android Multi Touch

Related Topics

Purchase

Discover

Learn

Support

About

Events