Juce receives the first touch event very late


#1

Hi,

i have a problem with the user event to the app reaction time.

When i touch the screen on my test tablet it takes about 250 to 500ms after e.g. a slider receives this event and change it's value- But when I stay touched I can change the slider position it's value will changed immediately without any delay. I have no idea what the problem can be for this "first-touch"-delay, but other apps are much faster on this device.

Does anybody know this problem or any idea what it can be?

Thanks


#2

This video show the "lag" or "delay": https://www.youtube.com/watch?v=EPPwCSTpyj4
(it's the same issue with the juce demo)

Any ideas?


#3

Is it just android delaying sending the first touch message until it has decided whether or not it's part of a gesture? There's certainly nothing in juce itself that would cause behaviour like that.


#4

Hi Jules,

the gesture was a good point to search - the onTouchEvent method will be called immediately after the touch and it is a ACTION_DOWN event. All the time get lost until the next onDraw call.

Later I have set the size of the app main window to 25% and it was much faster (your HelloWorkd project was also very slow (quit) if it is in fullScreen).

I think it's a performance problem of the device and how android handle the redraw???

I have tried to call invalidate() and some other workarounds, but no success to tweak it.

 

3 days for nothing :-(


#5

Ok, I have bought a new test device (intel atom dual core). (first test device arm quad core). On the intel device every thing working fine... can it be a thread sync problem on the quad core device?

 

Another test on the the sluggish arm device:

I have added a component directly to the desktop and have resized the main window a bit. The desktop component has no delay on the fist touch, but the main window things. Have a look on this vid: https://www.youtube.com/watch?v=ffufRoJTLoQ

...oO...

 


#6

EDIT: removed.


#7

Sorry for the remove before,  but it was ill formed.

Android seems to request always a redraw over the full screen. The canvas size in handlePaintCallback(...) has always the full screen size. Should this be? This will repaint all the components and this is may be the reason for the lag.

Also does android call a redraw after a touchevent without invalidate any rect on the ui from juce!


#8

Ok, sorry. That was a bug that i have added by my self to the juce code. Sorry.

But that android request a full redraw on touch is true.


#9

The sluggish device is running now: https://www.youtube.com/watch?v=C8lzkl984Yc

To get the sluggish device feeling better I have added a full screen cache class.

class FullScreenCache
{
    int width;
    int height;

    bool _is_init;
    GlobalRef buffer;
    jint* _dest;
    Image _image;

public:
    inline bool is_init()
    {
        return _is_init;
    }

    inline void update( int x_offset_, int y_offset_, Image& img )
    {
        Graphics g (_image);
        g.drawImageAt(img,x_offset_,y_offset_);
    }

    inline Image& get_image()
    {
        return _image;
    }

    inline GlobalRef get_buffer()
    {
        return buffer;
    }

    inline bool init()
    {
        if( not _is_init )
        {
            width = Desktop::getInstance().getDisplays().getMainDisplay().userArea.getWidth();
            height = Desktop::getInstance().getDisplays().getMainDisplay().userArea.getHeight();
            buffer = GlobalRef( getEnv()->NewIntArray(width * height) );
            _dest = getEnv()->GetIntArrayElements ((jintArray) buffer.get(), 0);
            _image = Image( new PreallocatedImage (width, height, _dest, false) );
        }
        _is_init = true;
    }

    inline static FullScreenCache& instance()
    {
        static FullScreenCache this_;
        return this_;
    }

private:
    FullScreenCache()
        :
        width( 0 ),
        height( 0 ),
        _image( Image::ARGB, 0,0, false ),
        _is_init( false )
    {}

Changed the handlePaintCallback to use the cache if it is just a touch down redraw request:

   void handlePaintCallback (JNIEnv* env, jobject canvas, bool is_after_touch_ )
    {
        jobject rect = env->CallObjectMethod (canvas, CanvasMinimal.getClipBounds);
        const int left = env->GetIntField (rect, RectClass.left);
        const int top = env->GetIntField (rect, RectClass.top);
        const int right = env->GetIntField (rect, RectClass.right);
        const int bottom = env->GetIntField (rect, RectClass.bottom);
        const int height = bottom - top;
        const int width = right - left;

        env->DeleteLocalRef (rect);
        const Rectangle<int> clip (left, top, width, height);

        // check if we need the cache hack
        FullScreenCache& cache = FullScreenCache::instance();
        if( use_cache_hack and is_fullscreen( width, height ) )
        {
            // make cache
            if( not cache.is_init() or not is_after_touch_ )
            {
                cache.init();

                LowLevelGraphicsSoftwareRenderer g (cache.get_image());
                handlePaint (g);
            }

            // use cache
            env->CallVoidMethod (canvas, CanvasMinimal.drawBitmap, (jintArray) cache.get_buffer().get(), 0, clip.getWidth(),
                                 (jfloat) clip.getX(), (jfloat) clip.getY(),
                                 clip.getWidth(), clip.getHeight(), false, (jobject) 0);
        }
        else

        {
            const int sizeNeeded = clip.getWidth() * clip.getHeight();
            if (sizeAllocated < sizeNeeded)
            {
                buffer.clear();
                sizeAllocated = sizeNeeded;
                buffer = GlobalRef (env->NewIntArray (sizeNeeded));
            }
            if (jint* dest = env->GetIntArrayElements ((jintArray) buffer.get(), 0))
            {
                Image temp (new PreallocatedImage (clip.getWidth(), clip.getHeight(), dest, ! component.isOpaque()));
                {
                    LowLevelGraphicsSoftwareRenderer g (temp);
                    g.setOrigin (-clip.getPosition());
                    handlePaint (g);
                }

                env->ReleaseIntArrayElements ((jintArray) buffer.get(), dest, 0);
                env->CallVoidMethod (canvas, CanvasMinimal.drawBitmap, (jintArray) buffer.get(), 0, clip.getWidth(),
                                     (jfloat) clip.getX(), (jfloat) clip.getY(),
                                     clip.getWidth(), clip.getHeight(), true, (jobject) 0);

                if( use_cache_hack and cache.is_init() )
                    cache.update(clip.getX(),clip.getY(),temp);

            }
        }
    }

   inline static bool is_fullscreen( int w_, int h_ )
  {
    static int w = Desktop::getInstance().getDisplays().getMainDisplay().userArea.getWidth();
    static int h = Desktop::getInstance().getDisplays().getMainDisplay().userArea.getHeight();

    if( w == w_ )
        if( h == h_  )
            return true;

    return false;
   }

JUCE_VIEW_CALLBACK (void, handlePaint,      (JNIEnv* env, jobject view, jlong host, jobject canvas, jboolean is_after_touch), handlePaintCallback (env, canvas, is_after_touch))

At the Java side:

        private native void handlePaint (long host, Canvas canvas, boolean is_after_touch );

        private boolean is_after_touch = false;

        @Override
        public void onDraw (Canvas canvas)
        {
            handlePaint (host, canvas, is_after_touch);
            is_after_touch = false;
        }

        @Override
        public boolean onTouchEvent (MotionEvent event)
        {
            ...
            case MotionEvent.ACTION_DOWN:
                is_after_touch = true;
                //Debug.startMethodTracing("handleMouseDown");
                handleMouseDown (host, event.getPointerId(0), event.getX(), event.getY(), event.getEventTime());
                return true;
            ...
        }

This is ~300ms faster than before.

NOTE: this on touch full redraw is only on the arm device. The x86 device does not request a full redraw on touch. I don't know what's going wrong! I have also opend a thread on stackoverflow: http://stackoverflow.com/questions/25832648/why-does-android-requests-a-full-redraw-after-a-touch-event-but-only-on-a-arm-d

Sorry for that bad thread quality - it was driving me crazy!

EDIT: change code: memory for cache will only allocated if a device have the onTouch full redraw "problem"


#10

From another thread, here is my comment on (what I believe) is this issue:

"There is an issue in the current JuceAppActivity java template when using an OpenGL context that leads to terrible touch event performance.  In the ComponentPeerView inner class, the onDraw() gets called by the runtime on every touch down event, and handlePaint (a native method) is called.  But the problem is that when it has added a GL context, the handlePaint() attempts to render in the *software rendering* pipeline causing massive slowdowns.  I added an additional boolean state to ComponentPeerView to keep track of when an OpenGL context was added (in its createGLView() method), and if so, *do not* call handlePaint, because it is unnecessary for the GL renderer.

I don't know why the java runtime is calling the onDraw method on touch down events in the first place, but it seems like a bad idea regardless.  Preventing onPaint from being called was critical when GL is the context, though.  Basically, if it is an openGL context, just doing nothing in  onDraw() has no adverse effect on the graphics of the app."

 

As for your fix above with the caching, maybe it's necessary in software rendering mode, but it isn't with OpenGL.

 


#11

+ 1000

You are right, I have added the boolean as you describe and use openGL now, because now the openGL performance is better than in software render mode. (Also i have disabled the the cache workaround)

Thanks!