Still with pen issues

Related: Pen issue

Apparenty, I am the only one using a touch computer. I haven’t worked on my projects for a while, today I updated to 5.1.1 and found that nothing has changed on that side. I still cannot use my pen with JUCE applications, including the Projucer. Since my tablet doesn’t have a mouse and keyboard that’s quite an issue for me.

Is it just a problem with my old tablet? It’s an ASUS B121, and maybe the drivers are not Windows 10 compliant; still, every other application works fine. Anyone on a newer tablet, maybe a Surface, can confirm everything is working there?

Hi!

I replied also in your other thread with a link to where I previously unsuccessfully tried starting discussion on this issue.

I’ll reply here as well to point out that no, it is not your computer. I test my software on a windows 7 dell latitude xt2, a windows 10 Lenovo Yoga, and a desktop PC with a wacom touch screen (cintiq), also windows 10. I have the same problem on all, with juce software only - be it my own software or the Juce demo, the problem is the same.

I only run Juce 4, since I did not want to pay for the upgrade to 5 (there’s nothing much new in 5 that I care for besides bug-fixes), but apparently nothing has been done on 5 either to fix this issue…

We added support for the Windows pointer API to JUCE a while ago that seemed to improve pen input considerably when tested on a Microsoft Surface running Windows 10, however this API isn’t supported on older devices which will fall back to using the old method for handling mouse/touch events. The method that checks for the availability of this is here - can you put a breakpoint in and see what the value of canUsePointerAPI is?

canUsePointerAPI gives true, all the three tested win APIs are !nullptr.

I have a different juce_win32_Windowing.cpp btw, checkForPointerAPI() is at line 252. I am not on the git version, I’m on official 5.1.1

The best pieces of software I run on my tablet are Reaper and Notepad++. They both work flawlessly regarding pen / touch response. Notepad++ even scrolls the text with a vertical touch gesture, while it selects text with an horizontal gesture. Hats off. Can we have a similar support in Juce?

EDIT: Notepad++ also increases / decreases the font size when pinch-zooming. Projucer flickers as hell when trying the same thing

OK. The next thing would be to check that you are getting WM_POINTER messages from the OS - can you put a breakpoint here and check that it’s being hit when you touch the screen with the pen? If it is then can you put a breakpoint here and check that the correct behaviour occurs i.e. the pointerType is pen, it’s not returning false and bailing out of the function etc.

WM_POINTERDOWN messages are received for both pen and touch actions, and they are correctly recognized by getPointerType().

@ed95… we should really try and understand why menus don’t work! That’s quite gross an error, and it should be easy to fix, and it could shed light on the rest of the issues. I’ll shot a short video and post it

The video is here:

https://app.box.com/s/evtevd2ju1aaiea25r48el83x89g94id

Note that:

  • I use the pen throughout the video
  • clicking on an item in the main menu bar opens the corresponding menu
  • moving the pen over an item in the popup menu makes it blink
  • clicking an item in the popup menu has no effect (obviously, it works if I connect an USB mouse)

I have overriden Component::mouseXXX() callbacks in PopupMenu::HelperClasses::ItemComponent, and printed some trace with DBG(). This is the sequence of messages received by an ItemComponent when opening a menu and clicking on it (numbers are mouseEvent.mods.getRawFlags()):

Show script parameters: mouseEnter 0
Show script parameters: mouseMove 0 (some dozens) (blinking happens here)
Show script parameters: mouseDown 16
Show script parameters: mouseDrag 16 (a dozen)
Show script parameters: mouseUp 16
Show script parameters: mouseMove 0
Show script parameters: mouseExit 0
Show script parameters: mouseEnter 0
Show script parameters: mouseMove 0 (a dozen)

I think it’s expected behaviour… isn’t it? A build of the same application made with the previous version works perfectly, on the same tablet. I could fetch the sources for that build from SVN (Juce modules including) and repeat the test, but would that be useful?

BTW, I liked the look of the old build more! (dark grey menus that hilight in black are quite hard to see…)

Besides Menu Items, Buttons don’t work with the pen (they work with the touch, though). An interesting behaviour is that, after clicking a button with the pen, I have to click it twice with the mouse for it to trigger its action.

There are differences betweem mouse and pen operation in the method PopupMenu::HelperClasses::MouseSourceState::checkButtonState(), at juce_PopupMenu.cpp(1066).

I put a breakpoint at line 1083, it’s the second branch of the main if() statement in that method. The line reads:

    else if (wasDown && timeNow > window.windowCreationTime + 250
           && ! (isDown || overScrollArea))

The condition of the breakpoint is (wasDown == true).

I click on an item in the main menu bar (say, “File”), and the menu opens. Then I move to the first Item in that menu (say, “Open”) and click on it.

  • When using the mouse, the breakpoint is hit only when clicking the mouse on the Item (that is, “Open”);

  • When using the pen, the breakpoint is hit as soon as the pen hovers on the Item (“Open”, as above); so, wasDown is true while I move the pen on “Open”, whereas it was false in the mouse case (until I actually clicked on “Open”)

Unfortunately we don’t have any pen-compatible devices here to test with but from what you’ve said it sounds like the modifier keys aren’t being cleared. Can you try putting this line:

currentModifiers = currentModifiers.withoutMouseButtons();

inside the if (isUp) block and before before the handleMouseEvent() call here?

That’s why it would be good if you made the pointer/pen/touch support optional, by way of some compile-time flag.

The change you suggest does something: menuItems now are triggered, but still flicker while being hovered by the pen. Buttons don’t work. After trying to cilck a button, a MenuItem needs 2 clicks to be triggered.

I have cleared the contents of checkForPointerAPI(), in juce_win32_Windowing.cpp, so that canUsePointerAPI remains at false, and rebuilt my application. Works like butter.

The driver knows better than Juce! :slight_smile: :slight_smile: :slight_smile:

@onar3d, can you try if this solves your problems too?

Grazie!

Forcing it to return false worked for the pen, but not for the finger, on a wacom cintiq on windows 10, with Juce 5. An improvement, but touch-right-click should still also be possible with finger touch…

With canUsePointerAPI as false, the rectangle from a finger long-press appears, but the right click menu doesn’t, unlike the case with the pen, where it now works as it should.

With checkForPointerAPI untouched in Juce 5, nothing at all happens with long press now, no animation appearing as when long pressing in the windows notepad, no nothing, neither with finger nor with pen. In Juce 4, the animation appeared, but the right-click event wasn’t triggered.

@onar3d you’re right, the same happens here on my tablet. I treated the function canUseMultiTouch() the same way and now right-click with finger-long-press works too.

This is how i modified the file juce_win32_Windowing.cpp, here’s the relevan portion:

static bool hasCheckedForMultiTouch = false;

static bool canUseMultiTouch()
{
#ifdef JUCE_USE_POINTER_API
    if (registerTouchWindow == nullptr && ! hasCheckedForMultiTouch)
    {
        hasCheckedForMultiTouch = true;

        registerTouchWindow   = (RegisterTouchWindowFunc)   getUser32Function ("RegisterTouchWindow");
        getTouchInputInfo     = (GetTouchInputInfoFunc)     getUser32Function ("GetTouchInputInfo");
        closeTouchInputHandle = (CloseTouchInputHandleFunc) getUser32Function ("CloseTouchInputHandle");
        getGestureInfo        = (GetGestureInfoFunc)        getUser32Function ("GetGestureInfo");
    }

    return registerTouchWindow != nullptr;
#else
    hasCheckedForMultiTouch = true;
    return false;
#endif // JUCE_USE_POINTER_API
}

typedef BOOL (WINAPI* GetPointerTypeFunc) (UINT32, POINTER_INPUT_TYPE*);
typedef BOOL (WINAPI* GetPointerTouchInfoFunc) (UINT32, POINTER_TOUCH_INFO*);
typedef BOOL (WINAPI* GetPointerPenInfoFunc) (UINT32, POINTER_PEN_INFO*);

static GetPointerTypeFunc      getPointerTypeFunction = nullptr;
static GetPointerTouchInfoFunc getPointerTouchInfo = nullptr;
static GetPointerPenInfoFunc   getPointerPenInfo = nullptr;

static bool canUsePointerAPI = false;

static void checkForPointerAPI()
{
#ifdef JUCE_USE_POINTER_API
    getPointerTypeFunction = (GetPointerTypeFunc) getUser32Function ("GetPointerType");
    getPointerTouchInfo    = (GetPointerTouchInfoFunc) getUser32Function ("GetPointerTouchInfo");
    getPointerPenInfo      = (GetPointerPenInfoFunc) getUser32Function ("GetPointerPenInfo");

    canUsePointerAPI = (getPointerTypeFunction != nullptr
                     && getPointerTouchInfo    != nullptr
                     && getPointerPenInfo      != nullptr);
#endif // JUCE_USE_POINTER_API
}
1 Like

I tested the above too and indeed finger long-press right-click works now.

BUT, multitouch no longer works :slight_smile:

I’ll eventually get to looking into this too before releasing, for now I’m too deep in other ends of the codebase to switch tasks :confused:

Thank you for the updates this far!

@onar3d, nothing describes it better than the English saying: you can’t have the cake and eat it! :smiley: In this case, you can’t eat touch events and also have emulated right-mouse-clicks! :wink:

@splisp It doesn’t seem to be as mutually exclusive as you say - Strobe2 behaves exactly the way I want, and assuming Strobe2 is done with Juce, maybe there isn’t necessarily too much to do to integrate this into the library either.

I’m actually curious as to the extent to which fxpansion (now Roli…) had to carry out custom development to achieve this with Strobe2:

The GUI has full multi-touch with fingers, I can play chords on the on-screen keyboard, and drag several sliders.

Sure, with more than one finger on the Strobe2 GUI, no right-click long-press animation appears. That is fine and as expected.

But, with a single-finger, and with pen, long-press right click works just fine!

So clearly, having enabled one for the GUI, does not disqualify from having also the other, and while I don’t know how much work it was to achieve, Roli does own the Strobe2 codebase doing this, since they acquired fxpansion…

That’s interesting @onar3d. I actually verified that Reaper behaves that way too: when enabling multi-touch support (in the Preferences) several gestures are handled (and even user-assignable), but a long-press still fires the emulated mouse-right-click (I can tell it’s the emulated one because it’s the very same ‘growing square’ animation appearing anywhere else).

Now, this MSDN page shows the correct way to handle touch messages (and, how to NOT handle them and pass them to DefWindowProc()). I have to verify whether this behaviour is possible in Juce:

https://msdn.microsoft.com/en-us/library/windows/desktop/dd371581(v=vs.85).aspx

BTW, are there ‘plain’ widgets in the Strobe 2 GUI? Maybe a custom Component has the necessary support from Juce to implement the full, ‘double’ behaviour, and that Strobe GUI is made of custom Components that behave good in that way - whereas plain widgets don’t.