Making Clean UMP on Mac

General JUCE discussion
1

I have been working on making my Juce APP listen and speak UMP. After weeks of struggling I enlisted Cluade who was able to help me craft a system of call backs and circular buffers to get the data flowing. Why was it really necessary to completely rearchitect the MIDI interface? Couldn’t we just get a stream of 32 bit words that can be parsed at run time?

I have the windows side working well, it was way to hard to do but I made it work.

The MAC side has been a much harder pull off. I have almost everything working but:

when I try to send from my app a message type 4 registered controller It gets demoted to a generic message type 2 data entry MSB before being put on the USB wire. Both Claude and Google think there is some sort of .connectOutput(myEndpointId,ump::PacketProtocol::MIDI_2_0). But I can’t get it to compile.

When I send in type 2 Messages are translates up message type 4. I know the spec spends many pages droning on about how to handle these translations. But really I just want the 32 bits i sent no more.

As you probably know I was heavily involved with creating the UMP architecture and specification. It really bothers me that you think you should trash my midi messages.

On another beef,

I suspect it is not the fault of juce that the Mac relies on CI for getDeviceInfo() which it not even reported correctly. Whereas the Windows reports the message type F endpoint info negotiation.

So please tell me how to get the UMP messages I send from my app to my device. And how to get the messages from my device back to my app. Please don’t change my messages.

Sorry for the harsh tone

Dave

2

Hi, Dave –

Not sure if this will be useful to you or not, but I showed this clean page rewrite of my MIDI 2 scope app at ADC Japan a few weeks ago. At some point I’ll write up my version of a guide to MIDI 2 programming with JUCE, but until then, see if you can get anything useful out of this (and maybe if all you need is to see and decode UMP packets this will be all you need).

I don’t actually have a true UMP device to test this with, so I’m curious to see what it does with your gear.

Midi_2_0_Update
Midi_2_0_Update932×646 122 KB

1 Like
3

Most OSs actually don’t provide the raw UMP stream with all the endpoint handshaking stuff to user applications. The information returned from the discovery process is cached at the OS layer. You might have multiple applications all communicating with the same hardware device, so getting the OS to cache discovery results means that it’s not necessary for each app to run its own discovery process, which reduces overall MIDI traffic. A lot of the rearchitecture in JUCE was done to provide new interfaces to surface this cached device metadata.

Since MIDI 2.0 is bidirectional, it’s very important to be able to match UMP inputs and outputs on the same endpoint. We didn’t really have a way of doing that in the old API, so we needed to add this functionality in the new API.

Some OSs also provide useful new features, like the concept of a ‘MIDI session’ on Windows, where a single app can host several distinct MIDI setups, which is useful for apps like web browsers where each tab might want to have its own MIDI setup. We wanted to surface this feature in JUCE, which also required adjusting the shape of the API.

Finally, the old API had some shortcomings, such as being unable to detect when a device got disconnected, which were addressed in the new API.

It’s difficult to know for sure what’s going on here, but on macOS JUCE’s ump::Output doesn’t do any translation before passing packets to the OS. JUCE can’t control any translation that the OS decides to do after that. If your UMP endpoint advertises itself as supporting MIDI 2.0 protocol, and if both sides have agreed to use MIDI 2.0 protocol, then (from memory) it’s an error to send a MIDI 1.0 protocol message to that endpoint unless you’re targeting a group on the endpoint that represents a proxied MIDI 1.0 port. It might be that the OS is trying to adhere to the spec by converting the sent message to match the agreed protocol.

I’m not sure this is possible on macOS. I think it’s a feature, not a bug, that the OS handles device discovery globally, along with message translation where appropriate. In general JUCE doesn’t change outgoing messages, but will translate incoming messages according to the protocol passed to connectInput(). If the OS filters any messages before they reach a client application, there’s not much that JUCE can do about that.

If you know that your tone is harsh, please fix it before posting.

Some parting ideas:

  • You’re already using Claude, so maybe you could ask it to build you a simple commandline app that communicates with your device using plain CoreMIDI calls, to check whether it’s possible. If what you’re asking can’t be done via CoreMIDI, then it can’t be done in JUCE (since we build on CoreMIDI).
  • You’re involved in the MIDI standards board, which also has delegates from Apple, so you could try contacting them to see whether the behaviour you want is supported under macOS.
2 Likes
4

Bret,

I will try download your thing and give it a try. I would be happy to provide you with one of my scanner boards and the USB interface so you could try it out. I can also give you my app that controls it. I use a lot of flexData and Sysex8 and the new type4 registered controllers we defined for the Piano Profile. It your interested in tinkering send me your address and I get it in the mail.

Thanks Dave

5

Hi Reuben,

I am digging in I see the MacOS is querying the endpoint info also. It appears it does. I have a capture buffer csv from my beagle 480.

Here is a summary provided by Claude:

Hi all,

Following up on this thread with hard evidence from a USB protocol analyzer (Total Phase Beagle), captured at the wire level independent of JUCE/CoreMIDI.

Environment

  • JUCE 8.0.12 (release)
  • macOS 26.3.1 (Tahoe)
  • Device: custom MIDI 2.0 class-compliant device, USB VID:PID = 0425:0F01, MIDI-CI manufacturer ID = 0x09
  • Same device/firmware tested cleanly on Windows with correct getDeviceInfo() results

Summary

getDeviceInfo() on universal_midi_packets::Endpoint returns all zeroes for manufacturer, family, model, and revision on macOS, even though the device is confirmed (via USB capture) to be sending a correct MIDI-CI Discovery Reply with manufacturer ID 0x09 on the wire. hasMidi2Support() also returns false for this endpoint on macOS, despite the device actively negotiating and exchanging MIDI 2.0 UMP / MIDI-CI traffic over USB.

Evidence

Captured the USB traffic during app launch with a Beagle USB 480 analyzer. The Discovery Reply from the device (UMP Type F, Sub-ID#2 = 0x71) is clearly present and correctly formed:

7F 7E 16 30 54 02 71 0D 00 00 26 30 7C 3A 73 00
09 6D 26 30 00 05 00 00 00 01 26 30 09 00 01 02
00 04 26 30 00 00 00 04 F7 7F 31 30 00 00 F7 00

This decodes to a Discovery Reply (Sub-ID#2 0x71) containing manufacturer ID 0x09, family 0x0201, and model 0x0400 — i.e., the device’s identity is correctly transmitted to the host. CoreMIDI is therefore receiving this data; it is not being surfaced through JUCE’s getDeviceInfo().

Reproduction

auto* endpoints = juce::universal_midi_packets::Endpoints::getInstance();
const auto availableEndpoints = endpoints->getEndpoints();
for (const auto endpointID : availableEndpoints)
{
    auto endpoint = endpoints->getEndpoint(endpointID);
    auto info = endpoint->getDeviceInfo();

    // info.manufacturer, info.family, info.modelNumber, info.revision
    // are all 00 00 00 on macOS for an endpoint that is actively
    // exchanging correct MIDI-CI Discovery data on the wire.

    logMessage(String::formatted("hasMidi2Support: %d", endpoint->hasMidi2Support()));
    // returns false on macOS for this same endpoint
}

The same code against the same device/firmware on Windows correctly populates info.manufacturer = 0x09 and hasMidi2Support() == true.

Question for the team

Is getDeviceInfo() on macOS reading from a CoreMIDI property that isn’t being populated until some later point in the connection lifecycle, or is this a parsing gap in juce_CoreMidi_mac.mm for the MIDI-CI Discovery Reply / UMP Endpoint Info Notification? Happy to test a patch or add more targeted logging if it’d help narrow this down — I have the full USB capture (CSV) available if useful for cross-referencing against CoreMIDI’s internal behavior.

Thanks for taking a look.

6

On Another note. Some day we need to deal with this unwanted translation issue. This is something we talked about very early in our discussions of HD MIDI that evolved to be UMP as you know it.

I have been using MIDI since the inception, Hence my sysex ID 9. I have always disliked the RPN system. I think it is a shame that someone trys to translate a clean type 4 message back to a pile or RPN bank, index, MSB,LSB stuff.

7

Another update. I just did a firmware update on the MacOs system. I sent over 26,000 Syex8 packets flawlessly. I also see that I receive flexData messages OK. So the UMP works ok on the big stuff. but those pesky translations have got to go

8

I think that last post might be misread. I updated the firmware in my target ESP32 using the sysex8 streaming system to update the 332KB .bin file in my target. The Data flowed from the my JuceApp running on the macOS 26.3.1 to my ESP32 target over USB being monitored by my Beagle analyzer.

9

How have you come to the conclusion that the Mac is relying on MIDI CI rather than UMP endpoint discovery? These are different mechanisms with different purposes. I believe that the Mac should use “UMP Stream” notifications as the source of this data, although I can’t easily check since I don’t have any MIDI 2.0 USB devices to hand at the moment.

Please familiarise yourself with our policy on posting LLM-generated content, and refrain from posting the output of LLM agents directly to the forum.

I don’t know what these hex bytes are supposed to be demonstrating.

A “UMP Type F” packet should have 0xF as its most significant 4 bits, and have 16 bytes total. So, maybe this is supposed to be three separate packets, but none of them start with 0xF, so that can’t be it.

Perhaps this is a MIDI CI payload, but a reply-to-discovery should start 7e 7f 0d 71. If I flip the order of each 4 bytes, then this becomes a sequence of UMP Type 3 messages (7-bit sysex)

30 16 7e 7f   0d 71 02 54 | sysex start with 6 bytes payload
30 26 00 00   00 73 31 7c | sysex continue with 6 bytes payload
etc.

As I said above, endpoint discovery and CI discovery serve different purposes, so I don’t think it’s surprising that information from the CI discovery reply would fail to show up as endpoint info on the JUCE side. It’s probably worth checking which UMP Stream messages macOS is sending (e.g. Endpoint Discovery, Stream Configuration, Function Block Discovery) and making sure that the hardware is replying with the correct notification messages in response.

No, I don’t think so.

Do you have any other UMP devices that you can test? JUCE includes a demo program, the UMPDemo, which displays the available info for connected endpoints. Last time I tried, this was displaying the expected results for an AmeNote ProtoZOA, and for a Roland A88-mk2, both on macOS. As a sanity check, perhaps you could try running the UMPDemo with a known-good UMP device, to make sure that JUCE is behaving properly. If it’s not working properly, it will be much easier for us to reproduce and diagnose the problem if we can do so using JUCE examples and publicly-available hardware (as opposed to private source code and custom hardware).

1 Like