Many thanks! But I get an error message when compiling:
You must define JUCE_MODAL_LOOPS_PERMITTED=1 to use Tracktion Engine
I’m sorry, I must be doing some basic mistake…
I added
#define JUCE_MODAL_LOOPS_PERMITTED 1
to AppConfig.h. Now the code compiles but instead I get a linker error:
Undefined symbols for architecture x86_64:
"tracktion_graph::getPoolCreatorFunction(tracktion_graph::ThreadPoolStrategy)", referenced from:
tracktion_engine::NodeRenderContext::NodeRenderContext(tracktion_engine::Renderer::RenderTask&, tracktion_engine::Renderer::Parameters&, std::__1::unique_ptr<tracktion_graph::Node, std::__1::default_delete<tracktion_graph::Node> >, std::__1::unique_ptr<tracktion_graph::PlayHead, std::__1::default_delete<tracktion_graph::PlayHead> >, std::__1::unique_ptr<tracktion_graph::PlayHeadState, std::__1::default_delete<tracktion_graph::PlayHeadState> >, std::__1::unique_ptr<tracktion_engine::ProcessState, std::__1::default_delete<tracktion_engine::ProcessState> >, juce::AudioFormatWriter::ThreadedWriter::IncomingDataReceiver*) in include_tracktion_engine_playback.o
tracktion_engine::NodeRenderContext::renderMidi(tracktion_engine::Renderer::RenderTask&, tracktion_engine::Renderer::Parameters&, std::__1::unique_ptr<tracktion_graph::Node, std::__1::default_delete<tracktion_graph::Node> >, std::__1::unique_ptr<tracktion_graph::PlayHead, std::__1::default_delete<tracktion_graph::PlayHead> >, std::__1::unique_ptr<tracktion_graph::PlayHeadState, std::__1::default_delete<tracktion_graph::PlayHeadState> >, std::__1::unique_ptr<tracktion_engine::ProcessState, std::__1::default_delete<tracktion_engine::ProcessState> >, std::__1::atomic<float>&) in include_tracktion_engine_playback.o
tracktion_engine::EditPlaybackContext::NodePlaybackContext::NodePlaybackContext(unsigned long, unsigned long) in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::prepareToPlay(double, int)", referenced from:
tracktion_engine::TracktionNodePlayer::prepareToPlay(double, int) in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::setNumThreads(unsigned long)", referenced from:
tracktion_engine::TracktionNodePlayer::setNumThreads(unsigned long) in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::enablePooledMemoryAllocations(bool)", referenced from:
tracktion_engine::TracktionNodePlayer::enablePooledMemoryAllocations(bool) in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::process(tracktion_graph::Node::ProcessContext const&)", referenced from:
tracktion_engine::TracktionNodePlayer::process(tracktion_graph::Node::ProcessContext const&) in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::setNode(std::__1::unique_ptr<tracktion_graph::Node, std::__1::default_delete<tracktion_graph::Node> >, double, int)", referenced from:
tracktion_engine::TracktionNodePlayer::setNode(std::__1::unique_ptr<tracktion_graph::Node, std::__1::default_delete<tracktion_graph::Node> >, double, int) in include_tracktion_engine_playback.o
tracktion_engine::TracktionNodePlayer::TracktionNodePlayer(std::__1::unique_ptr<tracktion_graph::Node, std::__1::default_delete<tracktion_graph::Node> >, tracktion_engine::ProcessState&, double, int, std::__1::function<std::__1::unique_ptr<tracktion_graph::LockFreeMultiThreadedNodePlayer::ThreadPool, std::__1::default_delete<tracktion_graph::LockFreeMultiThreadedNodePlayer::ThreadPool> > (tracktion_graph::LockFreeMultiThreadedNodePlayer&)>) in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::clearNode()", referenced from:
tracktion_engine::TracktionNodePlayer::clearNode() in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::LockFreeMultiThreadedNodePlayer(std::__1::function<std::__1::unique_ptr<tracktion_graph::LockFreeMultiThreadedNodePlayer::ThreadPool, std::__1::default_delete<tracktion_graph::LockFreeMultiThreadedNodePlayer::ThreadPool> > (tracktion_graph::LockFreeMultiThreadedNodePlayer&)>)", referenced from:
tracktion_engine::TracktionNodePlayer::TracktionNodePlayer(tracktion_engine::ProcessState&, std::__1::function<std::__1::unique_ptr<tracktion_graph::LockFreeMultiThreadedNodePlayer::ThreadPool, std::__1::default_delete<tracktion_graph::LockFreeMultiThreadedNodePlayer::ThreadPool> > (tracktion_graph::LockFreeMultiThreadedNodePlayer&)>) in include_tracktion_engine_playback.o
"tracktion_graph::LockFreeMultiThreadedNodePlayer::~LockFreeMultiThreadedNodePlayer()", referenced from:
tracktion_engine::TracktionNodePlayer::~TracktionNodePlayer() in include_tracktion_engine_playback.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Oh sorry, you’ll also need to add the tracktion_graph module to your Projucer (or CMake) project now.
I’ll add some messaging to communicate this now.
This is one reason why it’s best for synths to be robust enough to handle the MIDI SYSEX tuning command consistently no matter if it’s ‘before’ or ‘after’ other events with the same timestamp in the event list.
This is one reason why it’s best for synths to be robust enough to handle the MIDI SYSEX tuning command consistently no matter if it’s ‘before’ or ‘after’ other events with the same timestamp in the event list.
I absolutetly agree that synths should preferably handle continuous tuning messages! However the sorting issue is bigger than that: consider e.g. a program change message. If a program change is placed right before a noteOn, it is essential that it is actually executed before the noteOn or the note will be played with the wrong sound!
yeah, my synths treat the following as the exact same thing. (and the same for micro-tuning)
time event
0 note-on
0 program-change
time event
0 program-change
0 note-on
I have now tested the new MidiList conversion and it works! Great!
There is one thing though: I have a couple of Midi Controller messages at position 0. When these arrive to the synth plugin (in PluginRenderContext.bufferForMidiMessages) their order has been reversed. This is not a problem for single messages such as program changes but it is a problem for RPN messages which consist of multiple MIDI messages.
Hmm, good point. I think that might be the result of an experiment I did to improve MIDI processing time.
Can you take a look at tracktion_graph_SummingNode.h and replace the contents of sortByTimestampUnstable with:
messages.sortByTimestamp().
Does that keep the events in order? I didn’t think about RPN messages.
There’s still the case where you multiple MIDI streams being merged but I don’t think that would interrupt RPNs as they’d come from the same source (although a message might get inserted in-between I guess…).
No, unfortunately not: sortByTimestampUnstable is not called at all, apparently because nodesWithMidi is not greater than 1 in processSinglePrecision/processDoublePrecision.
But logging in processSinglePrecision shows that the order is already reversed in inputFromNode.midi.
Are all MIDI messages with the same timestamp reversed or is it random ones?
I’m looking in MidiNode::processSection, but that looks like it iterates forwards to me.
Does the juce::MidiMessageSequence passed to its constructor have the events in the correct order?
Are all MIDI messages with the same timestamp reversed or is it random ones?
It looks like all messages with the same timestamp are in strictly reversed order.
(One triple of RPN messages also seem to have disappeared completely!? But I haven’t dug into that so I wouldn’t put to much weight on it until I have it confirmed)
Does the
juce::MidiMessageSequencepassed to its constructor have the events in the correct order?
Yes! A bank select seems to have been automatically inserted into the sequence at some point but otherwise the sequence is identical to the one I used when creating the clip.
Does this happen during playback if it’s started before the CC messages or only if you start playback after the CC messages?
I’ve noticed that juce::MidiMessageSequence::createControllerUpdatesForTime which is called to set CC values on playback start that happen before the start position actually adds the messages in reverse.
You could test this by adding this on line 228 of tracktion_MidiNode.cpp:
std::reverse (controllerMessagesScratchBuffer.begin(), controllerMessagesScratchBuffer.end());
If that does solve your problem it would be good to know if if it’s elsewhere.
If it is that, I think the juce method should be fixed as it will break RPNs as you’ve highlighted.
Does this happen during playback if it’s started before the CC messages or only if you start playback after the CC messages?
Ah, you are right: it only happens if playback is started after (or exactly on) the CC messages!
You could test this by adding this on line 228 of
tracktion_MidiNode.cpp
I added it in the else clause on line 239 since useMPEChannelMode is not set in my case. Unfortunately this makes all CC messages disappears; they never reach processSinglePrecision. My guess is that the reversed order messes things up (such as the check if (meh->message.getTimeStamp() >= time) on line 260) but I haven’t confirmed that.
But I still think your observation is correct and makes sense: createControllerUpdatesForTime is the culprit. I tested this by changing the loop to go from 0 and upwards instead of backwards and it indeed solves the issue.
HOWEVER:
I believe the reason for going backwards through the MIDI events is that only the last controller of each kind should be collected. For example, after the first program change is found, doneProg is set to true and further (i.e. earlier) program changes are ignored. Therefore, changing the loop direction would not work. Instead, one could push the controller events to a intermediate list and then add them in reversed order to dest.
But I think this also explains the missing RPN message I mentioned earlier: as all RPN messages share the same controller number only one set of RPN messages — the last one — will be used, if there are multiple RPN messages the earlier will be discarded! (The same goes for NRPN messages)
Unless I am completely mistaken this is clearly a bug! RPN/NRPN messages have to be grouped by their RPN/NRPN number but that could prove to be a relatively complex task in order to get Data Entry messages (and Data Increment etc) correct. An easier workaround would be to just keep all RPN, NRPN and Data Entry CC messages, that should at least guarantee correct behavior.
So that does sound like it’s the juce::MidiMessageSequence::createControllerUpdatesForTime which is the problem.
Yes, you’re right it does go backwards for a reason.
Don’t RPN/NRPM messages come in CC pairs, one for the LSB, one for the MSB?
Even so I think the order will still be incorrect as the MSB will be first when it should be last?
I don’t think you can keep all RPN/NRPN messages as that could potentially dump a huge number to the bus every time you stop/started playback.
I do think this seems like a juce issue though and not something I can change directly in the Engine. Unless you think otherwise?
Don’t RPN/NRPM messages come in CC pairs, one for the LSB, one for the MSB?
I think rather in groups of three or four (see e.g. MIDI NRPN and RPN by Phil Rees): the LSB and MSB define the parameter number and then follows one or two messages with the actual value to use.
I don’t think you can keep all RPN/NRPN messages as that could potentially dump a huge number to the bus every time you stop/started playback.
True. (For me personally I could patch JUCE to do that as I know beforehand that my MIDI clips will only ever contain a very few CC messages but you are probably right that i wouldn’t work as a general solution)
I do think this seems like a juce issue though and not something I can change directly in the Engine. Unless you think otherwise?
Yes, I agree. I will report it to JUCE. Would you recommend filing an issue on JUCE’s GitHub or is it better to take it to General JUCE discussion here on the forum?
For this thread it’s case closed, I guess! Thanks again for all your help!
For reference, this has now been reported here:
1 Like
Hi there. Im a newbie in Juce, but long time musician and microtonal system power user 
Im loving following up in your discussion and looking forward to see where it leads. I’ve seen cute approaches and over complicated ones over the years.
One thing to consider is the already existing mpe pitch info per note.
Given a polyphonic synth that implements MPE, it’s quite trivial to apply the offsets +/- 48 for instance.
So if your host implements mpe on the clips, you only need the interfacing with that such as create tuning files, spread non repeating octaves, naming etc maybe.
Check the code base at Helio maybe https://github.com/helio-fm/helio-workstation
Apparently supports microtonality. I’ve tested it really briefly and wrongly, while compiling audio software for my pi3
A follow-up on SysEx handling (two years later…)
I made a simple patch to juce::MidiMessage that makes it possible to choose a larger pre-allocated space for MIDI messages. The official implementation reserves sizeof(uint8*) (8 bytes on 64-bit systems), but increasing it to e.g. 16 bytes means that single note tuning SysEx messages can be used without any heap allocation.
The price for this is an increased overall size of MidiMessage, from 24 to 32 bytes, and that memcpy (for 16 bytes) has to be used internally instead of a simple pointer assignment.
The default is to stick with sizeof(uint8*), in which case the patch should be completely transparent.
Please feel free to contact me should you be interested in the code!
2 Likes