I want a way for the user to be able to get multiple audio inputs on a single track and be able to mix them directly at the track input (volume, pan, mute parameters). The said inputs may be audio inputs from an audio interface and/or the audio outputs of other tracks in the edit.
After a quick look in the tracktion API, I can see that an audio track may have multiple inputs, however if I’m not mistaken they are summed “implicitly” by summing nodes. Their mix is therefore not controllable by the user. My problem is that I want the user to be able to mix them explicitly for 2 reasons. (1) To avoid clipping at the track input when multiple loud inputs are mixed. And (2) ideally, the mix parameters would be automatable through the automatable parameter API too. Because of (2) I think this feature must be implemented at least partially (if not fully) as a plugin (or a UI that binds to multiple plugins in the audio graph).
Another specification is that if an input device (be it from an audio interface or the output of another audio track) is connected to this input mixer plugin, then it must also be available to be connected simultaneously on another input mixer plugin (to allow for parallel processing).
My initial implementation idea for this feature would be to have an InputRouter backend system that would create on the fly hidden (to the user) input tracks set with the right input set and a hidden Send plugin. Then, when the user wants to set an InputsMixer (let’s call it that) on a track, the said InputsMixer would (1) ask the InputRouter to instantiate the right hidden input tracks, (2) create a hidden Return plugin as the first plugin of the user track and (3) have the UI expose the parameters of the Send plugins of the hidden input tracks.
If an input is used in multiple InputsMixer, then multiple hidden Send plugins would be added to the hidden input track so mix parameters can be controlled independently for each InputsMixer.
This implementation idea seems a bit cumbersome though and may have problems I cannot yet full grasp concerning recording and automatability (because the Send is on a hidden track, it is not clear how automation curves and modifier would work here).
Is this design fundamentally flawed in order to meet these specifications ? Is there a better way to implement the feature through some other API I missed in the tracktion codebase ?
Hi again. I thought of another way to implement this that may actually be better.
The idea is pretty similar, except that instead of having the UI expose the Send parameters of the hidden input tracks, the InputsMixer would actually be a rack and the mix parameters (volume, pan, mute) would actually be VolumeAndPan plugins for each inputs of this rack. This way the automatability stuff stays in the same track. In addition, SendPlugin only has a gain control and no pan or mutes. If we added plugins in series, it would not meet the requirements of allowing the same inputs to be connected to multiple user tracks as VolumeAndPan applies in series.
Is there a better way or that would be the most idiomatic way of doing this feature in tracktion engine ?
This would look like something like this if it would be done manually in waveform (only 2 inputs and 2 user tracks to simplify the problem). Of course it would be automatically managed by the InputRouter to avoid the user going through manual routing.
I’m still unsure about the implications/limitiations about MIDI (which track midi input the rack is receiving ? Is it merging all midi, if yes then I may need something to filter the right midi source), automation (which track automates what) and recording (does recording User track 1 would also automatically trigger recording on User track 1 ?) with such an architecture.
Hi, I tried prototyping this approach and found that:
VolumeAndPan plugins inside the rack may only be automated and modulated through global automation lanes and modifiers (on master track). I can work around that.
There is no way to separate the midi inputs/outputs track per track. This is a problem because then after InputMixer, all midi gets merged… I’m not sure how to tackle this…
Recording works as expected !
Retrospective recordings create clips on the hidden input track (not a big problem).
The midi aspect is the only really blocking aspect of the design and I don’t see a workaround yet… I kind of expected that each track would allow to expose its midi as an individual rack channel but this is unfortunately not the case and quite limiting.
Still open to suggestions for this input mixer plugin if anyone has any !
Sorry, I’m struggling to follow the requirements of this a bit…
I can answer specifics though:
Automation is part of the plugin/parameter so not limited to other tracks. Perhaps you’re talking about something UI specific?
Racks can also have Modifiers inside them.
Different MIDI channels?
These will appear on the tracks that the inputs are assigned to but you can always move them afterwards.
My question regarding things that seems tricky to do is always why are you trying to do this?
It sounds like what you’re describing is an actual mixer, why not just use separate tracks for each input then send these to other tracks?
This is for an embedded product with an integrated audio card.
Because of the CPU constraints, there will be a fixed and limited amount of user tracks (tracks available to the user by opposition to hidden tracks for internal routing) and plugins available such that, in the worst case scenario, we will never hit x runs for a given audio engine configuration.
Because of this limited amount of user tracks, I want a way for the user to “compress” multiple inputs in a single user track to let other user tracks available for other stuff (synths, samplers, mastering, …).
But if there are multiple inputs on a given track, we want some kind of mixing to avoid clipping at the entry point of the user track.
But I also want multiple user tracks to take the same input to allow for parallel processing.
So user tracks may have multiple mixable inputs and a given input may be routed to multiple user tracks.
Inputs may be from the integrated audio card, from an external usb audio card or the output of other tracks.
All of this may be hidden with complex routing but for the UI/UX there must be a kind of input mixer with multiple inputs selection and mixing capabilities on the user tracks.
Of course, the other features of a user track must work as expected (automation, modulation, recording, retrospective recording, midi inputs for CC control of parameters, …).
In particular, the mixing must be automatable and modulatable by automation lanes/clips or modifiers of the user track. Modifiers controlling the mix parameters of the input mixer must also be able to control other plugins in the user track.
Finally, there is a UX requirement being that I do not want the user to have to implement manually all this complex routing. I want the user to simply add an input mixer on a track, select inputs and mix them. The more work I can take away from the user (creating input tracks, routing them to mix tracks and routing the mix tracks to user tracks, …), the easier the interface is and the more fun and intuitive the experience will be. I want to take care of all this complex routing so it is hidden to the user.
Let me know if these requirements still have blind spots and I’ll try to answer you as best as I can.
Based on the requirements above, the reason why is mainly because of the limited amount of user tracks (let’s say 8) compared to the amount of inputs (5 stereo (or 10 mono) by default + more from external usb audio card).
The problem with sends in my tests is that they’re not a recordable input and so if the input mixer is implemented in terms of hidden input tracks with sends and hidden returns on the user track, then the user can’t arm the record and record the whole mix as a single clip. A solution would be to send to hidden “mix” tracks and then connect these to the user tracks. But then there would be 2 levels of indirections: hidden input tracks with sends → hidden mix tracks with returns → user tracks. This is not a problem per se but seems a bit convoluted and I wonder if there could be a better solution.
Then the question is, because user tracks may have multiple inputs, is there a way to mix these retrospective recording according to the mixing setup used internally (be it send volume parameters or dedicated volumeandpan plugins) ?
In terms of implementations, for now I have 2 ideas:
The first one uses hidden input tracks (1 per input) with sends (1 per connection) and hidden mix tracks (1 per user track with an input mixer) with returns. These mix tracks are then set as the input of user tracks with an input mixer. The input mixer UI of a given user track would actually expose the volume parameters of the sends sending to the mix track’s return. (but not clear how to have user track independent panning here… ?)
The second one uses a single rack routing all hidden input tracks (1 per input) to internal volumeandpan plugins (1 per connection between an input track and a mix track) to hidden mix tracks (1 per user tracks with an input mixer). In that case, the input mixer UI of a given user track would expose the volumeandpan plugins connected to this user track.
(I also had another idea to use hidden submix tracks but they don’t have an input device allowing user tracks to record them unfortunately).
In any case, hidden tracks creation/destruction/routing would be automatically done and hidden from the user when they would select an input in the input mixer UI.
These 2 solutions seem a bit convoluted (they both have 2 levels of indirections: hidden input tracks, hidden mix tracks and finally user tracks) and the main limitation is the retrospective recording (only “real” physical inputs have retrospective buffer and not track inputs). But maybe they are the best solutions for my requirements anyway ?
Do you really have to automate the pre-record “mixer” elements of this?
Automation is usually a post recording process as it’s for playback, not usually recording.
I don’t think there’s a “clean” way of doing what you want in TE at the moment, I think we’d need a way to manually specify input busses and also have mixer controls in them. Whilst we have definable input busses on the radar, complex mixing isn’t something we’ve considered and certainly not automating this.
If I was doing this, I’d probably create the mixer as a kind of separate layer and wrap a juce::AudioDevice that actually opens a real device but remaps the channels and applies mixing etc. That way, by the time it’s exposed to TE, it’s exactly the set of channels you want and you wouldn’t have to do any creating of dummy tracks etc.
That seems a relevant objection to my requirements ! I had not considered it. Let’s say we relax this requirement.
I’m not sure what you mean by “separate layer” in this context. Are you saying that the input mixer feature should be some kind of virtual input device that takes other input devices and mix them internally while exposing itself as an input device ?
Still digging through the API to understand how to implement this but I’m thinking: would this solution allow to follow the requirements of being able to use track wave inputs in the virtual input mixer device ?
Not really, you’d have to send the tracks to an output channel and then route that back in your mixer audio device. Like patching an audio cable from your audio device’s output to an input on your physical mixer.
Yup I thought so and this would create a loopback latency I guess. Also I think we would still need to manage hidden tracks to pass the user tracks wave input device to virtual output channels, if we want to keep the exclusive track output to connect to a “real” output at the same time. So this solution comes with tradeoffs I need to take into consideration.
I decided to let go of the “track as input” requirement and just let the user manually set sends and returns for this purpose.
I first looked at implementing mixing and routing directly in a AudioIODevice but this seems quite tricky. Then I found out about the HostedAudioDeviceInterface of tracktion engine and thought it could be useful. Basically I thought of implementing my own AudioIODeviceCallback that would sit between the real devices and the engine. This way I can do the input mixing just before passing the mixed inputs to the engine by calling the HostedAudioDeviceInterface processBlock method. This layer could also be useful later for output mixing as well as any pre or post processing. Does this seem a good idea to you ? Is there any gotchas I should be aware of using the HostedAudioDeviceInterface ? I’m thinking in particular to midi timing and playhead for instance.
Note for the purpose of this discussion that I am using a real time linux kernel with jack as audio backend.
