I have used the JUCE library in my C++ project, written functions using the JUCE library, and built a .so file for the Android platform. On the Dart side, I am using FFI to call the functions.
My point is that troubleshooting the problem is done by removing as many variables from the situation as possible. Using just the DemoRunner, which is the simplest project with only JUCE in the picture and no other layers of languages and libraries, can you reproduce the issue?
I believe there are multiple demos in the project that provide pitch change algorithms. A quick search here for pitch shows a sampler, audio synth, etc
As for ResamplingAudioSource being part of a demo somewhere explicitly, you won’t find it. Instead, the AudioPlaybackDemo uses an AudioTransportSource; internally this uses a ResamplingAudioSource when needed. The transport should already be configured to correct the sampling rate for the given audio file, especially when you change the device’s sampling rate. (ie: Change the device’s sampling rate to something other than that of the known file’s sampling rate, load that file up in the proper demo, then play back). Doing this should give you an idea.
In that same demo, you could probably get away with just adding a Slider and another ResamplingAudioSource in series with the transport to then hammer it with pitch changes.
Set a requested callback size in Oboe. Without setting it, you may be getting callback sizes that change almost randomly, and more importantly, can be very short sometimes, causing a lot of overhead.
If your process is working on buffers that need to be filled with samples first before a block of audio can be processed (for example for pitch shifting), you may be missing the callback deadline when your callback size is very small but you need to process a lot of samples in one call . In those cases, see if you can do the heavy processing in a separate thread while your audio callback thread is just collecting and passing on audio samples.
Some phone manufacturers have very aggressive power saving strategies, moving processes across cores dynamically and changing the CPU clock frequencies. Especially if you are doing block-based processing, the first bunch of audio callbacks might be very light on the CPU because you’re still collecting samples for an audio block to process in the future, causing the OS to scale down the CPU. When you then do have to do a lot of work at some point in time because you’ve collected a sufficiently large number of audio samples for your process, your processing call won’t make the deadline because the resources had been scaled down based on your CPU load from the previous frames.
Make sure you run your process at the native sample rate of the device.
It seems Oboe does not guarantee the number of samples you receive in each callback, regardless of the buffer size you set through JUCE’s API. This likely explains the crackling noise you are hearing.
That said, not having a fixed buffer size feels a bit insane to me. Is this just how Oboe works, or am I missing something? Hopefully, someone with more experience on Android can shed some light on this.
Yep, that is what I was referring to in my first bullet point. We have our own fork of JUCE, where one of the changes we made is to request a reasonable and constant callback size in samples using the builder.setFramesPerDataCallback function in juce_Oboe_android.cpp when creating a new stream.
It would be great if this functionality would be part of JUCE itself though; perhaps worth a FR…
@djb-2 Agree. From reading Oboe’s documentation, it seems that not having a fixed block size does provide some performance advantages. However, this makes JUCE behave drastically differently on Android compared to every other platform, and the performance gain is not worth it in the grand scheme of things.
@reukI think the root problem here is a conflict of concepts.
On platforms like iOS, buffer size sets the number of samples you receive in each callback, which is what most people mean when they talk about buffer size.
On Android, Oboe’s buffer size means something entirely different. Here, buffer size refers to the buffer sitting between your program and the hardware, typically configured as burst * 2. It has nothing to do with the number of samples you get in each callback.
These are two completely different concepts, but in JUCE they are both exposed under the same term, bufferSize, across all of juce::AudioIODevice’s APIs. That is really confusing.
Here is what I would suggest:
Do not mix Oboe’s concept of “buffer size” with the rest of JUCE’s API. Hide Oboe’s buffer size from developers.
Surface available buffer sizes as multiples of the burst size that JUCE queries from Oboe (JUCE already does this).
Set Oboe’s buffer size as buffer size * 2.
Call builder.setFramesPerDataCallback and set frames per callback equal to the buffer size. (This is essential, without it, building real-time FFT-based applications becomes nearly impossible.)
I’ve also received reports from some devices that the audio playback sounds distorted or “demonic,” even during normal playback without any effects. Is there any chance this could be fixed? @reuk
Current Setup
What works
What doesn’t
Questions
Any guidance or shared experience would be very helpful.