Converting audio from a microphone into torch::Tensor

Audio Plugins
1

Hello,
Has anyne done this?
I have Cmake projects which compiles with torch, but I dont know how to connect JUCE audio stream and convert to torch::Tensor so that I can run a model there.

2

You should give a a lot more information. A few options that could lead to very different best-practice solutions:

  • Do you want to do this in a standalone application that manages the audio I/O device itself or in a plugin inside a DAW? In the first case, should this be a command line or GUI application?
  • Do you want to perform continuous analysis of the stream or perform a one-shot analysis at some time point? In the first case: Do you need to meet realtime requirements running your model continuously?
  • Does your model expect an audio sample buffer containing time domain data or e.g. a spectrum as computed by an FFT?
  • Do you need a certain block-size (I guess you do…)?

And then: Where excactly are you struggling? Did you go through the basic tutorials about how to set up JUCE-based audio applications in general? Show us how your current implementation looks like and where you are stuck. Otherwise this question is way to broad to be easily answered.

3

Thank you so much for your detailed response.

  1. A GUI application that supports both a VST and s stand alone app, but I can live with a standalone app to start with
  2. Continuous analysis of an audio stream, RT requirements of no more than 1 second delay
  3. Audio sample buffer, I do the rest in the model
  4. A block size of 1/10 of a second

I am struggling with converting AudioBuffer& buffer to and from torch:Tensor.

This always crashes:
torch::Tensor x= torch::from_blob(buffer , size);

Thanks,

4

Based on its documentation, torch::from_blob expects an array of (or pointer to) “raw” data. juce::AudioBuffer is not the audio data itself. Instead, it wraps the audio data to make it more convenient to work with.

However, juce::AudioBuffer has member functions that give access to the audio data. I’ll let you read the documentation of juce::AudioBuffer and find which function works best in your case.

I hope this helps!