Converting audio from a microphone into torch::Tensor

Has anyne done this?
I have Cmake projects which compiles with torch, but I dont know how to connect JUCE audio stream and convert to torch::Tensor so that I can run a model there.

You should give a a lot more information. A few options that could lead to very different best-practice solutions:

  • Do you want to do this in a standalone application that manages the audio I/O device itself or in a plugin inside a DAW? In the first case, should this be a command line or GUI application?
  • Do you want to perform continuous analysis of the stream or perform a one-shot analysis at some time point? In the first case: Do you need to meet realtime requirements running your model continuously?
  • Does your model expect an audio sample buffer containing time domain data or e.g. a spectrum as computed by an FFT?
  • Do you need a certain block-size (I guess you do…)?

And then: Where excactly are you struggling? Did you go through the basic tutorials about how to set up JUCE-based audio applications in general? Show us how your current implementation looks like and where you are stuck. Otherwise this question is way to broad to be easily answered.

Thank you so much for your detailed response.

  1. A GUI application that supports both a VST and s stand alone app, but I can live with a standalone app to start with
  2. Continuous analysis of an audio stream, RT requirements of no more than 1 second delay
  3. Audio sample buffer, I do the rest in the model
  4. A block size of 1/10 of a second

I am struggling with converting AudioBuffer& buffer to and from torch:Tensor.

This always crashes:
torch::Tensor x= torch::from_blob(buffer , size);


Based on its documentation, torch::from_blob expects an array of (or pointer to) “raw” data. juce::AudioBuffer is not the audio data itself. Instead, it wraps the audio data to make it more convenient to work with.

However, juce::AudioBuffer has member functions that give access to the audio data. I’ll let you read the documentation of juce::AudioBuffer and find which function works best in your case.

I hope this helps!