I’m currently working on a project where I need to capture a webcam (cross platform) using OpenCV and display it in a juce::Image. After not finding a satisfactory, quick-to-calculate solution, I had to delve deeper into the topic and would like to share my findings with you in case someone else needs a solution in the future.
As the previously stated, the OpenCV data can be copied via Image::BitmapData and std::memcpy is a relatively fast option to copy the data compared to the double for-loop.
However, the main problem arises when the OpenCV data has to be converted into the correct pixel format, as this will lead to errors, if the memory layout does not match.
The juce pixel formats RGB/ARGB suggest that the layout in memory looks exactly like that, but this is in fact only partially true, because the layout is platform-specific and differs.
For example: the juce::Image pixel memory layout is on a little endian macOS 64bit machine not ARGB but actually BGRA! Knowing that, the image can be correctly copied with following code:
juce::Image img (juce::Image::ARGB, 1920, 1080, true); // ::RGB or ::ARGB make no difference
...
// capture the next frame from the webcam
cv::Mat frame;
camera >> frame;
// get image bitmap
juce::Image::BitmapData imgData (img, 0, 0, img.getWidth(), img.getHeight(), juce::Image::BitmapData::readWrite);
cv::Mat bgra;
cv::cvtColor(frame, bgra, cv::COLOR_BGR2BGRA); // convert to pixel format
std::memcpy(imgData.data, bgra.data, bgra.total() * bgra.elemSize()); // copy to image
As @splintersilk suggested, the cv::mixchannelst is in fact an option, depending on how the conversion must be done. Following all conversions for the different platform/endianness. Please not that I have only tested the non-Android little endian case on a macOS 64bit.
juce::Image img (juce::Image::ARGB, 1920, 1080, true); // ::RGB or ::ARGB make no difference
...
// capture the next frame from the webcam
cv::Mat frame;
camera >> frame;
juce::Image::BitmapData imgData (img, 0, 0, img.getWidth(), img.getHeight(), juce::Image::BitmapData::readWrite);
// Note: depending on system the pixel format convertion might differ // a hint on the layouts can be found in juce_PixelFormats.h @292 ff
#if JUCE_ANDROID
#if JUCE_BIG_ENDIAN
// A = 0, R = 3, G = 2, B = 1; todo: needs verification!
cv::Mat rgba;
cv::cvtColor(frame, rgba, cv::COLOR_BGR2RGBA);
cv::Mat out (rgba.size(), rgba.type());
int from_to[] = { 0,3, 1,2, 2,1, 3,0 };
cv::mixChannels(&rgba, 1, &out, 1, from_to, 4);
std::memcpy(imgData.data, out.data, out.total() * out.elemSize());
#else
// A = 3, R = 0, G = 1, B = 2; todo: needs verification!
cv::Mat rgba;
cv::cvtColor(frame, rgba, cv::COLOR_BGR2RGBA);
std::memcpy(imgData.data, rgba.data, rgba.total() * rgba.elemSize());
#endif
#else
#if JUCE_BIG_ENDIAN
// A = 0, R = 1, G = 2, B = 3; todo: needs verification!
cv::Mat rgba;
cv::cvtColor(frame, rgba, cv::COLOR_BGR2RGBA);
cv::Mat out (rgba.size(), rgba.type());
int from_to[] = { 0,1, 1,2, 2,3, 3,0 };
cv::mixChannels(&rgba, 1, &out, 1, from_to, 4);
std::memcpy(imgData.data, out.data, out.total() * out.elemSize());
#else
// A = 3, R = 2, G = 1, B = 0
cv::Mat bgra;
cv::cvtColor(frame, bgra, cv::COLOR_BGR2BGRA);
std::memcpy(imgData.data, bgra.data, bgra.total() * bgra.elemSize());
#endif
#endif
Hope the helps, cheers Julian.
It’s been a while since I was looking at this but great to know of alternative and likely better ways of handling the issue than my suggestion.