Multidimensional array of vars?

General JUCE discussion
#1

I noticed that var can contain an array of vars. Is it possible to make this as multidimensional array of array or array of vars as I like and it still saves into and loads from file without any extra code at all?

Or would it be a better idea to use ValueTree with multiple properties, each of them being a one dimensional array of vars?

Basically I’m thinking of converting the following into using arrays of vars:
std::vector<std::vector<std::vector<uint8_t>>>

Only the innermost vector size will change during the lifetime of the software. The two outermost vectors are always the same size, no matter what.

#2

if the outter ones are always the same size you might create clearer readable code by using std::array instead. that doesn’t answer your question but maybe smth to think about

#3

What about if I want to save that into the file and load it from there so that I don’t need to do any manual work to create those std::arrays? I.e. these would be used with ValueTrees.

#4

The documentation and tutorials aren’t clear on what actually happens when you get a property from ValueTree. ValueTree::getProperty returns a var, but not a pointer to it.

I want to be using juce::Array type as the var held in ValueTree. Does this mean that every time I want to access that array from ValueTree, even to modify a single int from it, the whole array will be cloned into another var, which I then modify. Then the ValueTree::setProperty() copies the whole juce::Array (which is inside the var) back into the ValueTree?

Am I missing something obvious here?

#5

AFAIK and at first view, the var would be just a wrapper around a reference counted array (thus pass-by-value doesn’t imply a major cost).

< JUCE/juce_Variant.cpp at master · juce-framework/JUCE · GitHub >

Most of time in all the juce::ValueTree machinery, stuff are lightweight references to shared objects.

#6

juce::ValueTree stores its properties as juce::vars so you can use any type (unlike juce::XmlElement for example that only takes strings).

juce::var though doesn’t depend on or know anything about juce::ValueTree. When you call juce::ValueTree::getProperty() you’re getting a copy of that property. You can use juce::ValueTree::getPropertyPointer() to get a pointer to the juce::var object. Although a better approach is to use a juce::Value or juce::CachedValue to refer to a property in a tree.

juce::var can hold a juce::ReferenceCountedObject* in which case the underlying object will be shared as you’re getting a copy of the pointer, not the object itself.

To answer your original question, there’s nothing stopping you from having a multidimensional juce::var. When acting as an array, a juce::var uses a juce::Array<juce::var>, so each of the values could also be arrays.

#7

If you know the type in your container then don’t use ValueTree nor var. It will be much slower than it needs to be.

Ideally I would create a struct like:

template<typename T>
struct Matrix3D
{
    Matrix3D (int x, int y, int z)
    {
        setDimensions (x, y, z);
    }

    void setDimensions (int x, int y, int z)
    {
        width =x;
        height = y;
        depth = z;
        storage.resize (width * height * depth);
    }

    T& at (int x, int y, int z)
    {
        return storage.at (getIndex (x, y, z));
    }

    void set (const T& item, int x, int y, int z)
    {
        storage [getIndex (x, y, z)] = item;
    }

    int width = 0;
    int height = 0;
    int depth = 0;
private:
    int getIndex (int x, int y, int z)
    {
        return ((z * height + y) * width) + x;
    }
    std::vector<T> storage;
};

Just as inspiration. You can add all kind of operators now to your new Matrix3D class…

And you use it like:

Matrix3D<uint8_t> matrix (255, 100, 32); // or whatever dimensions
#8

The main reason I’m hoping to do this is to:

  1. Easily save and load a ValueTree into a file with that multidimensional matrix in it.
  2. To later be able to easily add Undo feature into editing that data.

If the var won’t get copied/cloned when I access it, then it’s probably the best approach to simply use Array as the var type and put several of those as properties inside a ValueTree? I mean that each Array would represent a 2D table and I would have X amount of them in one ValueTree.

#9

I never used it that way, but I’m almost sure that with…

const var* ValueTree::getPropertyPointer (const Identifier& name) const;
Array<var>* var::getArray() const;

… you can fetch your data without any copy/clone.

But such way to store a Matrix is not very efficient if you need fast computation. I mean it is far from a fashionable DOD approach! What’s your data? What’s the size of?

#10

The data I have is basically musical note data and it’s related parameters. That matrix of data will then get preprocessed into another buffer for realtime playing purposes whenever the data changes. So the main concerns for me are the ability to automatically and easily load/save data between a file and ValueTree. Also the upcoming Undo feature would work automatically if I keep all that data in ValueTree friendly data structures.

Here’s my current thought of how I could make it to ensure the var will always live inside the ValueTree and that it actually gets initialised. All comments are welcome:

    juce::ValueTree track_node(some_node_type);

    for (int column = 0; column < COLUMN_COUNT; ++column)
    {
        track_node.setProperty(column_id_list[column], 0xFF, nullptr);  // Adds a single int so it is guaranteed to live in the ValueTree
        juce::var column_array = track_node.getProperty(column_id_list[column], 0);
        jassert(int(column_array) == 0xFF);  // Make sure everything worked right so far
        
        column_array.resize(m_rows_per_pattern);    // Resize the array to the preferred size
        for (int i = 0; i < m_rows_per_pattern; ++i)    // Fill the array with preferred values
            column_array[i] = 0xFF;
    }
#11

The max size of ints in the matrix is 88 x 512. I have many of those matrices as they’re basically music note (MIDI) clips which get preprocessed into actual realtime play buffers every time they get edited.

The DOD approach for this part of my code went out the window once I hoped to make the load/save and upcoming Undo feature as easy as possible using ValueTrees. Hence I rely more on the preprocessing, which I need to do anyway and has been done a long time ago already. So this approach would require minimal changes to the code base.

#12

I’ll see if I should just use Values then. I’ll experiment with that idea and see what comes out of it.

#13

It seems that anything that returns a var is either const var reference/pointer or pure var. Which means you can’t write into them. You’re allowed to only read them. This include ValueTree’s getProperty() and getPropertyPointer(). That means I can’t use them for my purposes since the data needs to live in the ValueTree instead of getting cloned/copied outside of it every time I want to write to it.

Value::getValue() returns plain var, which again doesn’t point to the original data which would be living in the ValueTree. So that can’t be used either.

So my final hope is CachedValue if I understand correctly. Can CachedValue hold a var which is an array of integers? So I could access the array like:

CachedValue my_array;
my_array[ n ] = 10;

If that doesn’t work, I’m probably forced to use ReferenceCountedObject for the job to encapsulate all my data and then write a custom serializer method to read and write the files.

Or one more idea that might work if it’s even possible to do is this:
How to have an 1D array of integers living inside a single property of a ValueTree?
I must be able to access and modify those integers.
If that’s possible, then I can have X amount of properties, each containing their own 1D array of integers.

Is that possible?

#14

I don’t think there is any way at all to do what I’m after. Here’s closest I got to implementing what I want and still I ran into the problem of having to actually copy the whole var (and thus the whole array) from one place to another in the memory:

juce::ValueTree test_node(test_type);
test_node.setProperty(test_id, 2.4, nullptr);

juce::CachedValue<juce::var> test_value;
test_value.referTo(test_node, test_id, nullptr);
test_value->resize(64);

test_value[10] = 1.0;   // Can't be done, there is no way around it nor any method to do this

So I’m basically left with the choice that I have to accept that var’s containing arrays (in ValueTrees) need to be copied. There is no way around it. You always have to access the whole array at once, instead of being able to access just a single element of it. I guess it’s no biggie. Just thinking that the undo feature might be a bit of a memory hog, but that’ll work for my purposes.

#15

If you were to write directly into the var, you would bypass the UndoManager. The CachedValue keeps a pointer to the UndoManager, that way a change would be registered.
The second reason why you don’t get a reference is, because the property might not exist.

I would still recommend not to use ValueTree as storage, because it creates quite some overhead.
And accessing properties by name is expensive as well.

You can have the UndoManager for your bespoke struct as well. Just implement an UndoableAction for setting and for resizing.

#16

OK, I’ll check how that would fit my project.

#17

If I do what you suggest, what methods do I need to implement to get ValueTree save my custom class into file and load it from there? The class is inherited from ReferenceCountedObject.

By this I mean that ReferenceCountedObject seems to be inherited from var class, which in turn has getBinaryData() and writeToStream(), but neither of them are virtual methods so I’m not sure what’s going on there.

EDIT:
I found an old thread asking a similar question and what Jules answered to it:

Jules said that he does things so that ValueTree’s don’t have any ReferenceCountedObjects in them, but plain old data which can be saved directly to a file. This sounds like it would create an issue with the Undo functionality later on, if my own class with custom data isn’t part of that tree.

EDIT:
Maybe ValueTreeSynchroniser could be used to achieve this? If I would use std::vectors for the data internally inside my ReferenceCountedObject and var (Array of ints) in the ValueTree. Then use ValueTreeSynchroniser to update any changes from ValueTree to that ReferenceCountedObject…

#18

Below a quick and simple example to get/set an Array<var> in a ValueTree.

juce::ValueTree tree ("Foo");
    
const int n[] = { 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 };
    
tree.setProperty ("Bar", juce::var (juce::Array<juce::var> (n, juce::numElementsInArray (n))), nullptr);
    
const juce::var v = tree.getProperty ("Bar");
    
for (int i = 0; i < v.getArray()->size(); ++i) { DBG (v.getArray()->getUnchecked (i).toString()); }

Note that you can use std::initializer_list instead.

juce::Array<juce::var> n = { 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 };
    
tree.setProperty ("Bar", juce::var (std::move (n)), nullptr);

You can also change the values.

for (int i = 0; i < v.getArray()->size(); ++i) { v.getArray()->setUnchecked (i, 42); }
    
for (int i = 0; i < v.getArray()->size(); ++i) { DBG (v.getArray()->getUnchecked (i).toString()); }

And with range-based for loop to iterate over the array.

juce::ValueTree tree ("Foo");
    
tree.setProperty ("Bar", juce::var (juce::Array<juce::var> ({ 9, 8, 7, 6, 5, 4, 3, 2, 1, 0 })), nullptr);
    
const juce::var v = tree.getProperty ("Bar");   /* Cached to guarantee the array lifetime. */
    
for (const auto& t : *v.getArray()) { DBG (t.toString()); }
#19

Seems to work. Thank you!

1 Like