Using ValueTrees to store large-ish text arrays

I want to use ValueTrees to store large-ish amounts of text. By large-ish I mean probably 200-300 characters, so not actually that large, but still probably quite different than the usual case if using ValueTrees to store individual parameters. The idea is that the ValueTree will store a few lines of code which will be interpreted in runtime (probably by JUCE JavaScriptEngine, but we’ll leave that for another time). The system will have to retrieve these lines of code very frequently, so it’s important that the whole process runs smoothly and efficiently.

My question is: if I store a large-ish StringArray in a ValueTree, can I access it quickly and without having to make a copy of it? I am hoping that the var will give me a reference to the StringArray which I can then use to scan through the text quickly. Is this how it works, or would I be forced to make a copy of the StringArray before I can access it?

If accessing the StringArray directly from the ValueTree turns out to be too slow then I am thinking about making some sort of a shared resource which loads all of the text at startup. This feels like it would be pretty complicated, so I’m hoping that I can avoid it.

You will not be given a pointer to StringArray but to Array<var>. You don’t need to make a copy of it. Access to such array is fast but traversing through a tree itself and accessing the appropriate node can take more time. But if done properly it will work ok.

Have you considered using HashMap or std::unordered_map instead of ValueTree?

Thanks for the feedback, @MBO. I am aware that traversing ValueTrees can take time. However, the structure that I’m building is genuinely a tree, so I suspect that ValueTree is still my best option. The idea is that the user will be able to access a particular node by providing a path, for instance “3/4/5” for the fifth leaf of the fourth branch of the third branch. I’ve built a class which interprets these paths and traverses the ValueTree using getChild() without needing to search, so hopefully the access is as fast as a few array look-ups. (Of course it’s slowed down by the fact that I have to parse the String, but that’s another matter which I may be able to streamline in the future).
Does this seem like a well conceived model? I have little prior experience with these sorts of things and have never actually worked with hash maps, so it’s quite possible that I’m missing something big.

If you’re only storing a single data type, and aren’t using any of the fancy ValueTree features like transactions, listeners, etc then you’d probably be better off keeping things simple and just writing your own dumb tree struct, containing a std::vector of child trees + a value.

Actually I am storing multiple data types and use many of the fancy features! ValueTrees are at the heart of the software I’m building and seem tailor made for my needs. It would be pretty hard to talk me out of using them at this point, although if there are limitations for data access, I definitely want to know about them.

Ah well it sounds like you’re doing the right things then!

1 Like

Yes, if you use this model of access it will be fast and you should not worry at all. I have some experience with such approach and huge ValueTrees. The most complicated thing I ever created in JUCE is a client-server database system with full transaction support based on ValueTrees and internally I use very similar way of traversing - of course there is a lot of additional stuff like indexing, filtering, queries, locks, rollbacks etc. but in one word: yes, if looking for a node is implemented properly the rest is surprisingly fast even under heavy load.

1 Like

Returning to the first thing you said, why did you say that I should be using Array<var> instead of StringArray? I can see a var constructor for both, and I would have thought that StringArray would be better suited for what I’m trying to do.

Did you mean that if I use Arrav<var> I could get a pointer to it, but if I used StringArray I would have to make a copy?

I mean that you can create a var variable passing a StringArray object in a constructor but strings will be copied (I assume, did not look into the constructor) to Array<var>. There is only getArray method which returns a pointer to such array but there is no getStringArray. So you do not need to copy your array, you will just have access to Array<var> object with your strings (as if you have Array<String>).

Got it, thanks!