Good stuff Vinnie. I see a lot of generic solutions for problems I’ve had to deal with -AudioBufferPools, cross-thread object lifetimes with defined deletion threads, etc. I would probably have wanted to use them.
I don’t see what you’re doing with lua though. Lubridge implies you’re mapping some classes to lua?
Bruce
Cool thanks. The library is tuned towards everything that an audio plugin or audio host author would want to do (including making cool user interfaces, see LabColour, FreeTypeFaces, and RadialImageConvolutionKernel). It consolidates all of the code that I’ve posted in the Juce forums over the years into a single unified library in the Juce module style.
Some of the external libraries in there are only being used by my private application code. I’ve included these external projects as individual modules so that someone can build a project that uses them in a way that is largely “idiot proof.” I personally despise when I pull sources to an open source project, and then I have to go out and download a bunch of external dependencies. Ever have to deal with downloading and building libCurl, getting the Steinberg headers, getting some ODBC toolkit, etc… yadda yadda? Yuck.
So with these externals I can build code on top of it and distribute the stuff in a way where the average JUCE user or plugin writer doesn’t have to fiddle with downloading and configuring projects (or worse, install and run autotools, yikes!) Sometime soon the externals will be upgraded so you can use existing libraries when it makes sense (like the SQLite that comes with iOS).
As for luabridge, its a pretty nifty wrapper that lets you easily create Lua bindings for your own C++ classes. The author is no longer developing it so I am taking it over and cleaning it up since I use it in my own project.
On the other hand, some externals are needed for classes in VFLib. FreeTypeFaces requires FreeType. Rather than forcing the user to go out and download FreeType and configure include paths, I put the amalgamated version directly into VFLib, as its own module (vf_freetype). So you can get FreeTypeFaces working easily without any hassle.
The entire module vf_db (which is based on soci) provides a robust C++ front end for accessing a SQLite database. There is a vf_sqlite module which has the latest SQLite amalgamated sources included. So to use vf_db requires no additional effort just add the module and go! Now if you’re on iOS, SQLite is built into the operating system and soon I will update my headers so that they use the operating system provided SQLite. vf_db is pretty cool, the following code actually works:
using vf::db;
session sql ("mydatabase.db");
int id = ...;
string name;
int salary;
sl << "select name, salary from persons where id = " << id, into(name), into(salary);I’m also planning on building some goodies on top of the externals. For example, a CompressedHeapBlock that extends the juce::HeapBlock to allow compressing and uncompressing of the data using the included bZip2 module.
VFLib requires that you add a few lines to your existing AppConfig.h, a template for doing so is provided in AppConfigTemplate.
Vincent, you should use @param in the template class declaration so the template parameter is documented.
This is useful to understand the role of each template argument in your library.
Anyway, I’m amazed by this work, and I’m wondering if you’ve written test vector for this code.
I know that writing test vectors for threading code is very boring/difficult, but from my own experience, until you do this, lock free code tends to fail in very specific way.
[quote=“X-Ryl669”]Vincent, you should use @param in the template class declaration so the template parameter is documented.
This is useful to understand the role of each template argument in your library.[/quote]
Yep, I did that for a couple of the classes. The docs still have a way to go (its a work in progress).
Thanks!
Yes, I did. I assume you’re talking about CallQueue and Listeners. They are both rock solid (so is the ReadWriteMutex). The throughput on CallQueue is massive - both of the underlying memory allocators (with and without thread local storage) have been extensively tested, analyzed, and optimized.
Ain’t that the truth! Solving the ABA problem without resorting to hazard pointers or 64-bit CAS was tricky, but it works. Rather well in fact, there’s already a commercial app thats using it.
Can you explain in two lines how you did this ?
In my case, I’ve used deferred deletion with atomic ref counting (I know it’s bad, but still, it works), so that the deletion of object always happens when the “userCount” of the container reaches 0.
I’m interested to figure out a better/faster way.
Can you explain in two lines how you did this ?
In my case, I’ve used deferred deletion with atomic ref counting (I know it’s bad, but still, it works), so that the deletion of object always happens when the “userCount” of the container reaches 0.
I’m interested to figure out a better/faster way.[/quote]
Actually, that’s the same method I’m using. Except that instead of applying it to individual objects, I apply it to entire pages of memory (8 kilobyte blocks if I recall). I designed it specifically for the memory usage pattern of call queues and listeners, which tend to allocate and free in mostly FIFO order. There’s a small percentage of wasted storage but its incredibly fast. When the reference count on a page drops to zero it gets set aside to “cool down” before being garbage collected once per second. The garbage collection executes in O(1).
This all happens in PagedFreeStore. The FIFO allocators build on the paged free store so they inherit the ABA-avoiding properties of the paged free store.
Clever.
I wonder if this scales up when the number of function to call overcome the page size (possible with large object pushed on the stack ?).
While reading your code, I’ve found 2 subtile bugs:
page = new (::malloc (m_pageBytes)) Page (this);
if (!page)
Throw ...
// and twice this:
if (page)
{
page->~page();
::free(page);The former will throw if malloc fails (in placement new), so the second Throw will never be reached.
The latter is exactly equivalent to “delete page;”, except that it’s not exception safe (if the destructor throws, you’ll never free “page”). This is not really a bug, since you wrote the Page’s desctructor, but it’s an optimization.
You could call it a feature / limitation. You’re not supposed to pass big structures by value using the CallQueue. For that, create a ReferenceCountedObject and pass it by ReferenceCountedObjectPtr. I should mention that in the docs.
Something else I should probably mention in the docs. Exceptions thrown in VFLib are fatal, they indicate undefined behavior. There’s no catching the exceptions thrown in the library, except to show an error dialog and exit.
[quote=“X-Ryl669”]While reading your code, I’ve found 2 subtile bugs:
page = new (::malloc (m_pageBytes)) Page (this);
if (!page)
Throw ...
[/quote]
Come to think of it, there probably is a bug in there! Nice find. Of course, that code path has never been tested (because malloc() never fails for me). It figures that the code path that was never tested has a bug (isn’t that always how it goes?). I changed it:
void* storage = ::malloc (m_pageBytes);
if (!storage)
Throw (Error().fail (__FILE__, __LINE__,
TRANS("a memory allocation failed")));
page = new (storage) Page (this);If you were to keep looking through the code and commenting, I certainly wouldn’t complain!
Any assistance with documentation is GREATLY appreciated. I’m working on it now, trying to get everything down. If something is not clear or missing, now is the time to speak up because I would like to just get this over with and done and never come back to the docs again!
If it was the truth…
When documenting the enum value, I’m using “//!< some doc” so each enum value gets documented inline, like this:
/** General enum reason */
enum Some
{
Value1 = 0, //!< The documentation for Value1
Value2 = 1, //!< The doc for val2 etc...
};The documentation is dense in the code already, but it’s only when you’re browsing it online that you realize what’s missing.
My remarks:
Can you point me to one of the enums from http://vinniefalco.github.com/VFLib/ ?
Hmm…you can use either one. When I get done with the docs, only classes that are designed to be used directly should be appearing. Implementation classes and what not should be removed (using the DOXYGEN macro).
I probably have missed a few spots. My strategy is to do quick passes through the docs and refine everything a little at a time. I’m still learning Doxygen.
I don’t understand. There’s three types of groups: modules, members, and pages. Do you mean member function groups? Which classes do you think would benefit from member function groups?
I defined groups for each module so you should be able to see which classes belong to each module, via the “Modules” tab in the top navigation.
The vf_concurrent module specifically provides the synchronization framework based on thread queues. My first pass at describing this framework is in the docs for CallQueue and Listeners. I need to combine all of that into one general discussion of synchronization using thread queues and put it into the vf_concurrent group page, then \ref link to it from each class that participates.
The atomic wrappers are general purpose. They just put a more readable interface around the juce::Atomic<> object. If you already have a concurrent programming framework in place, you might not want to use vf_concurrent but still have the tools in vf_core. That’s why the modules are organized that way.
Error for one.
see @internal. I’m using this instead of preprocessor which I find ugly.
I mean, in Doxygen, when you document a class, the first sentence is a “brief” comment. If you provide one when you declare your group, the “Module” page will show this single line in the table (IIRC)
Yeah great stuff Vinn! Cant wait to try some of these.
Though I would be more likely to use it all if you looked more like a spaceman or were kissing a dolphin, whats up with the stylish photos yall?
Gotta change it up from time to time to keep things fresh!
Vinnie - I just noticed your handy NoiseAudioSource class. You might want to consider implementing the Park-Miller-Carta PRNG for this. (I notice that fellow Jucer Andrew Simper has contributed an optimisation to it too.)
I do love an over-engineered PRNG! I will incorporate that at some point…refinements to the noise audio source are on my todo.
Have you tried connecting LuaBridge shared pointers to ReferenceCountedObjectPtrs?
Bruce
[quote=“Bruce Wheaton”]Have you tried connecting LuaBridge shared pointers to ReferenceCountedObjectPtrs?[/quote] Not yet. Improving luabridge is on my to-do (since I’m now the project maintainer, and I use it in my own projects).
I’m ready to label this as version 1.0 - the docs are complete, although I could always add more exposition in the section on objects for concurrent systems (vf_concurrent).
If anyone wants to have a look at the docs, they are here:
http://vinniefalco.github.com/VFLib
I’m going to give this a few days in case anyone has questions and I might need to update the docs, and then I’m going to label it as v1.0 release.
