Is it garbage?!? a Background Multiuser Logger & Multi-Producer-Single-Consumer Fifo

General JUCE discussion
1

In one of my projects I needed to be able to log debug messages from multiple threads and also retain the order that the messages were created so I could debug some multi-threaded code that was doing things in a weird order.

So, I built a Multi-Producer Single-Consumer Fifo so I could send debug messages from all of my threads into a single place for consumption.

Then I built a BackgroundMultiuserLogger that basically wraps that MPSCFifo and simplifies the usage to be as simple as BML::writeToLog(message) and BML::printAllMessages().
It timestamps every message you log, and also sorts them so they never appear out of order in the log file or console.

Again, meant for debugging purposes

It seems to works great for my use-case!
It doesn’t trigger any Thread Sanitizer assertions, or Address Sanitizer assertions.

But is it garbage???

The usage of the MPSCFifo is:

  • call mpscfifo.createProducer() from your thread to get an index of a producer fifo.
  • on your thread: call mpscfifo.add(data, index) to add elements.
  • Depending on how you declare the MPSCFifo, elements will be sorted or unsorted when flushed to the internal ‘consumer’ fifo.
  • It automatically flushes all producer fifos to the internal consumer fifo every 25ms.
  • It has a singular bool pull(T& t) to retrieve elements.

The BackgroundMultiuserLogger works like this:

  • It’s a JUCE singleton, so you use it like this: BML::writeToLog("your message");
  • It uses the threadID of whatever thread you’re calling that function from to see if a Producer index already exists.
  • if a producer index does not exist for the calling thread, one is created. Then the message is added to the MPSCFifo using that index.
  • It has a 25ms timer that pulls all messages from the MPSCFifo’s consumer fifo, and then writes them to a log file, and also optionally prints them to the console.
  • When the BML is destroyed via BML::clearSingletonInstance(), it performs a final flush of all producer fifos and the consumer fifo in the MPSCFifo to the log file, so no messages are omitted.

Check out the code here:

2

It appears that you’ve guarded access to the queues with a mutex. So it looks pretty robust in terms of thread safety.

In theory the locks could cause a priority inversion e.g. in the case that both the real-time thread and the message thread use the logger at the same time. i.e. logging could cause audio-dropouts/stuttering.
But so long as you are aware of that. seems good.

Verdict: not garbage. But don’t brag about it to Timur :wink:

1 Like
3

haha awesome! :joy:

I haven’t tested it with audio threads yet, just with networking stuff.

I bet it would be useful for game engine stuff, though… :thinking:

4

As as a side note, have you checked spdlog ?

5

I haven’t but this is awesome! Thanks for hipping me to it! lots of cool stuff in there to study and learn from!

6

This seems pretty interesting, and useful for the debugging log case you mentioned.
When reading it made me remember the farbot library, which has a FIFO with a lot of options, including a multi-producer single-consumer mode. I’m not sure if it has the same focus on strictly retaining the order from the multiple threads, but might be interesting to compare against?

1 Like