Jassert and Google Test EXPECT_DEATH

1

I want to use Google Test’s EXPECT_DEATH macros to run tests on code that involves JUCE jassert.

JUCE jassert doesn’t work…

TEST(MyDeathTest, jassertTry) {
  ASSERT_DEBUG_DEATH (jassert(1==0), "");
}

[ RUN      ] MyDeathTest.jassertTry
Failure
Death test: do { if (! (1==0)) do { juce::logAssertion ("PluginProcessor.cpp", 183);; if (juce::juce_isRunningUnderDebugger()) { asm ("int $3"); }; juce_assert_noreturn(); } while (false); } while (false)
    Result: failed to die.
 Error msg:
[  DEATH   ] JUCE Assertion failure in PluginProcessor.cpp:183
[  DEATH   ] 
[  FAILED  ] MyDeathTest.jassertTry (6 ms)

Regular assert does work…

TEST(MyDeathTest, assertTry) {
  ASSERT_DEBUG_DEATH (assert(1==0), "");
}

[ RUN      ] MyDeathTest.assertTry
[       OK ] MyDeathTest.assertTry (865 ms)

Any way to get this to work?

Thanks in advance, John.

P.s. I’ve tried nearly every combination of macros and flags as per here: https://github.com/google/googletest/blob/master/googletest/docs/AdvancedGuide.md#how-to-write-a-death-test

2

Or even better altogether would be some kind of a JUCE

expectAssert (...);
expectDoesNotAssert (...);

(to complement the expectThrows/DoesNotThrow)

Would this be possible?

The reason I turned to Google Test was because this kind of functionality wasn’t in the JUCE unit tests.

3

Hmm. Not sure how that could be implemented without defining jassert in some other way which is aware of unit-tests, but that’d impact its behaviour in normal circumstances. Can anyone think of a good trick for doing this?

4

Bit hacky I know, but what if at the beginning of expectAssert() and expectDoesNotAssert() you set the Logger to write to some string, you could then read it to find if any lines start with "JUCE Assertion failure in ", at the end of the method it could set the logger back and print the messages stored back into the original logger? You could probably make an object some sort of ScopedLogger maybe that could handle a lot of the work.

5

OK the following works for me as long as you enable JUCE_LOG_ASSERTIONS, if you don’t then you should hit static asserts at compilation time. I suspect if some effort is put into either highjacking the stderr stream or attempting to read it in some way, a solution could be implemented for when JUCE_LOG_ASSERTIONS is not enabled. I’ve only tested this on Mac but I assume Linux and Windows should work too, however due to the for loop it’s C++11 onwards only (just change the syntax to be a normal for loop and it should backwards compatible).

class ScopedLogger : public Logger
{
public:
    ScopedLogger()
        : logger (Logger::getCurrentLogger())
    {
        Logger::setCurrentLogger (this);
    }
    
    ~ScopedLogger()
    {
        Logger::setCurrentLogger (logger);
        
        for (String message : messages)
        {
            Logger::writeToLog (message);
        }
    }
    
    bool containsMessageStartingWith (StringRef text)
    {
        for (String message : messages)
        {
            if (message.startsWith (text))
                return true;
        }
        
        return false;
    }
    
    void logMessage (const String& message) final
    {
        messages.add (message);
    }
private:
    Logger* logger;
    StringArray messages;
};

#if JUCE_LOG_ASSERTIONS
    #define expectJassert(expr)                                                                 \
        {                                                                                       \
            ScopedLogger scopedLogger;                                                          \
            {expr;}                                                                             \
            expect (scopedLogger.containsMessageStartingWith ("JUCE Assertion failure in "),    \
                    "Expected: does jassert, Actual: does not jassert.");                       \
        }

    #define expectJassertIn(expr, file)                                                             \
        {                                                                                           \
            ScopedLogger scopedLogger;                                                              \
            {expr;}                                                                                 \
            expect (scopedLogger.containsMessageStartingWith ("JUCE Assertion failure in " file),   \
            "Expected: does jassert, Actual: does not jassert.");                                   \
        }

    #define expectDoesNotJassert(expr)                                                          \
        {                                                                                       \
            ScopedLogger scopedLogger;                                                          \
            {expr;}                                                                             \
            expect ( ! scopedLogger.containsMessageStartingWith ("JUCE Assertion failure in "), \
                    "Expected: does not jassert, Actual: does jassert.");                       \
        }

    #define expectDoesNotJassertIn(expr, file)                                                          \
        {                                                                                               \
            ScopedLogger scopedLogger;                                                                  \
            {expr;}                                                                                     \
            expect ( ! scopedLogger.containsMessageStartingWith ("JUCE Assertion failure in " file),    \
            "Expected: does not jassert, Actual: does jassert.");                                       \
        }
#else
    #define expectJassert(expr) \
        static_jassert (false)

    #define expectDoesNotJassert(expr) \
        static_jassert (false)
#endif


class JassertUnitTest : public UnitTest
{
public:
    JassertUnitTest()  : UnitTest ("jassert testing") {}
    
    void runTest() override
    {
        beginTest ("jassert tests");
        
        expectJassert (jassertfalse);
        expectJassert (jassert (true)); // should fail
        expectJassertIn (jassertfalse, "Main.cpp");
        expectJassertIn (jassert (true), "Main.cpp"); // should fail
        expectDoesNotJassert (jassert (true));
        expectDoesNotJassert (jassertfalse); // should fail
        expectDoesNotJassertIn (jassert (true), "Main.cpp");
        expectDoesNotJassertIn (jassertfalse, "Main.cpp"); // should fail
        expectDoesNotJassertIn (jassertfalse, "Other.cpp");
    }
};

static JassertUnitTest staticJassertUnitTest;

//==============================================================================
int main (int argc, char* argv[])
{
    UnitTestRunner runner;
    runner.runAllTests();
}
6

Awesome I will look into this, thanks for taking the time! John.

7

I’ve only just started putting bits up online but I started creating a new set of JUCE modules recently that are available on github and I added this to it if that helps?

just add the jucey_core module and define JUCE_LOG_ASSERTIONS if you intend to use any of the macros, which include…

expectMessageLogged
expectMessageNotLogged
expectMessageLoggedStartingWith
expectMessageNotLoggedStartingWith
expectJassert
expectJassertIn
expectDoesNotJassert
expectDoesNotJassertIn

Also if you are running under a debugger you may like to define JUCE_DISABLE_ASSERTIONS to prevent the debugger breaking for each jassert.

8

Redefining jassert was my original ‘solution’ created before Anthony’s suggestion.

But as you say it neuters jassert to break at unhelpful points.

Works with a preprocessor flag that switches in/out unit tests MY_UNIT_TESTS which can be wrapped around any

//==============================================================================
/** Special version of jassert that throws exception instead of assertion when
    unit tests are switched on with the MY_UNIT_TESTS=1 preprocessor flag.
    
    Note: this won't break at the proper point during debugging. If your hitting
    an assert unrelated to the unit tests, try set MY_UNIT_TESTS=0
    
    This will trigger warning:
    "Lexical or Preprocessor Issue 'jassert' macro redefined"

    @see jassert
*/
#if MY_UNIT_TESTS
 #define jassert(expression) JUCE_BLOCK_WITH_FORCED_SEMICOLON (if (! (expression)) throw std::logic_error ("");)
#endif
9

Great Anthony I’ll look into this.

Do you think yours would be much cleaner than my attempt above?

10

No not at all seems like a reasonable solution to me, the only minor downsides I see are that the unit test will presumably use expectThrows() and expectDoesNotThrow(), which I think makes the intention less clear when reading the unit tests, and also an exception of another type or even another logic_error could be thrown that is not due to a jassert. You could maybe make a custom exception type, then make the expectJassert be defined to call expectThrowsType(). The only other minor advantage with what I done is that a staticAssert will be hit if you try to use the macros without defining JUCE_LOG_ASSERTIONS meaning you catch misuse at compilation time, but I suspect something similar could be implemented with what you have too.

11

I’ve realised an issue with your implementation, you want jassertfalse to be redefined to throw instead of jassert, otherwise someone could still use jassertfalse and it won’t be picked up.

Also I think I might have an idea that trumps both of our initial ideas I’ll have a play tonight and post my findings.

12

Another thing to consider when redefining any of the jassert macros is that it will need to happen in juce_PlatformDefs.h or some code will be compiled with it defined in one way and other code defined differently, you can’t just define it before in AppConfig.h either as juce_PlatformDefs.h will just redefine it, a simple change for the JUCE team could help out here however.

So I’ve just tried a simple / crude implementation that requires changes in juce but importantly doesn’t require any additional macros to be defined in a project for it to work.

Create a global int to track the number of jasserts, then increment the number for every jassert, set it to 0 in expectJassert and expectDoesNotJassert, execute the code and test the global variable to see if it has incremented or not.

Pros:

  • No need for any macro definitions to make it work.
  • Very simple.

Cons:

  • It would mean every case of a jassert in the code would in ALL build configurations increment this global when ideally in release builds it would be preferable that the jassert does nothing.
  • Not thread safe.
  • Requires changes to JUCE.
13

I should probably add to the Cons that it requires a global variable which is normally considered bad practice.

The more I look at this the more I think that the safest way to achieve a generic solution without asking everyone else using JUCE to take on unnecessary changes is to read the logger as I have done in my current implementation. Without changes to JUCE all cases will require JUCE_LOG_ASSERTIONS to be defined or the jassert macro will just be defined as empty.

A question for @jules however, is this really needed in juce_PlatformDefs.h

  #if JUCE_LOG_ASSERTIONS
   #define jassert(expression)      JUCE_BLOCK_WITH_FORCED_SEMICOLON (if (! (expression)) jassertfalse;)
  #else
   #define jassert(expression)      JUCE_BLOCK_WITH_FORCED_SEMICOLON ( ; )
  #endif

When you already have this earlier on in the file?

 #if JUCE_LOG_ASSERTIONS || JUCE_DEBUG
  #define JUCE_LOG_CURRENT_ASSERTION juce::logAssertion (__FILE__, __LINE__);
 #else
  #define JUCE_LOG_CURRENT_ASSERTION
 #endif

Can we not rely on the compiler to optimise away the code for us as you have with jassertfalse?

If we can rely on the compiler to optimise away the code then wrapping the definition of JUCE_LOG_CURRENT_ASSERTION with an #ifndef JUCE_LOG_CURRENT_ASSERTION would mean that projects can easily redefine what happens in the case of an assertion which would be useful in this scenario and keeps the changes to JUCE to a minimum.

Also I think I may have discovered some slightly odd behaviour, if JUCE_DEBUG and JUCE_DISABLE_ASSERTIONS are both defined as 1 then DBG() will no longer print which I think is unexpected, however if DBG() is simply redefined jassertfalse will print an assertion because it currently calls logAsserton() in this scenario (it just doesn’t do anything except create a string) while jassert will continue to not print assertions. I suspect few people if any are actually using JUCE_DISABLE_ASSERTIONS.

UPDATE:
I’ve just realised that if jassert is changed as I have suggested above then expressions which change state will stop it from being optimised away, and potentially will change behaviour of existing code (I could see how this could be considered both a good and a bad thing, but ultimately it’s a change in behaviour which on balance is probably negative), I guess there are other expressions that may prevent the optimisation too.

So I think I would stick with my original suggestion of reading the log and static_asserts if any of the expectJassert… macros are used when JUCE_LOG_ASSERTIONS or JUCE_DEBUG is not 1 or JUCE_DISABLE_ASSERTIONS is 1

1 Like
14

Exactly! I was just about to post this.

15

Anthony thanks, you’ve gone above and beyond :slight_smile:

16

Sorry to revive such an old thread but I revisited this code this weekend and managed to come up with something a little cleaner that doesn’t involve any macros.

I now have a JassertListener class that monitors the logger for assertions (similar to what was happening before), and then a simple getter on the class allows a way to ask how many jasserts occurred, optional arguments to the getter allow you to more precisely identify the jassert based on the thread it executed on, the file it was triggered in, and the line number it was triggered on within the file.

It’s also done in such a way that it should work regardless of what unit test framework you use, for example Google test.

If anyone does happen to be interested in using this please see the unit tests for examples.

2 Likes