Adding Time to UnitTest

Feature Requests
1

I’ve been experimenting with JUnit exports of test runs recently and upon looking at the format noticed that you can display a duration for each testcase.
This would be really useful for us as it could be a quick way to track performance over time and ensure we don’t make any changes that destroy this.

I think I can get this information quite easily if UnitTest::TestResult had Time startTime, endTime members which were filled in by beginTest/endTest.

Would that be a possible addition?

1 Like
2

Sound like a good addition indeed.

I also miss the fact that the test runner doesn’t exit the application with anything else than 0 exit code in case any error was encountered during tests. This will make it harder for CI to fail on tests failures.

1 Like
3

Yeah, I manually check if any tests have failed and then either set the exit code via JuceApplication or return 1 from main if its a console app.

To be honest, I’d rather the UnitTest runner didn’t do this for me as the exit code should be determined by me rather than running some tests.

4

yeah true. there should be an easier way to do that other than scanning all the test results.

5

Agreed, even a free function like int getNumFailures (const UnitTestRunner&); would be handy for that. I’ve got one of my own I use.

1 Like
6

i noticed the UnitTestRunner app that runs all juce unit tests has an hardcoded 0 return code, so automating juce unit tests in travis and appveyor is never failing

7

There is a hard-coded 0 at the very end indeed, but UnitTestRunner does exit with 1 if there are any failures:

8

nice! didn’t spot it, i might have an old version

9

It was introduced between versions 5.0.2 and 5.1.0 in this commit:

10

I’m going to give this a gentle bump seeing as it’s a small request and would really help me out with developing this new module.

The key thing here is that it really has to be part of juce::UnitTest in order to parse the unit test results in to JUnit format so it’s shown in our CI. Just adding timestamps/durations to our log files isn’t really an option as you can’t track them over time easily.

I can create a PR if that would be useful?

11

This is on juce6 and develop now - apologies for the wait!

2 Likes
12

Perfect! Thanks Ed!