More compact parameterized tests reporting in Spock 2.1

Marcin Zajączkowski — Tue, 05 Apr 2022 10:00:00 +0200

Get even more compact reporting for parameterized tests with Spock 2.1.

Historical background

TL;TR. To just get a recipe, jump to the “More compact reporting” section.

Parameterized tests, a place where Spock excels. A beautiful and powerful mechanism to reduce duplication across tests with a very readable table-like data representation (and powerful data providers if sophisticated custom logic is required).

When I started using Spock years ago (0.6?), one question perpetually dinned in somebody’s ears. Why unrolling was not enabled by default? Adding @Unroll (even at the test class level) was for me redundant, error prone and… just boring. Due to that, I created a small spock-global-unroll extension which was dead simple. It was enough to just put it on a classpath (i.e. add as a dependency in Gradle or Maven) to have all the parameterized tests automatically unrolled in the whole project. Zero configuration, it just worked. You could remove all @Unroll annotation (without parameters) in your project. I really liked it and used it in every project I had my fingers in. Have you ever heard about it?

It was a rhetorical question. The vast majority of you did not. Why? Because it was an extension and people start looking for extensions when something does not work. For the extensions that improve (even dramatically) some aspect of your tests you would need to hear about them before (and I am hopeless at self promotion :-/).

That was a reason why I pushed hard to have it available and enabled by default in the core of Spock 2.0. With a comprehensive implementation by Björn Kautler in landed in 2.0-M3 to stay there for good. However, preparing some time ago my presentation “What you can expect from Spock 2?" , I’ve noticed that it could be improved even more.

Default (verbose) reporting

Performance tip. For Gradle projects, I prefer to execute unit and integration tests directly by IDEA, not by IDEA using Gradle (which - unfortunately - has been a default option since 2019 or so). First of all, it is much faster - a crucioal thing for TDD - and in the majority of cases it works perfectly fine. The examples below use that mode. You can read how to speed up your unit tests execution in my another blog post.

Let’s assume we have a simple parameterized test:

class UnrollSpec extends Specification {

    void "should calculate number of characters in given animal name"() {
        expect:
            animalName.length() == expectedLength
        where:
            animalName     || expectedLength
            "dog"          || 3
            "wolf"         || 4
            "hippopotamus" || 12
    }
}

Spock 2 unrolls parameterized tests by default displaying also all the used parameters. However, the output in IDEA is hard to read:

To see the parameters it is usually needed to scroll the result window horizontally (if luckily your fancy mouse supports that :-) ) or reduce space available for the test standard output.

The crazy idea sprang in my mind - could the (repeated) feature name be omitted in the iterations name?

To remove it for a particular test it is easy:

@Unroll("animaName: #animalName, expectedLength: #expectedLength")    //inconvenient - just for one test
void "should calculate number of characters in given animal name"() {
    //...
}

However, we have been just celebrating the removal of @Unroll, not to be forced to put it back everywhere… Luckily, Spock has a global configuration mechanism, could not it be used?

After some minutes when I was analyzing the UnrollExtension code, it turned out that no. SpockConfig.groovy allows to define a custom defaultPattern, but there is no way to display just the parameters. It is a FOSS project, what should be done in that case? :-)

More compact reporting

The idea described in the previous section resulted in the improvements in Spock 2.1. There is a new configuration parameter which needs to be placed inside the unroll {} closure in the SpockConfig.groovy file (located for example in src/test/resources - NOT together with the test classes):

//in SpockConfig.groovy
unroll {
    includeFeatureNameForIterations false
}

Much shorter, still meaningful output. And for all tests in the project!

Note. If for some reason, you want to use it just for individual features @Unroll('#dataVariablesWithIndex') can be used as well. Please refer the official documentation for other use cases.

Keep in mind that SpockConfig.groovy can be placed in various places, including your home directory - to more global impact.

Limitations

Why that marvelous feature was not enabled by default in Spock 2.1? We were thinking about that, but there two arguments against:

Backward compatibility.
Poor support in some reporting tools.

The second point was more important. While the IDEA represents the test results with a very nice tree-like view, some other tools are not. The test report generated by Gradle or Maven’s Surefire displays them as a flat list, without the (parent) test name. As a result, not having the feature name in every iteration could be confusing.

As a result you might want to enable that feature conditionally only if executed in IDEA, for example:

//in SpockConfig.groovy
static boolean isExecutedFromIdea() {
    return System.getProperty("java.class.path").contains("idea_rt.jar")    //any more reliable way?
}

unroll {
    includeFeatureNameForIterations !isExecutedFromIdea()
}

That check is somehow fragile (e.g. doesn’t work if IDEA executes tests not directly, but using Gradle), but I wasn’t able to find anything better. Feel free to put your proposal in the comments.

Summary

I hope that you also found includeFeatureNameForIterations false useful. Hopefully, having better support in the reporting tools there will be no obstacles to make it enabled by default in Spock 3.0.

Btw, in the middle of May, I will be talking about that and other new and noteworthy in Spock 2 at the GeeCON conference . This time, finally, on-site, in Kraków, Poland (but the whole conference is in English).