Deep Dive on Solid Soft

Spock vs JUnit 5 - the ultimate feature comparison

Marcin Zajączkowski — Wed, 15 Apr 2020 10:00:00 +0200

What is the best testing framework for Java code nowadays? Spock or JUnit 5? Check it out in this ultimate feature comparison.

Spock was a game changer for all the people struggling with unit testing in JUnit 4. Compact syntax, parameterized tests or flexibility to mention just a few advantages. Over 10 years after JUnit 4.0, the brand new, written from scratch, Java 8 optimized Junit 5 has been released. And time has not stopped then.

In this blog post, I will compare selected areas of Spock and JUnit 5 to give you an overview how the situation looks like nowadays. I will try to answer the question if its time for Spock to fade into oblivion or maybe quite the opposite it is still light years ahead of JUnit 5.

This blog post is a written version of my presentation titled “Spock vs JUnit 5 - Clash of the Titans”. If you prefer, feel free to see the slides or watch the video instead.

Historical background

As a warm-up, let’s take a look at the important dates in the automatic code testing in Java.

2000 - JUnit - first xUnit for Java
2004 - TestNG - Java 5 (annotations) leveraged in tests
- with some unique (at the time) features (such as parameterized tests, test repetition or test dependency)
2006 - JUnit 4 - Java 5 support
- catching up TestNG features over the next few years
- de facto standard (the world chose :-/ ), steady evolution rather than revolution
2009 - Spock - revamped test creation
- fresh ideas and power of Groovy under the hood
2017 - JUnit 5 - redesigned and rewritten from scratch
- new king of the hill? - let’s check it out!

This comparison covers the latest released stable versions (as of April 2020) of JUnit (5.6) and Spock (1.3). I am fully aware of in-development Spock 2.0 and I plan to update this comparison (or write a supplement post) once 2.0-final is available.

Development comparison

Let’s start with a comparison of some biased aspects of development of JUnit 5 and Spock.

	JUnit 5 (as of 5.6)	Spock (as of 1.3)
inception year	2015	2009
number of GitHub stars	3,7K	~2,6K
number of commits	~6K	~2,5K
development activity	high (young project)	medium (mature project)
number of active committers	3	1 + 3
number of contributors (ever)	~130	~80

The first thing to notice is the fact that JUnit 5 is much younger project with very high development activity. There are 3 active committers with a bunch of external contributors.

One the other hand, Spock is much more mature with lower development activity. However, many of the features developed in JUnit 5 recently were already available in Spock. Therefor, the need for lots of commit in Spock is lower. Spock has currently one main maintainer and 2-3 people regularly contributing to the project (of course also with a pack of external contributors).

Partial verdict

Development: JUnit 5 wins

Reason: Mainly thanks to more active development activity in recent years.

Tool support

The next thing to cover is Java 11 compatibility and support in tools.

	JUnit 5	Spock
Java 11+	very good	good (with modern Groovy 2.5.x)
IntelliJ IDEA	built-in	built-in (some Groovy limitations)
Eclipse	build-in	built-in (some Groovy limitations)
Netbeans	10.0+ (Maven only)	unknown
Maven	plugin (official, Surefire)	plugin (GMavenPlus for Groovy)
Gradle	built-in	built-in
SonarQube	built-in	plugin (official for Groovy)
PIT - mutation testing	plugin (official)	build-in

Both of the frameworks are currently very well supported in the Java ecosystem. However, it is worth to mention that tests (specifications) in Spock are written in Groovy (which notabene is the main power of Spock). As a downside, due to much more dynamic nature of Groovy as a language, IDEs have much harder time to provide as good support for it as for Java (e.g. refactoring or compile time error reporting).

Regarding Java 11 and Spock, with modern Groovy 2.5.x it should work flawlessly. In general, most of the things should work fine also with Java 12, 13 and (with Groovy 2.5.10+) also 14. The official Java 14+ support should be available in Spock 2.0 (still in development) with Groovy 3.0.

On the other hand, JUnit 5 is a role model on that field. It is continuously tested on a CI server with the recent Java versions (including the early access builds, such as OpenJDK 15 EA).

Partial verdict

Tool support: JUnit 5 wins

Reason: Great IDE support for Java code and continuous testing with the bleeding edge versions of Java.

Test structure

Let’s move on to less technical aspects of test development - test structure.

As die-hard followers of my blog may know, I am a big propagator of well structured automatic tests with the BDD-originated given-when-then concept. In short, it unifies test creation, improves their readability and makes it easier to write, especially for less experienced people. To learn more, please take a loot at a separate blog post that I wrote some time ago.

JUnit 5

class SimpleCalculatorTest {
    @Test
    void shouldAddTwoNumbers() {
        //given
        Calculator calculator = new Calculator();
        //when
        int result = calculator.add(1, 2);
        //then
        assertEquals(3, result);
    }
}

The given-when-then sections are marked just by plain comments in code. It is not perfect. They might be forgotten and are easy to lost/misplace during refactoring. Could it be done better?

Spock

class SimpleCalculatorSpec extends Specification {
    def "should add two numbers"() {
        given:
            Calculator calculator = new Calculator()
        when:
            int result = calculator.add(1, 2)
        then:
            result == 3
    }
}

Well, in Spock the (almost not used in Java) label construction is leveraged. It is crucial to highlight that they are not just comments in code - they are a part of the specification. For example, having when: removed in the example above, generates real compilation errors. Nice.

Btw, of course, in Spock it is possible to write just one-liners, if feasible. However, developers have to do it intently.

Partial verdict

Test structure: Spock wins

Reason: Spock cares more about well structured (and more readable) tests.

Btw, with both JUnit 5 and Spock it is worth to use code templates to generate test skeleton to fill instead of writing all those given-when-then by hand :-).

Exception testing

Another topic I want to cover is exception testing. In general, to test corner cases (alternative scenarios) it is needed to verify that an exception with a given type was thrown. Usually, it is followed by some extra assertions such as an exception message and a cause.

JUnit 5

In JUnit 5, it is no longer possible to use @Test(expected = NullPointerException.class), useful for one-liners, but potentially risky with more complicated test code. Instead of, we have the assertThrows() construction, similar to catchThrowable() from AssertJ.

@Test
void shouldThrowBusinessExceptionOnCommunicationProblem() {
    //when
    Executable e = () -> client.sendPing(TEST_REQUEST_ID)
    //then
    CommunicationException thrown = assertThrows(CommunicationException.class, e);
    assertEquals("Communication problem when sending request with id: " + TEST_REQUEST_ID,
                 thrown.getMessage());
    assertEquals(TEST_REQUEST_ID, thrown.getRequestId());
}

The syntax is very compact (thanks to the lambda expression which is used to catch an exception). In addition, it is easy to perform any required further assertion(s) on the returned exception instance.

Spock

Spock, on the other hand, still provides @FailsWith(NullPointerException) for one-liners. However, it is almost completely not used on a daily basis. All thanks to the thrown() construction.

def "should capture exception"() {
    when:
        client.sendPing(TEST_REQUEST_ID)
    then:
        CommunicationException e = thrown()
        e.message == "Communication problem when sending request with id: $TEST_REQUEST_ID"
        e.requestId == TEST_REQUEST_ID
}

It is worth to notice that there is no lambda expression to catch an exception. To implement that Spock leverages Groovy AST transformations which under the hood transparently add required code. As a result, the exception thrown in the when block is caught and returned from the thrown() method. Really smart.

In addition, it is nice that in Spock, there is a smart type inference. It is not needed to precise the exception type twice. Based on the variable type on the left side, Spock is able to create the proper try-catch block.

As a bonus, there as a separate utility class Exceptions in Spock which provides extra assertions to play with the cause chain.

Partial verdict

Exception testing: Spock wins

Reason: Even shorter and less intrusive mechanism for exception testing.

Conditional test execution

In multiple situation it is required to (not) execute a given test only if given condition is (not) met.

The most common cases include:

particular Java version
- reflection hack to use newer version features
- compatibility testing for lower version
specific operating system
- notifications about changed files on Mac are much delayed
- testing symlinks on Windows makes no sense
tests executed only on CI server, stage environment, …

JUnit 5

JUnit 5 introduced brand new support for annotation-based @Enabled* and @Disabled* conditions.

There is a predefined set of conditions for:

JVM version
operating system
system property
environment variable

For example:

@Test
@DisabledOnOs(OS.WINDOWS)
void shouldTestSymlinksBasedLogic() {
    ...
}

@Test
@EnabledIfSystemProperty(named = "os.arch", matches = ".*32.*")
void shouldBeRunOn32BitSystems() {
    ...
}

The JVM version and operating system conditions are enums and thanks to that, there is a nice code completion provided by an IDE. In addition the conditions can be used as meta-annotations to make more complex scenarios easier to use in multiple places.

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Test
@EnabledOnOs(WINDOWS)
@EnabledIfSystemProperty(named = "os.arch", matches = ".*32.*")
@interface TestOn32BitWindows {
}

With the following usage:

@TestOn32BitWindows
void shouldDoStrangeThings1OnLegacyWindows() { ... }

@TestOn32BitWindows
void shouldDoStrangeThings2OnLegacyWindows() { ... }

Originally, there was also support for custom logic in script-based inline conditions. However, due to limited usability (code written in String) and deprecation of Nashorn in Java 11 it has been removed in JUnit 5.6. Writing own condition logic currently requires a custom implementation of ExecutionCondition.

Spock

Since time immemorial, Spock has provided support for annotation based conditions with @Requires and @IgnoreIf. Out of box it is possible to check:

JVM version
operating system
system property
environment variable

The same set implemented later on in JUnit 5. However, the usage is slightly different.

@IgnoreIf({ !jvm.java8Compatible })
def "should return empty Optional by default for unstubbed methods with Java 8+"() { ... }

@Requires({ sys["targetEnvironment"] != "prod" })
def "should execute smoke testing on non production environment"() { ... }

Instead of enums, the logic is written in Groovy Closures leveraging delegation to the jvm/os/sys/... objects providing a dedicated methods (such as isJava11Compatible() or isLinux()). Unfortunately there is no code completion available by default, but it can be enabled with a small trick (Update. Just please be aware that the linked slides are from 2016 - Spock 1.0/1.1 - and in some places are outdated).

Using Closures provides one more important benefit - custom logic can implemented inline with pure Groovy.

1
2

@Requires({ isStrongCryptographyEnabled() })    //custom static method
def "should test strong cryptography-based features"() { ... }

Partial verdict

Conditional test execution: Spock wins

Reason: Higher flexibility with non standard conditions.

Mocking

Mocking is a crucial part of automatic code testing. It is especially beneficial in unit testing, but can be also usable in integration testing (e.g. with Spring Framework).

JUnit 5

In JUnit 5 tests, Mockito is the first port of call when it comes to mocking.

@Test
public void should_not_call_remote_service_if_found_in_cache() {
    //given
    given(cacheMock.getCachedOperator(CACHED_MOBILE_NUMBER)).willReturn(Optional.of(PLUS));
    //when
    service.checkOperator(CACHED_MOBILE_NUMBER);
    //then
    then(webserviceMock).should(never()).checkOperator(CACHED_MOBILE_NUMBER);
//   verify(webserviceMock, never()).checkOperator(CACHED_MOBILE_NUMBER);   //alternative syntax
}

Mockito is an industrial standard for mocking in Java. It is very mature, well known with a rich set of features. In addition, Spring Boot provides the official support for injecting Mockito’s mocks and spies into its context, which can be very useful in integration tests.

Spock

Spock on the other hand contains a built-in mocking subsystem.

def "should not hit remote service if found in cache"() {
    given:
        cacheMock.getCachedOperator(CACHED_MOBILE_NUMBER) >> Optional.of(PLUS)
    when:
        service.checkOperator(CACHED_MOBILE_NUMBER)
    then:
        0 * webserviceMock.checkOperator(CACHED_MOBILE_NUMBER)
}

While the syntax in Mockito is pretty readable, there are all those given/thenReturn/thenAnswer/... to write. Spock, however, leverages the power of operator overloading in Groovy (and of course its AST transformations) to make stubbing and verification as simple as possible.

given(cacheMock.getCachedOperator(CACHED_MOBILE_NUMBER)).willReturn(Optional.of(PLUS)); //Mockito
// vs
cacheMock.getCachedOperator(CACHED_MOBILE_NUMBER) >> Optional.of(PLUS) //Spock

verify(webserviceMock, never()).checkOperator(CACHED_MOBILE_NUMBER); //Mockito
// vs 
0 * webserviceMock.checkOperator(CACHED_MOBILE_NUMBER) //Spock

In a basic case, it’s enough to write a method, use an operator and provide a returned value. In verification it’s just a method call with requested cardinality. Most likely, it will not be possible to achieve that in Java in the predictable future.

Historically, the main limitation with Spock’s mocks was their tight coupling with the specification (the test itself). As a result it was not possible to use Spock’s stubs, mocks and spies in the Spring context for integration tests. However, starting with Spock 1.1 it was relaxed and Spock 1.2 introduced the first class support for mocking and spying with Spring. As a bonus, it is available also for pure Spring context (Spring Boot is not required as it takes place with Mockito’s mocks).

To be fair, it is required to admit that more magic in Spock generates some quirks with some corner cases. Nevertheless, once knowing all of them the readability and compactness definitely pay off :-).

One more thing. It is worth to remember that with Spock used for testing, we are not forced to use it also for mocking. It is perfectly fine to use Mockito if wanted (or needed).

Partial verdict

Mocking: Spock wins

Reason: The syntax and really readable and compact. And with Spring support in Spock 1.2+ the last big limitation of the mocking subsystem in Spock was removed.

Parameterized tests

Parameterized tests in general provide a way to call one test with different set of input (and expected) parameters. It can dramatically reduce duplication in specific cases (e.g. in testing with the user defined acceptance data set).

A word of warning here. In some scenarios, using parameterized tests can hide specific business use cases which would be clearly exposed in our non-parameterized tests otherwise. There are techniques to avoid that, but it is a topic for another article.

JUnit 5

Parameterized tests were one of the key features of TestNG from its early days. In JUnit 4 the feature has been not available for year and even once implemented, the design made it very unpractical to use in most of the cases. To improve the situation the 3rd-party runners were available, but it was still sad (for JUnit 4).

Luckily, JUnit 5 is the first version of JUnit with sane built-in parameterized tests support.

@ParameterizedTest
@CsvSource({ "dog, 3", "wolf, 4", "hippopotamus , 12" })
void testStringLengthWithParameterizedTestInJUnit5(String word, int expectedNumberOfLetters) {
    assertEquals(word.length(), expectedNumberOfLetters);
}

For simple scenario with inline values (provided as CSV strings), there is a nice implicit argument conversion from String for various types (e.g. dates, files, path or UUID). It just requires to add @ParameterizedTest and @CsvSource (or @ValueSource, @EnumSource, etc.).

However, not everything can be put into string. The commonly encountered scenario is an object creation or calling a method defined in a test fixture. To support that JUnit 5 gives an ability to call a local method (or the one from external object) to return required input/output arguments. They should be provided as Stream (or List) of Arguments.

@ParameterizedTest(name = "value to pay for invoice {0} should be {1}")
@MethodSource("invoiceProvider")
void shouldCalculateToPayValueForInvoice(Invoice invoice, BigDecimal expectedValueToPay) {
    //when
    int valueToPay = invoice.toPayValue();
    //expect
    assertEquals(expectedValueToPay, valueToPay);
}

private static Stream<Arguments> invoiceProvider() {
    return Stream.of(
            Arguments.of(regularInvoice(), 54),
            Arguments.of(overduedInvoice(), 81),
            Arguments.of(paidInvoice(), 0)
    );
}

It is just required to use @MethodSource to point the method (or use a naming convention to resolve the correct method name automatically).

The thing I don’t like is a lack of the arguments type check. In tests with multiple parameters it can be quite confusing to determine which is which.

As a bonus it is also possible to customize a test name which will be displayed in the report (in IDE, HTML report, etc.).

Spock

In Spock, first let’s take a longer look at the simple parameterized test.

@Unroll
def "should sum two integers (#x + #y = #expectedResult)"() {
    when:
        int result = calculator.add(x, y)
    then:
        result == expectedResult
    where:
         x |  y || expectedResult
         1 |  2 ||  3
        -2 |  3 ||  1
        -1 | -2 || -3
}

The words that come to my mind are “state of the art”. This is a place where Spocks excels.

With table-like formatting after the where keyword, input data looks very natural and is easy to read. What is worth mentioning, there is no need to define x, y, expectedResult as method arguments in a test (as it takes place in JUnit 5). The variables are added implicitly with full visibility and type inference in IDE.

The method name can be parameterized inline with the #var syntax (it’s Groovy - methods can have spaces and other “strange” characters).

In Spock it is also not a problem to use table-like syntax in a situation where constructor or methods are needed to be called (which in JUnit 5 requires using a separate provider method).

@Unroll("Value to pay for invoice #invoice should be #expectedValueToPay")
def "should calculate value to pay for invoice"() {
    expect:
        invoice.valueToPay() == expectedValueToPay
    where:
        invoice           || expectedValueToPay
        regularInvoice()  || 54
        overduedInvoice() || 81
        paidInvoice()     || 0
}

While working with acceptance testing where results are presented to less technical users, it is also possible to distinguish a test name for developers with a scenario name for QA engineers (or product owners) by writing it in the @Unroll annotation.

Of course, for more complex/advanced use cases (e.g. generating input parameters dynamically based on exterenal source), data pipes and data providers can be used. They are not as natural as the aforementioned table, but are very powerful.

@Unroll("#pesel is valid (#dbId)")
def "should validate PESEL correctness (CSV)"() {
    expect:
        sut.validate(pesel)
    where:
        [dbId, _, _, pesel] << readValidPeopleFromCSVFile()
                                   .readLines().collect { it.split(',') }
                                   //ugly way to read CSV - don't do this :-)
}

Partial verdict

Parameterized tests: Spock wins

Reason: It is enough to visually compare parameterized tests in those two frameworks.

Migration

The next point is very important for people currently using JUnit 4.

JUnit 4 → JUnit 5

The JUnit 5 tests can co-exist with the JUnit 4 tests. There is a dedicated module junit5-vintage which executes old JUnit 4 tests on the new JUnit Platform (part of JUnit 5).

Writing new tests with JUnit 5 is also easy. The test structure is very similar. We just need to learn some new keywords, annotations and assertions (and remember to use @Test from the new org.junit.jupiter.api package :-) ).

JUnit 4 → Spock

The Spock tests can also co-exists with the JUnit 4 tests. In fact Spock 1.x is a (sophisticated, but still) JUnit 4 runner. However, besides that there is a completely new test structure which we need to get familiar with. It is easy to grasp and, in the end, it is more readable, but still, it is something new to learn.

Partial verdict

Tool support: JUnit 5 wins

Reason: The JUnit 5 tests are more similar to the JUnit 4 tests and therefor, that migration path should be a little bit easier.

One more important migration-related thing here. During my training or consulting , I do not recommend rewriting existing hundreds and thousands of tests from JUnit 4 to the new technology, just for fun. Most of the existing tests will stay untouched for the foreseeable future. It is much better to write the new tests in the new technology and rewrite existing tests only along the way of doing some production logic related changes in them.

Summary - final verdict

Inarguably, JUnit 5 is a great progress over JUnit 4. Many features (previously) unique to Spock now are available in JUnit 5. However, Spock still excels in some areas. The best choice is no longer obvious.

Here, an anecdote, proposed my a colleague of mine. There are BMW and Audi cars. Both brands are very good and the choice usually depends on our personal preferences.

There same with Spock and JUnit 5. First, let’s take a look at the features comparison of the presented testing frameworks.

	JUnit 5 (as of 5.6)	Spock (as of 1.3)
development	very good	good (new features and bugfixes)
learning curve	very good (similar to 4)	good (Groovy to grasp)
tool support	very good (trending up)	good (weaker compile time checks)
test structure	good	very good (BDD by default)
exception testing	good	very good
conditional test execution	good	very good (custom logic)
mocking	good (Mockito)	good+ (very compact, some quirks)
parameterized tests	good	very good (exceptional!)
migration from JUnit 4	very good	good

As you see, they both have their strengths and weaknesses. In addition to covered aspects, you should add to that list other aspects which are important for you (and for your team) and see which framework fits you more.

For instance.

If you prefer:

old good Java as the only language
stability and being mainstream
stronger compile time error checking

you should choose JUnit 5.

On the other hand,

if - like me - you favor:

simplicity and readability
power of Groovy under the hood
beautiful parameterized tests and exception testing

then Spock should be better for you.

One final reflection. Preparing my presentation (and later on that blog post), I realized it would be great to have only that kind of dilemma while crafting software, choose between two pieces of good software :-).

A note about Spock logo. As of earthdate 2020-04, Spock doesn’t have the official logo. The one used in the lead photo is a proposal by Søren Berg Glasius made back in 2014. There are also some other proposals, but if you have a good idea, feel free to propose it there (or vote for existing candidates).

Mysteriously broken reduction operation on parallel streams in Java (explained)

Marcin Zajączkowski — Thu, 12 Mar 2020 14:00:00 +0200

How to not get into trouble writing custom reduction operation (.reduce()) on (parallel) streams with Java 8+ (including Java 14).

Introduction

Recently…, well this article has spent years in my freezer for the blog posts. Originally, I have encountered this problem back in 2014 while preparing my Java 8 training session (recently renamed to Java - functional thinking ). Then, I noticed that it is possible to achieve interesting results writing a custom reduce operation implementation when a parallel stream is used. For various reasons, the partially written article was waiting for its completion around 6 years. Time flies, however, the case is still valid with the latest - as a time of writing - Java 14. So, let’s check it out.

Our case - simple number concatenation using reduce

As a short introduction, a reduction operation (also called a fold) takes input elements (here of a stream) and collects them into combined summary result. Some well know specialized reduction variants, available in the Stream API, include, inter alia, sum(), min() and count(). Nevertheless, we can write our custom, non-standard reduction operation (if needed).

Let’s assume we would like to concatenate a list of numbers into a string. A case which could be easier to achieve in other ways, but simple enough to do not focus on realized logic, but rather a nuances of a reduction operation itself. A reduce method on stream can be used:

@Test
public void shouldConcatNumbersUsingReduce() {
    //given
    final List<Integer> input = Arrays.asList(1, 2, 3, 4);

    //when
    final String result = input.stream()
            .reduce(new StringBuilder(),
                    StringBuilder::append,
                    StringBuilder::append)
            .toString();

    //then
    assertThat(result).isEqualToIgnoringCase("1234");   //passes
}

Very compact and does its work. Numbers are concatenated. Similar implementations are common in internet and everything is ok unless you try to use it with the parallel stream:

@Test
public void shouldConcatNumbersUsingReduceParallel() {   //broken!
    //given
    List<Integer> input = Arrays.asList(1, 2, 3, 4);

    //when
    String result = input.parallelStream()
            .reduce(new StringBuilder(),
                    StringBuilder::append,
                    StringBuilder::append)
            .toString();

    //then
    assertThat(result).isEqualToIgnoringCase("1234");   //ups!
}

This test fails with the assertion error similar to that:

java.lang.AssertionError: 
Expecting:
 <"343421343421343421343421">
to be equal to:
 <"1234">

It looks strange. Very often incomprehensible things are caused by some race condition in the parallel code which matches parallelStream(). But first let’s analyze what happens in a reduction operation step by step (if you are savvy in sequential reduction operations skip to the subsequent section).

Reduction in sequential stream

If you are not very familiar with method references the example can look a little bit cryptic. Let’s rewrite it using lambda expressions (instead method references):

String result = input.stream()
        .reduce(new StringBuilder(),
                (acc, x) -> acc.append(x),      //accumulation
                (sb1, sb2) -> sb1.append(sb2))  //combining
        .toString();

The new StringBuilder is used as an initial value (identity). For every element from an input list the second line in reduce will be called ((acc, x) -> acc.append(x)) with acc (an accumulator) containing concatenated already processed elements and x the current number. x is appended to the previously concatenated elements.

From the algorithm analysis, the expected values in subsequent iterations should be as follows:

iteration	0	1	2	3
acc	"”	“1”	“12”	“123”
x	1	2	3	4
returned	“1”	“12”	“123”	“1234”

Is this ok? Let’s verify it in code.

As debugging (prospectively) concurrent code is hard, especially in a blog post, let’s use plain old debug outputs, well known from C.

String result = input.stream()
    .reduce(
        new StringBuilder(),
        (acc, x) -> {
            String accString = acc.toString();
            StringBuilder ret = acc.append(x);
            log.info("Accumulation: '{}', x: '{}', returned: '{}'", format("%4s", accString), 
                                     format("%4s", x), format("%4s", ret));
            return ret;
        },
        (sb1, sb2) -> {
            String sb1String = sb1.toString();
            StringBuilder ret = sb1.append(sb2);
            log.info("Combining: sb1: '{}', sb2: '{}', returned: '{}'", format("%4s", sb1String),
                                 format("%4s", sb2), format("%4s", ret));
            return ret;
        })
    .toString();

We slightly complicated the code, but it is only for the test (debug) purpose. The output is as expected:

18:09:05.839 [main] - Accumulation: acc: '    ', x: '   1', returned: '   1'
18:09:05.844 [main] - Accumulation: acc: '   1', x: '   2', returned: '  12'
18:09:05.844 [main] - Accumulation: acc: '  12', x: '   3', returned: ' 123'
18:09:05.844 [main] - Accumulation: acc: ' 123', x: '   4', returned: '1234'

Let’s skip for a while the log from the combining operation (line 14). It is not used in sequential streams.

In the output, we also clearly see that all operations take place in the main thread. Which is expected for sequential synchronous stream processing.

Reduction in parallel stream - behavior in working implementation

In a parallel stream, accumulation operations can be performed in different threads. Those partial results are later combined using lambda expression passed as the third argument (a combiner) in a reduce method.

Let’s take a look at the diagnostic messages emitted during a sample execution with parallel stream.

One important note here. The output is from the correctly implemented reduction operation (presented at the end of the blog post) rather than the version above to make the whole thing easier to grasp.

[ForkJoinPool.commonPool-worker-3] - Accumulation: acc: '    ', x:   '   1', returned: '   1'
[ForkJoinPool.commonPool-worker-1] - Accumulation: acc: '    ', x:   '   2', returned: '   2'
[main]                             - Accumulation: acc: '    ', x:   '   3', returned: '   3'
[ForkJoinPool.commonPool-worker-1] - Combining   : sb1: '   1', sb2: '   2', returned: '  12'
[ForkJoinPool.commonPool-worker-2] - Accumulation: acc: '    ', x:   '   4', returned: '   4'
[ForkJoinPool.commonPool-worker-2] - Combining   : sb1: '   3', sb2: '   4', returned: '  34'
[ForkJoinPool.commonPool-worker-2] - Combining   : sb1: '  12', sb2: '  34', returned: '1234'

Let’s analyze it line by line.

In the first 2 lines:

1
2

[ForkJoinPool.commonPool-worker-3] - Accumulation: acc: '    ', x:   '   1', returned: '   1'
[ForkJoinPool.commonPool-worker-1] - Accumulation: acc: '    ', x:   '   2', returned: '   2'

the accumulation operation is performed in parallel on the two first elements on the list. In the forth line those partial results are combined into a larger list:

`4`	`[ForkJoinPool.commonPool-worker-1] - Combining : sb1: ' 1', sb2: ' 2', returned: ' 12'`

The same situation, for the rest of the items, takes place in the line 3, 5 and 6.

[main]                             - Accumulation: acc: '    ', x:   '   3', returned: '   3'
...
[ForkJoinPool.commonPool-worker-2] - Accumulation: acc: '    ', x:   '   4', returned: '   4'
[ForkJoinPool.commonPool-worker-2] - Combining   : sb1: '   3', sb2: '   4', returned: '  34'

And the end, two partial result: ‘12’ and ‘34’ are combined together to provide the final result:

`7`	`[ForkJoinPool.commonPool-worker-2] - Combining : sb1: ' 12', sb2: ' 34', returned: '1234'`

Do you have - after logging at the listings - any suspicious what could be wrong with the original implementation? With the debug statements it much easier to spot it.

Reduction in parallel stream with widespread (broken) implementation

Let’s recall the widespread (broken when running in parallel) implementation once more:

String result = input.parallelStream()
        .reduce(new StringBuilder(),
                (acc, x) -> acc.append(x),      //accumulation
                (sb1, sb2) -> sb1.append(sb2))  //combining
        .toString();

together with its debug enhanced variant:

String result = input.parallelStream()
    .reduce(
        new StringBuilder(),
        (acc, x) -> {
            String accString = acc.toString();
            StringBuilder ret = acc.append(x);
            log.info("Accumulation: '{}', x: '{}', returned: '{}'", format("%4s", accString), 
                                     format("%4s", x), format("%4s", ret));
            return ret;
        },
        (sb1, sb2) -> {
            String sb1String = sb1.toString();
            StringBuilder ret = sb1.append(sb2);
            log.info("Combining: sb1: '{}', sb2: '{}', returned: '{}'", format("%4s", sb1String),
                                 format("%4s", sb2), format("%4s", ret));
            return ret;
        })
    .toString();

The output from the parallel processing execution:

10:42:53.334 [ForkJoinPool.commonPool-worker-5]  - Accumulation: acc: '   3', x: '   4', returned: '  34'
10:42:53.334 [ForkJoinPool.commonPool-worker-23] - Accumulation: acc: ' 342', x: '   1', returned: '3421'
10:42:53.334 [ForkJoinPool.commonPool-worker-19] - Accumulation: acc :'  34', x: '   2', returned: '3421'
10:42:53.334 [main]                              - Accumulation: acc :'    ', x: '   3', returned: '   3'
10:42:53.339 [ForkJoinPool.commonPool-worker-19] - Combining: sb1: '3421', sb2: '342134213421', returned: '342134213421'
10:42:53.339 [main]                              - Combining: sb1: '3421', sb2: '342134213421', returned: '342134213421'
10:42:53.340 [main]                              - Combining: sb1: '342134213421', sb2: '342134213421342134213421', returned: '342134213421342134213421'

The strange things start to happen from the first line. Instead of having an empty accumulator (the “acc” variable) there is “3”. In the second line there is already “342” (still instead of an empty StringBuilder). We discover empty accumulator in the 3rd line, which anyway could be the 1st line (it is a matter of logging operations from multiple threads - they were executed at “the same moment”). Nonetheless, all the first 4 lines should have the accumulator empty. It clearly shows there is a problem with a mutable data structure shared across the threads, in both accumulation and combining phrases.

A look at the StringBuilder documentation ensures us that “append” adds a provided value to the StringBuilder and returns its instance (aka this). Mutating input variables is not recommended in the stream processing, however the root of the problem is a fact that initial “neutral” value (identity) passes to every initial accumulation operation is one and the same StringBuilder instance. It is passed as the first element of the “reduce” method execution.

Someone could say: “let’s use StringBuffer - its operations are thread safe”. However, it will not fix the situation here. Having multiple threads operating on the same structure (even thread safe) is also broken assuming that all initial operations assumes a private accumulator instance to create an independent list of elements which will be concatenated into the biggest one in the following steps.

Reduction in parallel stream - working implementation

Main case

With StringBuffer and string concatenation the only required modification is not mutating the input objects:

String result = input.parallelStream()
        .reduce(
                new StringBuilder(),
                (acc, x) -> new StringBuilder(acc).append(x),
                (sb1, sb2) -> new StringBuilder(sb1).append(sb2))
        .toString();

In every iteration the new StringBuilder instance is created. Thanks to that there is no mutation of the initial identity element shared across threads. In general mutation of the stream elements (or doing side effects in general) can lead to some unpredictable situations, especially in the parallel execution.

Java API limitations

The above change solves the problem (at the cost of greatly increased number of created objects, but let’s skip it for a moment). Unfortunately, many structures, especially those from Java Collection Framework which remembers Java 1.2 (releases 1998 - over 20 years ago!), does not provide so friendly API to perform operations with the method chaining syntax/approach.

Let’s take a look at the (artificial) working example of using the java.util.List API in a reduce operation simulating mapping to double every element of the input list:

List<Integer> doubledValues = stream
    .reduce(Collections.emptyList(),
            (list, x) -> {
                List<Integer> resultList = new ArrayList<>(list);
                resultList.add(x * 2);
                return resultList;
            },
            (list1, list2) -> {
                List<Integer> resultList = new ArrayList<>(list1);
                resultList.addAll(list2);
                return resultList;
            });

True one-liners with a method chaining would make this code more readable.

Please note that the aforementioned logic (doubling list elements) can be implemented easily in native operations available in Java 8+. However, the reduce operation has its own usages and that example emphatically depicts and limitation of the collection API.

Immutable collections

In addition to more verbose syntax, Java collections are in general mutable. This makes it harder (more error prone) to use them in (to share the instance between) multiple threads. Thread safety for collections might help, but as I already mentioned in that case it will not (as the initial empty collection is distributed across diffrent threads). However, in some more functional languages such as Scala, not to mention Haskel, it is quite popular to use immutable collections.

They typically has the “first” element, called head and the rest called tail. Adding a new element before head is cheap as the new head can share the old one as tail (which is immutable, remember?). Of course, in some other use cases (such as putting elements between other elements) they cannot keep up with a classic mutable counterparts. and it can be used in different use cases. Nevertheless, it might be an alternative worth consideration. For Java, there is - among others - the collections API in Vavr .

Mutable reduction operation

To deal with cases where mutable operation are much more memory/CPU effective a dedicated Collector can be written which realizes mutable reduction operation . More info about writing a custom Collector can be found in this interesting blog post by Tomasz Nurkiewicz.

Summary

I hope that you were hard enough to get with me to the end of this (quite long and quite low level) blog post :-).

Using this example with reduce, I wanted to show one of the possible problems while dealing with parallel execution and its explaination. It is not a bug in JDK, but rather a presumption in the (parallel) stream API which - as I observe during my testing classes - are not emphasised enough to be common knowledge. It is good to know your tools and use it properly, as well as extend ones awareness about new and alternative solutions. And remember, state mutation is problematic. Prefer immutable data structures if only possible (and feasible).

Lead photo by Free-Photos, published in Pixabay, Pixabay License.