Tricks & Tips on Solid Soft

Mysteriously broken reduction operation on parallel streams in Java (explained)

Marcin Zajączkowski — Thu, 12 Mar 2020 14:00:00 +0200

How to not get into trouble writing custom reduction operation (.reduce()) on (parallel) streams with Java 8+ (including Java 14).

Introduction

Recently…, well this article has spent years in my freezer for the blog posts. Originally, I have encountered this problem back in 2014 while preparing my Java 8 training session (recently renamed to Java - functional thinking ). Then, I noticed that it is possible to achieve interesting results writing a custom reduce operation implementation when a parallel stream is used. For various reasons, the partially written article was waiting for its completion around 6 years. Time flies, however, the case is still valid with the latest - as a time of writing - Java 14. So, let’s check it out.

Our case - simple number concatenation using reduce

As a short introduction, a reduction operation (also called a fold) takes input elements (here of a stream) and collects them into combined summary result. Some well know specialized reduction variants, available in the Stream API, include, inter alia, sum(), min() and count(). Nevertheless, we can write our custom, non-standard reduction operation (if needed).

Let’s assume we would like to concatenate a list of numbers into a string. A case which could be easier to achieve in other ways, but simple enough to do not focus on realized logic, but rather a nuances of a reduction operation itself. A reduce method on stream can be used:

@Test
public void shouldConcatNumbersUsingReduce() {
    //given
    final List<Integer> input = Arrays.asList(1, 2, 3, 4);

    //when
    final String result = input.stream()
            .reduce(new StringBuilder(),
                    StringBuilder::append,
                    StringBuilder::append)
            .toString();

    //then
    assertThat(result).isEqualToIgnoringCase("1234");   //passes
}

Very compact and does its work. Numbers are concatenated. Similar implementations are common in internet and everything is ok unless you try to use it with the parallel stream:

@Test
public void shouldConcatNumbersUsingReduceParallel() {   //broken!
    //given
    List<Integer> input = Arrays.asList(1, 2, 3, 4);

    //when
    String result = input.parallelStream()
            .reduce(new StringBuilder(),
                    StringBuilder::append,
                    StringBuilder::append)
            .toString();

    //then
    assertThat(result).isEqualToIgnoringCase("1234");   //ups!
}

This test fails with the assertion error similar to that:

java.lang.AssertionError: 
Expecting:
 <"343421343421343421343421">
to be equal to:
 <"1234">

It looks strange. Very often incomprehensible things are caused by some race condition in the parallel code which matches parallelStream(). But first let’s analyze what happens in a reduction operation step by step (if you are savvy in sequential reduction operations skip to the subsequent section).

Reduction in sequential stream

If you are not very familiar with method references the example can look a little bit cryptic. Let’s rewrite it using lambda expressions (instead method references):

String result = input.stream()
        .reduce(new StringBuilder(),
                (acc, x) -> acc.append(x),      //accumulation
                (sb1, sb2) -> sb1.append(sb2))  //combining
        .toString();

The new StringBuilder is used as an initial value (identity). For every element from an input list the second line in reduce will be called ((acc, x) -> acc.append(x)) with acc (an accumulator) containing concatenated already processed elements and x the current number. x is appended to the previously concatenated elements.

From the algorithm analysis, the expected values in subsequent iterations should be as follows:

iteration	0	1	2	3
acc	"”	“1”	“12”	“123”
x	1	2	3	4
returned	“1”	“12”	“123”	“1234”

Is this ok? Let’s verify it in code.

As debugging (prospectively) concurrent code is hard, especially in a blog post, let’s use plain old debug outputs, well known from C.

String result = input.stream()
    .reduce(
        new StringBuilder(),
        (acc, x) -> {
            String accString = acc.toString();
            StringBuilder ret = acc.append(x);
            log.info("Accumulation: '{}', x: '{}', returned: '{}'", format("%4s", accString), 
                                     format("%4s", x), format("%4s", ret));
            return ret;
        },
        (sb1, sb2) -> {
            String sb1String = sb1.toString();
            StringBuilder ret = sb1.append(sb2);
            log.info("Combining: sb1: '{}', sb2: '{}', returned: '{}'", format("%4s", sb1String),
                                 format("%4s", sb2), format("%4s", ret));
            return ret;
        })
    .toString();

We slightly complicated the code, but it is only for the test (debug) purpose. The output is as expected:

18:09:05.839 [main] - Accumulation: acc: '    ', x: '   1', returned: '   1'
18:09:05.844 [main] - Accumulation: acc: '   1', x: '   2', returned: '  12'
18:09:05.844 [main] - Accumulation: acc: '  12', x: '   3', returned: ' 123'
18:09:05.844 [main] - Accumulation: acc: ' 123', x: '   4', returned: '1234'

Let’s skip for a while the log from the combining operation (line 14). It is not used in sequential streams.

In the output, we also clearly see that all operations take place in the main thread. Which is expected for sequential synchronous stream processing.

Reduction in parallel stream - behavior in working implementation

In a parallel stream, accumulation operations can be performed in different threads. Those partial results are later combined using lambda expression passed as the third argument (a combiner) in a reduce method.

Let’s take a look at the diagnostic messages emitted during a sample execution with parallel stream.

One important note here. The output is from the correctly implemented reduction operation (presented at the end of the blog post) rather than the version above to make the whole thing easier to grasp.

[ForkJoinPool.commonPool-worker-3] - Accumulation: acc: '    ', x:   '   1', returned: '   1'
[ForkJoinPool.commonPool-worker-1] - Accumulation: acc: '    ', x:   '   2', returned: '   2'
[main]                             - Accumulation: acc: '    ', x:   '   3', returned: '   3'
[ForkJoinPool.commonPool-worker-1] - Combining   : sb1: '   1', sb2: '   2', returned: '  12'
[ForkJoinPool.commonPool-worker-2] - Accumulation: acc: '    ', x:   '   4', returned: '   4'
[ForkJoinPool.commonPool-worker-2] - Combining   : sb1: '   3', sb2: '   4', returned: '  34'
[ForkJoinPool.commonPool-worker-2] - Combining   : sb1: '  12', sb2: '  34', returned: '1234'

Let’s analyze it line by line.

In the first 2 lines:

1
2

[ForkJoinPool.commonPool-worker-3] - Accumulation: acc: '    ', x:   '   1', returned: '   1'
[ForkJoinPool.commonPool-worker-1] - Accumulation: acc: '    ', x:   '   2', returned: '   2'

the accumulation operation is performed in parallel on the two first elements on the list. In the forth line those partial results are combined into a larger list:

`4`	`[ForkJoinPool.commonPool-worker-1] - Combining : sb1: ' 1', sb2: ' 2', returned: ' 12'`

The same situation, for the rest of the items, takes place in the line 3, 5 and 6.

[main]                             - Accumulation: acc: '    ', x:   '   3', returned: '   3'
...
[ForkJoinPool.commonPool-worker-2] - Accumulation: acc: '    ', x:   '   4', returned: '   4'
[ForkJoinPool.commonPool-worker-2] - Combining   : sb1: '   3', sb2: '   4', returned: '  34'

And the end, two partial result: ‘12’ and ‘34’ are combined together to provide the final result:

`7`	`[ForkJoinPool.commonPool-worker-2] - Combining : sb1: ' 12', sb2: ' 34', returned: '1234'`

Do you have - after logging at the listings - any suspicious what could be wrong with the original implementation? With the debug statements it much easier to spot it.

Reduction in parallel stream with widespread (broken) implementation

Let’s recall the widespread (broken when running in parallel) implementation once more:

String result = input.parallelStream()
        .reduce(new StringBuilder(),
                (acc, x) -> acc.append(x),      //accumulation
                (sb1, sb2) -> sb1.append(sb2))  //combining
        .toString();

together with its debug enhanced variant:

String result = input.parallelStream()
    .reduce(
        new StringBuilder(),
        (acc, x) -> {
            String accString = acc.toString();
            StringBuilder ret = acc.append(x);
            log.info("Accumulation: '{}', x: '{}', returned: '{}'", format("%4s", accString), 
                                     format("%4s", x), format("%4s", ret));
            return ret;
        },
        (sb1, sb2) -> {
            String sb1String = sb1.toString();
            StringBuilder ret = sb1.append(sb2);
            log.info("Combining: sb1: '{}', sb2: '{}', returned: '{}'", format("%4s", sb1String),
                                 format("%4s", sb2), format("%4s", ret));
            return ret;
        })
    .toString();

The output from the parallel processing execution:

10:42:53.334 [ForkJoinPool.commonPool-worker-5]  - Accumulation: acc: '   3', x: '   4', returned: '  34'
10:42:53.334 [ForkJoinPool.commonPool-worker-23] - Accumulation: acc: ' 342', x: '   1', returned: '3421'
10:42:53.334 [ForkJoinPool.commonPool-worker-19] - Accumulation: acc :'  34', x: '   2', returned: '3421'
10:42:53.334 [main]                              - Accumulation: acc :'    ', x: '   3', returned: '   3'
10:42:53.339 [ForkJoinPool.commonPool-worker-19] - Combining: sb1: '3421', sb2: '342134213421', returned: '342134213421'
10:42:53.339 [main]                              - Combining: sb1: '3421', sb2: '342134213421', returned: '342134213421'
10:42:53.340 [main]                              - Combining: sb1: '342134213421', sb2: '342134213421342134213421', returned: '342134213421342134213421'

The strange things start to happen from the first line. Instead of having an empty accumulator (the “acc” variable) there is “3”. In the second line there is already “342” (still instead of an empty StringBuilder). We discover empty accumulator in the 3rd line, which anyway could be the 1st line (it is a matter of logging operations from multiple threads - they were executed at “the same moment”). Nonetheless, all the first 4 lines should have the accumulator empty. It clearly shows there is a problem with a mutable data structure shared across the threads, in both accumulation and combining phrases.

A look at the StringBuilder documentation ensures us that “append” adds a provided value to the StringBuilder and returns its instance (aka this). Mutating input variables is not recommended in the stream processing, however the root of the problem is a fact that initial “neutral” value (identity) passes to every initial accumulation operation is one and the same StringBuilder instance. It is passed as the first element of the “reduce” method execution.

Someone could say: “let’s use StringBuffer - its operations are thread safe”. However, it will not fix the situation here. Having multiple threads operating on the same structure (even thread safe) is also broken assuming that all initial operations assumes a private accumulator instance to create an independent list of elements which will be concatenated into the biggest one in the following steps.

Reduction in parallel stream - working implementation

Main case

With StringBuffer and string concatenation the only required modification is not mutating the input objects:

String result = input.parallelStream()
        .reduce(
                new StringBuilder(),
                (acc, x) -> new StringBuilder(acc).append(x),
                (sb1, sb2) -> new StringBuilder(sb1).append(sb2))
        .toString();

In every iteration the new StringBuilder instance is created. Thanks to that there is no mutation of the initial identity element shared across threads. In general mutation of the stream elements (or doing side effects in general) can lead to some unpredictable situations, especially in the parallel execution.

Java API limitations

The above change solves the problem (at the cost of greatly increased number of created objects, but let’s skip it for a moment). Unfortunately, many structures, especially those from Java Collection Framework which remembers Java 1.2 (releases 1998 - over 20 years ago!), does not provide so friendly API to perform operations with the method chaining syntax/approach.

Let’s take a look at the (artificial) working example of using the java.util.List API in a reduce operation simulating mapping to double every element of the input list:

List<Integer> doubledValues = stream
    .reduce(Collections.emptyList(),
            (list, x) -> {
                List<Integer> resultList = new ArrayList<>(list);
                resultList.add(x * 2);
                return resultList;
            },
            (list1, list2) -> {
                List<Integer> resultList = new ArrayList<>(list1);
                resultList.addAll(list2);
                return resultList;
            });

True one-liners with a method chaining would make this code more readable.

Please note that the aforementioned logic (doubling list elements) can be implemented easily in native operations available in Java 8+. However, the reduce operation has its own usages and that example emphatically depicts and limitation of the collection API.

Immutable collections

In addition to more verbose syntax, Java collections are in general mutable. This makes it harder (more error prone) to use them in (to share the instance between) multiple threads. Thread safety for collections might help, but as I already mentioned in that case it will not (as the initial empty collection is distributed across diffrent threads). However, in some more functional languages such as Scala, not to mention Haskel, it is quite popular to use immutable collections.

They typically has the “first” element, called head and the rest called tail. Adding a new element before head is cheap as the new head can share the old one as tail (which is immutable, remember?). Of course, in some other use cases (such as putting elements between other elements) they cannot keep up with a classic mutable counterparts. and it can be used in different use cases. Nevertheless, it might be an alternative worth consideration. For Java, there is - among others - the collections API in Vavr .

Mutable reduction operation

To deal with cases where mutable operation are much more memory/CPU effective a dedicated Collector can be written which realizes mutable reduction operation . More info about writing a custom Collector can be found in this interesting blog post by Tomasz Nurkiewicz.

Summary

I hope that you were hard enough to get with me to the end of this (quite long and quite low level) blog post :-).

Using this example with reduce, I wanted to show one of the possible problems while dealing with parallel execution and its explaination. It is not a bug in JDK, but rather a presumption in the (parallel) stream API which - as I observe during my testing classes - are not emphasised enough to be common knowledge. It is good to know your tools and use it properly, as well as extend ones awareness about new and alternative solutions. And remember, state mutation is problematic. Prefer immutable data structures if only possible (and feasible).

Lead photo by Free-Photos, published in Pixabay, Pixabay License.

PIT, JUnit 5 and Gradle - with just one extra line of configuration

Marcin Zajączkowski — Thu, 27 Feb 2020 00:00:00 +0000

Discover dead simple, improved PIT and JUnit 5 configuration in Gradle (with gradle-pitest-plugin 1.4.7+).

JUnit 5 is undeniably more and more popular nowadays. While there is a dedicated plugin for PIT for JUnit 5 and it has been supported by gradle-pitest-plugin for years, it was required to add a few lines of boilerplate code to achieve that. Recently, I’ve got a question if it could be simplified. I liked it. Challenge accepted :-).

Generic approach with ‘buildscript {}’

First, take a look at the generic approach with the buildscrip {} code block, which remembers times of Gradle 0.x:

buildscript {
   repositories {
       mavenCentral()
       gradlePluginPortal() //optionally, if any plugin is not available in Maven Central
   }
   configurations.maybeCreate('pitest')
   dependencies {
       classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:1.4.6'
       pitest 'org.pitest:pitest-junit5-plugin:0.12'
   }
}

apply plugin: 'java'
apply plugin: 'info.solidsoft.pitest'

pitest {
    testPlugin = 'junit5'
}

Just 3 extra lines. Acceptable, however buildscript {} for plugin configuration is somehow verbose by itself.

Modern approach with ‘plugins {}‘
(with older gradle-pitest-plugin)

The modern variant with plugins {} should be shorter:

buildscript {
   repositories {
       mavenCentral()
   }
   configurations.maybeCreate('pitest')
   dependencies {
       pitest 'org.pitest:pitest-junit5-plugin:0.12'
   }
}

plugins {
    id 'java'
    id 'info.solidsoft.pitest' version '1.4.6'
}

pitest {
    testPlugin = 'junit5'
}

Unfortunately, the compact syntax of the plugin {} block is wasted by a need to add an extra dependency pitest-junit5-plugin used by gradle-pitest-plugin in runtime in the buildscript {} block - 10 lines extra. Highly disappointing ;-).

Modern improved approach with ‘plugins {}‘
and gradle-pitest-plugin 1.4.7+

With just released gradle-pitest-plugin 1.4.7 we can forget about all the boilerplate code:

plugins {
    id 'java'
    id 'info.solidsoft.pitest' version '1.4.7'
}

pitest {
    //adds dependency to org.pitest:pitest-junit5-plugin and sets "testPlugin" to "junit5"
    junit5PluginVersion = '0.12'
}

Just one line junit5PluginVersion = '0.12' which under the hood adds a pitest-junit5-plugin dependency in the required versions and activates junit5 as testPlugin used by PIT. Doesn’t it look nice? :-)

Summary

In this short blog post I presented how the configuration of the PIT, JUnit 5 and Gradle(-pitest-plugin) could be simplified with just a few changes in the plugin itself. All thanks to the question by John Scancella and the idea that sprang into my mind how to implement it in the smart way.

Therefore, I encourage you to report (sensible) ideas for improvements and things that limit you in the projects you use (or even better a pull/merge request after the initial discussion with the maintainer). Maybe it will implemented (accepted) for common good :-).

Auto-switchable multiple Git identities on one computer - Git Tricks #1

Marcin Zajączkowski — Tue, 27 Aug 2019 00:00:00 +0000

Have you even committed to Git using wrong email address working on/for different projects/companies? Luckily with a little configuration Git can auto-switch the identities for you.

(Too long) introduction and reasoning

Being an (experienced) IT professional can give you an opportunity to work on different things in the same time frame. For instance, in addition to work for the main client, I do some consultancy stuff with code quality & testing and Continuoues Delivery for other companies. What’s more, I also conduct training sessions (will a lot of exercises with code) and work on my own and other’s FOSS projects. Usually, it is convenient to do it on the same computer. It can happen to commit something with the wrong email address (to be detected later on by an external auditor ;-) ) and a push with force to a remote master after rebase is not the best possible solution :-). I started with some Fish -based script to deal with it, but in the end I have found a built-in mechanism in Git itself.

Disclaimer. This particular blog post doesn’t treat about anything new or revolutionary. However, I’ve been living in unawareness long enough to give my blog readers a chance to know that mechanism right now.

Project situation

My ~/Code directory structure could be simplified to something like that:

Code/
├── gradle-nexus-staging-plugin
├── mockito-java8
├── spock-global-unroll
├── ...
├── external-foss
│   ├── ...
│   ├── awaitility
│   └── spock
├── training
│   ├── ...
│   ├── code-testing
│   ├── java11
│   └── jenkins-as-code
└── work
    ├── ...
    ├── codearte
    └── companyX

The goal is to keep an email address in the company’s projects appropriate.

Solution

The first idea which can spring to your mind is to manually override git config user.email "..." in every clonned company repository. However, it’s tedious and error prone (especially in the microservice-based architecture :) ).

Luckily, one of the features introduces in Git 2.13.0 (long time ago - May 2017) is a conditional configuration applying (aka Conditional includes ). Armed with it our task is pretty simple.

First, keep your default name and email defined in the [user] section in ~/.gitconfig:

1
2
3

[user]
  name = Marcin Zajączkowski
  email = foss.hacker@mydomain.example.com

Next, create a company related config file in the home directory, e.g. ~/.gitconfig-companyX with just overriden email address:

1
2

[user]
  email = marcin.zajaczkowski@companyx.example.com

Glue it together by placing in ~/.gitconfig the following code:

1
2

[includeIf "gitdir:~/Code/work/companyX/"]
  path = .gitconfig-companyX

The .gitconfig-companyX is applied only if a current path prefix matches the one defined in includeIf. Simple, yet powerful. That mechanism can be also used to configure different things - such as conditionally using GPG to sign your commits.

Btw, being a paranoid you can even remove the email field from the root configuration to be notified if you forgot to add an email for any new company you colaborate with.

Summary

Thanks to the includeIf mechanism you will never (*) again commit to a repo with wrong name or email. That and other Git related tricks and commands/aliases (such as collected in git-kurka make it easier to focus on delivering the stuff :).

Feel free to leave your favorite Git tips & tricks in the comments.

Update. From the comment made on priv:

Git works great in that case, but for more generic changes in the environment variables Wybczu suggested very powerful direnv (with support for bash, zsh, tcsh, fish and elvish)

The lead photo based on the Elias Sch ‘s work published in Pixabay, Pixabay License.

Simplify integration testing of legacy applications with Spock 1.2

Marcin Zajączkowski — Fri, 14 Sep 2018 10:00:00 +0200

Learn how leverage Spock 1.2 to slice a Spring context of a legacy application writing integration tests.

Have you ever wanted, having some legacy application which you were starting to work on, to write some tests to get know what is going on and possibly be notified about regressions? That feeling when you want to instantiate a single class and it fails with NullPointerException. 6 replaced (with difficulty) dependencies later there are still some errors from the classes that you haven’t heard about before. Sounds familiar?

There are different techniques to deal with hidden dependencies. There is the whole dedicated book about that (and probably a few other that I haven’t read). Occasionally, it may be feasible to start with the integration tests and run through some process. It may be even more “entertaining” to see what exotic components are required to just setup the context, even if they are completely not needed in our case. Thank you (too wide and carelessly used) @ComponentScan :).

Injecting stubs/mocks inside the test context is a way to go as an emergency assistance (see the last paragraph, there are better, yet harder approaches). It can be achieved “manually” with an extra bean definition with the @Primary annotation (usually a reason to think twice before doing that) for every dependency at which level we want to make a cut of (or for every unneeded bean which is instantiated by the way). @MockBean placed on a field in a test is more handy, but still, it is needed to define a field in our tests and put the annotation on it (5? 10? 15 beans?). Spock 1.2 introduces somehow less know feature which may be useful here - @StubBeans.

It can be used to simply provide a list of classes which (possible) instances should be replaced with stubs in the Spring test context. Of course before the real objects are being instantiated (to prevent for example NPE in a constructor). Thanks to that up to several lines of stubbing/mock injections:

@RunWith(SpringRunner.class) //Spring Boot + Mockito
@SpringBootTest //possibly some Spring configuration with @ComponentScan is imported in this legacy application
public class BasicPathReportGeneratorInLegacyApplicationITTest { //usual approach

    @MockBean
    private KafkaClient kafkaClientMock;

    @MockBean
    private FancySelfieEnhancer fancySelfieEnhancerMock;

    @MockBean
    private FastTwitterSubscriber fastTwitterSubscriberMock;

    @MockBean
    private WaterCoolerWaterLevelAterter waterCoolerWaterLevelAterterMock;

    @MockBean
    private NsaSilentNotifier nsaSilentNotifierMock;

    //a few more - remember, this is legacy application, genuine since 1999 ;)
    //...

    @Autowired
    private ReportGenerator reportGenerator;

    @Test
    public void shouldGenerateEmptyReportForEmptyInputData() {
        ...
    }
}

can be replaced with just one (long) line:

@SpringBootTest //possibly some Spring configuration with @ComponentScan is imported
@StubBeans([KafkaClient, FancySelfieEnhancer, FastTwitterSubscriber, WaterCoolerWaterLevelAterter, NsaSilentNotifier/(, ... */])
  //all classes of real beans which should be replaced with stubs
class BasicPathReportGeneratorInLegacyApplicationITSpec extends Specification {

    @Autowired
    private ReportGenerator reportGenerator

    def "should generate empty report for empty input data"() {
        ....
    }
}

(tested with Spock 1.2-RC2)

It’s worth to mention that @StubBeans is intended just to provide placeholders. In a situation it is required to provide stubbing and/or an invocation verification @SpringBean or @SpringSpy (also introduced in Spock 1.2) are better. I wrote more about it in my previous blog post .

There is one important aspect to emphasize. @StubBeans are handy to be used in a situation when we have some “legacy” project and want to start writing integration regression tests quickly to see the results. However, as a colleague of mine Darek Kaczyński brightly summarized, blindly replacing beans which “explode” in tests is just “sweeping problems under carpet”. After the initial phase, when we are starting to understand what is going on, it is a good moment to rethink the way the context - both in production and in tests - is created. The already mentioned too wide @ComponentScan is very often the root of all evil. An ability to setup a partial context and put it together (if needed) is a good place to start. Using @Profile or conditional beans are the very powerful mechanisms in tests (and not only there). @TestConfiguration and proper bean selection to improve context caching are something worth to keep in your mind. However, I started this article to present the new mechanism in Spock which might be useful in some cases and I want to keep it short. There could be an another, more generic blog post just about managing the Spring context in the integration tests. I have to seriously thing about it :).

Deploy to Maven Central using API key (aka auth token)

Marcin Zajączkowski — Tue, 08 Sep 2015 10:00:00 +0200

How to communicate with Maven Central/Nexus without using the password kept locally unencrypted (especially with Gradle, but not limited to it).

Rationale

Unfortunately, Gradle (and many other build tools) does not provide any mechanism to locally keep passwords encrypted (or at least encoded). Without that even such a simple activity like showing your global Gradle configuration (~/.gradle/gradle.properties) to a colleague is uncomfortable, not to mention more serious risks associated with storing passwords on a disk in a plain-text form (see among others Sony Pictures Entertainment hack ). It is Gradle, so with all Groovy magic under the hood it would be possible to implement an integration with a system keyring on Linux to fetch a password, but I’m not aware of any existing plugin/mechanism to do that and I would rather prefer not to write it.

Another issue is that nowadays, in the world of ubiquitous automation and cloud environments it is common to use API keys which allow to perform given operation(s). However, its lost doesn’t provide an attacker a possibility to hijack the account (e.g. token cannot be used neither to log into an administration panel nor to change of email or password which requires additional authentication).

It is very important if you need to keep valid credentials on a CI server to make automatic or even continuous releases. Thanks to my ~~gradle-nexus-staging-plugin~~ (Update 2021. Check out my new plugin gradle-nexus-publish-plugin ) there is no need to do any manual steps in Nexus GUI to promote artifacts to Maven Central, so this was the next issue I wanted to deal with for my private and our FOSS projects in Codearte .

Nexus API key generation

Internet search for “maven central api key” wasn’t helpful, so I started digging into Nexus REST API documentation and I’ve found that in fact there is a (non widely known) way to generate and use an API key (aka an auth token).

Log into Nexus hosting Sonatype OSS Repository Hosting (or your own instance of Nexus).
Click on your login name in right-upper corner and choose “Profile”.
From the drop-down list with “Summary” text select “User Token”.
Click “Access User Token”.

Enter your password
Copy and paste your API username and API key (into your ~/.gradle/gradle.properties file or a CI server configuration).
Work as usual with a little safer way.

Summary

It is good that using API keys is possible to deploy artifacts to Maven Central/Nexus and it is very easy to set it up. Someone could argue that the permission policy is coarse-grained (nothing or all operations except password/email change), but in my opinion it seems to be enough for the artifact repository system class. In addition, such an approach should work also with Sbt, Ivy, Leiningen and everything else that tries to upload artifacts into Maven Central (including Maven itself by removing limitations of the master password encryption with settings-security.xml). Hopefully, that post will make it widely known.

Tricks & Tips on Solid Soft

Mysteriously broken reduction operation on parallel streams in Java (explained)

Introduction

Our case - simple number concatenation using reduce

Reduction in sequential stream

Reduction in parallel stream - behavior in working implementation

Reduction in parallel stream with widespread (broken) implementation

Reduction in parallel stream - working implementation

Main case

Java API limitations

Immutable collections

Mutable reduction operation

Summary

PIT, JUnit 5 and Gradle - with just one extra line of configuration

Generic approach with ‘buildscript {}’

Modern approach with ‘plugins {}‘ (with older gradle-pitest-plugin)

Modern improved approach with ‘plugins {}‘ and gradle-pitest-plugin 1.4.7+

Summary

Auto-switchable multiple Git identities on one computer - Git Tricks #1

(Too long) introduction and reasoning

Project situation

Solution

Summary

Simplify integration testing of legacy applications with Spock 1.2

Deploy to Maven Central using API key (aka auth token)

Rationale

Nexus API key generation

Summary

Modern approach with ‘plugins {}‘
(with older gradle-pitest-plugin)

Modern improved approach with ‘plugins {}‘
and gradle-pitest-plugin 1.4.7+