DataProviderRunner.class do not use rerunFailingTestsCount in parallel

[emaks]

Maven version 3.5.4

I have maven project with next pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>test</groupId>
    <artifactId>test</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>
        <dependency>
            <groupId>com.tngtech.junit.dataprovider</groupId>
            <artifactId>junit4-dataprovider</artifactId>
            <version>2.3</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.0</version>
                <configuration>
                    <rerunFailingTestsCount>1</rerunFailingTestsCount>
                    <parallel>classes</parallel>
                    <useUnlimitedThreads>false</useUnlimitedThreads>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <threadCount>2</threadCount>
                    <rerunFailingTestsCount>1</rerunFailingTestsCount>
                    <includes>
                        <include>Test1.class</include>
                        <include>Test2.class</include>
                    </includes>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

And 2 test classes

import com.tngtech.java.junit.dataprovider.DataProvider;
import com.tngtech.java.junit.dataprovider.DataProviderRunner;
import com.tngtech.java.junit.dataprovider.UseDataProvider;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(DataProviderRunner.class)
public class Test1 {
    private static int value = 0;

    @DataProvider
    public static Object[][] dataProviderAdd() {
        return new Object[][]{
            {0},
        };
    }

    @Test
    @UseDataProvider("dataProviderAdd")
    public void failing1(int value11) {
        if (value != 0) {
            System.out.println("SUCCESS");
            Assert.assertTrue(true);
        } else {
            System.out.println("FAIL");
            value = 1;
            Assert.assertTrue(false);
        }
    }
}

and

import com.tngtech.java.junit.dataprovider.DataProvider;
import com.tngtech.java.junit.dataprovider.DataProviderRunner;
import com.tngtech.java.junit.dataprovider.UseDataProvider;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(DataProviderRunner.class)
public class Test2 {
    private static int value = 0;

    @DataProvider
    public static Object[][] dataProviderAdd() {
        return new Object[][]{
            {0},
        };
    }

    @Test
    @UseDataProvider("dataProviderAdd")
    public void failing2(int value11) {
        if (value != 0) {
            System.out.println("SUCCESS");
            Assert.assertTrue(true);
        } else {
            System.out.println("FAIL");
            value = 1;
            Assert.assertTrue(false);
        }
    }
}

Actual result

If I run mvn test tests will be executed without re-run and fail

Expected result

Tests will pass(with re-run)

Note

If I change

                    <includes>
                        <include>Test1.class</include>
                        <include>Test2.class</include>
                    </includes>

on

                    <includes>
                        <include>Test1.class</include>
                    </includes>

Test will pass

[aaschmid]

Puh ... I at least identified the problem:

• Maven uses a special MatchDescriptions Filter for rerunning failed test cases.
• This causes that junit-dataprovider is not able to filter the tests to executed correctly.
• Therefore no test are executed for the second run.

junit-dataprovider relays on the filter description but the one from MatchDescriptions cannot be parsed for now. It contains a not recognized "OR":

failing1[0: 0](Test1) OR Method failing2[0: 0](Test2)

Having only a single failed test case, though, the filter description does not contain an "OR" and works correctly.

[emaks]

@aaschmid Is the any plans when you could fix this issue?

[aaschmid]

@emaks: there is but I was and I am a bit busy with some complex private issue ...

How urgent is it for you? Sounds like you have a lot blinking test cases, do you?

[emaks]

@aaschmid
unfortunately, yes I do. I use this approach for selenium tests which are testing application with a lot of 3rd-party integrations. Those integrations are unstable and at that moment only one possible solution for this situation - re-run failed tests

[aaschmid]

Understood. Can you try version "2.4-SNAPSHOT" which I just created and uploaded to Maven Central Snapshop Repository?

        <dependency>
            <groupId>com.tngtech.junit.dataprovider</groupId>
            <artifactId>junit4-dataprovider</artifactId>
            <version>2.4-SNAPSHOT</version>
        </dependency>

[emaks]

works as expected with this version

[aaschmid]

Thanks for the feedback. I will release version 2.4 as soon as oss.sonatype.org is available again.

[aaschmid]

Release is done and will appear on MavenCentral within the next hours, thanks @emaks for the feedback.