Maven ignores tests defined as ArchTest fields

[Scaronthesky]

I'm under the impression that Maven seems to ignore ArchUnit tests which are defined as fields with the @archtest annotation.

I copied some of the CodingRulesTest from the ArchUnit-Examples project, e. g. "No Access to Standard streams". These tests work fine when I execute them with IntelliJ. Maven completely ignores them, though. Maven executes only tests which are defined as "proper" test methods.

Is this a configuration issue or a bug? I had a look at the documentation and issues but could not find anything.

import static com.tngtech.archunit.library.GeneralCodingRules.NO_CLASSES_SHOULD_ACCESS_STANDARD_STREAMS;
import com.tngtech.archunit.core.domain.JavaClasses;
import com.tngtech.archunit.junit.AnalyzeClasses;
import com.tngtech.archunit.junit.ArchTest;
import com.tngtech.archunit.lang.ArchRule;

@AnalyzeClasses(packages = "de.package")
public class CodingRulesTest {

  // Will silently be ignored by Maven
  @ArchTest
  public final ArchRule no_access_to_standard_streams = NO_CLASSES_SHOULD_ACCESS_STANDARD_STREAMS;

  // Will be executed
  @ArchTest
  public void assertNoAccessToStandardStreams(JavaClasses classes) {
    NO_CLASSES_SHOULD_ACCESS_STANDARD_STREAMS.check(classes);
  }
}

In the examples project the ArchRule fields are private. I made the public but it makes no difference. They are still ignored.

maven-surefire-plugin:3.1.2
Using configured provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
com.tngtech.archunit:archunit-junit5-api:1.1.0
com.tngtech.archunit:archunit-junit5-engine:1.1.0
org.junit.jupiter:junit-jupiter-api:5.10.0

[hankem]

Thanks for raising the issue! I couldn't exactly reproduce your problem, but saw that ArchUnit 1.1.0 is not fully compatible with JUnit 5.10, so I assume that your issue is related to that.

This example

git clone https://github.com/TNG/ArchUnit-Examples.git

cd ArchUnit-Examples/example-junit5

cat > pom.xml << EOF
<?xml version="1.0"?>
<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.tngtech.archunit</groupId>
  <artifactId>example-junit5</artifactId>
  <version>1.1.0</version>
  <properties>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.tngtech.archunit</groupId>
      <artifactId>archunit-junit5</artifactId>
      <version>1.1.0</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-api</artifactId>
      <version>5.10.0</version>
      <scope>test</scope>
    </dependency>

    <!-- These are the 'production' dependencies of the Demo src/main/java files -> just for Demo purposes, otherwise irrelevant -->
    <dependency>
      <groupId>javax.annotation</groupId>
      <artifactId>javax.annotation-api</artifactId>
      <version>1.3.2</version>
    </dependency>
    <dependency>
      <groupId>org.apache.geronimo.specs</groupId>
      <artifactId>geronimo-ejb_3.1_spec</artifactId>
      <version>1.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.geronimo.specs</groupId>
      <artifactId>geronimo-jpa_2.0_spec</artifactId>
      <version>1.0</version>
    </dependency>
    <dependency>
      <groupId>joda-time</groupId>
      <artifactId>joda-time</artifactId>
      <version>2.0</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-beans</artifactId>
      <version>5.2.2.RELEASE</version>
    </dependency>
    <dependency>
      <groupId>com.google.inject</groupId>
      <artifactId>guice</artifactId>
      <version>4.2.3</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.1.2</version>
      </plugin>
    </plugins>
  </build>
</project>
EOF

mvn test

fails with

java.lang.NoSuchMethodError: 'java.util.Set org.junit.platform.engine.TestDescriptor.getAncestors()'

– which can be resolved by downgrading junit-jupiter-api to 5.9.3 or upgrading archunit-junit5 to a version built against junit-platform-engine:1.10.0.

[hankem]

The issue of my example above can also be solved by explicitly adding the new junit-platform-engine:

    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-engine</artifactId>
      <version>1.10.0</version>
      <scope>test</scope>
    </dependency>

@Scaronthesky, does this help in your case, too?

[Scaronthesky]

Hmm, I don't know what changed between last week and now, but I can't seem to reproduce this issue anymore. It works even without adding the aformentioned "org.junit.platform:junit-platform-engine" dependency. Maybe some one else is suffering from the same problem in the future.

For what it's worth I ran into the same issue as you while trying to create a minimum working example. After adding the "org.junit.platform:junit-platform-engine" dependency the tests can be run and both tests (field and method) are executed. Thanks for your support.

[enver-haase]

I am bitten by this bug too - unfortunately, adding the junit-platform-engine:1.10.0 results in an exception.

java.lang.NoClassDefFoundError: org/junit/platform/engine/reporting/OutputDirectoryProvider
        at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:47)
        at org.apache.maven.surefire.api.util.DefaultScanResult.applyFilter(DefaultScanResult.java:87)

Leaving out the version number gives me a working configuration with junit-platform-engine:1.12.2 but then the problem persists and no field-style tests are being executed.

[enver-haase]

The problem is present with maven-surefire-plugin:3.5.3 .
Downgrading to 3.5.2 makes the problem go away.

[LoboTheDark]

The problem is present with maven-surefire-plugin:3.5.3 . Downgrading to 3.5.2 makes the problem go away.

Thanks, it saves my day.