Discrepancy with combination of (Shared Thread pool and Method Interceptor)

[krmahadevan]

I'm trying to setup a simple/proof of concept project to see the new features regarding the maximization of parallelization.
Im encountering the following error after what seems to be the "first" batch of data-provider cases.

[ERROR] tests.DemoTest.testStrings -- Time elapsed: 0.070 s <<< FAILURE!
java.lang.ClassCastException: class java.util.concurrent.CompletableFuture$AsyncSupply cannot be cast to class java.lang.Comparable (java.util.concurrent.CompletableFuture$AsyncSupply and java.lang.Comparable are in module java.base of loader 'bootstrap')
        at java.base/java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:349)
        at java.base/java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:475)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at java.base/java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1782)
        at java.base/java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:2005)
        at org.testng.internal.invokers.MethodRunner.runInParallel(MethodRunner.java:143)
        at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:950)
        at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:201)
        at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
        at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at org.testng.internal.thread.graph.TestNGFutureTask.run(TestNGFutureTask.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

More details:
The test class:

package tests;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import java.util.ArrayList;
import java.util.List;

public class DemoTest {

    @DataProvider(parallel = true)
    public static Object[] parallelDpStrings() {
        List<String> l = new ArrayList<>();
        for (int i = 0; i < 100; i++) {
            l.add(STR."string \{i}");
        }
        return l.toArray();
    }

    @Test(dataProvider = "parallelDpStrings")
    public void testStrings(String s) throws InterruptedException {
        print(s);
        Thread.sleep(3000);
    }

    public static void print(String s) {
        System.out.println(Thread.currentThread().threadId() + "  " + Thread.currentThread().getStackTrace()[2].getMethodName() + " " + s);
    }
}

testng.xml file:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.1.dtd">
<suite name="testng.xml"
       parallel="instances"
       use-global-thread-pool="true"
       share-thread-pool-for-data-providers="true"
>

    <test name="tng">
        <packages>
            <package name="tests.*"/>
        </packages>
    </test>
</suite>

maven command:

mvn clean test -DthreadCount=10

more console output:

[INFO] --- surefire:3.2.1:test (default-test) @ tng-gtc ---
[INFO] Using auto detected provider org.apache.maven.surefire.testng.TestNGProvider
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
42  testStrings string 8
40  testStrings string 6
35  testStrings string 1
41  testStrings string 7
38  testStrings string 4
36  testStrings string 2
39  testStrings string 5
34  testStrings string 0
37  testStrings string 3
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.457 s <<< FAILURE! -- in TestSuite
[ERROR] tests.DemoTest.testStrings -- Time elapsed: 0.070 s <<< FAILURE!
java.lang.ClassCastException: class java.util.concurrent.CompletableFuture$AsyncSupply cannot be cast to class java.lang.Comparable (java.util.concurrent.CompletableFuture$AsyncSupply and java.lang.Comparable are in module java.base of loader 'bootstrap')
        at java.base/java.util.concurrent.PriorityBlockingQueue.siftUpComparable(PriorityBlockingQueue.java:349)
        at java.base/java.util.concurrent.PriorityBlockingQueue.offer(PriorityBlockingQueue.java:475)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
        at java.base/java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1782)
        at java.base/java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:2005)
        at org.testng.internal.invokers.MethodRunner.runInParallel(MethodRunner.java:143)
        at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:950)
        at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:201)
        at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
        at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at org.testng.internal.thread.graph.TestNGFutureTask.run(TestNGFutureTask.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   DemoTest.testStrings » ClassCast class java.util.concurrent.CompletableFuture$AsyncSupply cannot be cast to class java.lang.Comparable (java.util.concurrent.CompletableFuture$AsyncSupply and java.lang.Comparable are in module java.base of loader 'bootstrap')
[INFO] 
[ERROR] Tests run: 10, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.317 s
[INFO] Finished at: 2024-02-29T15:59:43+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.2.1:test (default-test) on project tng-gtc: There are test failures.

Did i miss something?

Originally posted by @dmirica-gpsw in #3080

[krmahadevan]

Analysis on this so far:

• TestNG resorts to creating a PriorityBlockingQueue for the following two conditions [ Either the number of interceptors are greater than 1 or if it finds atleast 1 test that has a non zero priority ]
• When TestNG is used along with allure (As in this case), Allure has a service loader powered listener called io.qameta.allure.testng.AllureTestNg which is also a org.testng.IMethodInterceptor and so TestNG assumes that sorting on priority is needed and plugs in a PriorityBlockingQueue

TestNG needs to handle the below two use cases when using the globally shared thread pool feature

• There are more than 1 method interceptors available within TestNG
• There are tests that use priority and thus require for a soft ordering.