Discover test configuration on enclosing class for nested test class [SPR-15366]

[spring-projects-issues]

Sam Brannen opened SPR-15366 and commented

Status Quo

Spring's support for JUnit Jupiter already supports detection of test configuration (e.g., (@ContextConfiguration) on @Nested classes.

However, if a @Nested class does not declare its own test configuration, Spring will not find the configuration from the enclosing class.

See also this discussion on Stack Overflow regarding nested test classes and @Transactional.

Proposal

Inspired by issue 8 from the spring-test-junit5 project, it would perhaps be desirable if the Spring TestContext Framework would discover test configuration on an enclosing class for a @Nested test class.

Deliverables

• Introduce @NestedTestConfiguration with EnclosingConfiguration enum supporting INHERITED and OVERRIDE modes.
• Make the default EnclosingConfiguration mode globally configurable via SpringProperties
• Switch the default EnclosingConfiguration mode to INHERITED
• @ContextConfiguration / @ContextHierarchy
• @ActiveProfiles
• @TestPropertySource / @TestPropertySources
• @WebAppConfiguration
• @TestConstructor
• @BootstrapWith
• @TestExecutionListeners
• @DirtiesContext
• @Transactional
• @Rollback / @Commit
• ❌ @Sql / @SqlConfig / @SqlGroup
  • see Discover @Sql, @SqlConfig, @SqlGroup, & @SqlMergeMode on enclosing class for nested test class  #25913
• ❌ Document @NestedTestConfiguration support in the reference manual
  • see Document @NestedTestConfiguration support #25912
• ❌ Document @NestedTestConfiguration support and the switching of the default mode to INHERITED in the upgrade notes in the wiki
  • see Document @NestedTestConfiguration support #25912

---

Affects: 5.0

Issue Links:

• Support nested test classes with SpringClassRule & SpringMethodRule [SPR-14150] #18722 Support nested test classes with SpringClassRule & SpringMethodRule
• JUnit Jupiter @Nested class cannot share enclosing class's ApplicationContext if nested class is deemed to be a configuration candidate [SPR-16595] #21136 JUnit Jupiter @Nested class cannot share enclosing class's ApplicationContext if nested class is deemed to be a configuration candidate

6 votes, 10 watchers

[spring-projects-issues]

Sam Brannen commented

Current work is being performed in the following branch:

https://github.com/sbrannen/spring-framework/tree/SPR-15366

[spring-projects-issues]

Andy Wilkinson commented

Sam asked me to comment on this with reference to this Spring Boot issue. In looking at that issue I observed that the following test would start two separate application contexts:

@ExtendWith(SpringExtension.class)
@SpringBootTest
public class TwoDifferentContextsTests {

	@Autowired
	ApplicationContext context;

	@Nested
	class NestedTests {

		@Test
		public void test() {
		}

	}

}

One context is started for TwoDifferentContextsTests. Due to @SpringBootTest, it finds the @SpringBootApplication-annotated class and uses that as the basis for the context's configuration. A second context is started for NestedTests as it hasn't inherited @SpringBootTest from the enclosing class and, therefore, has different configuration. In fact, NestedTests doesn't really have any configuration. It's only able to have a context created for it due to the context customiser factories declared by spring-boot-test in spring.factories. It enters the if block rather than throwing in this code in AbstractDelegatingSmartContextLoader:

// If neither of the candidates supports the mergedConfig based on resources but
// ACIs or customizers were declared, then delegate to the annotation config
// loader.
if (!mergedConfig.getContextInitializerClasses().isEmpty() || !mergedConfig.getContextCustomizers().isEmpty()) {
	return delegateLoading(getAnnotationConfigLoader(), mergedConfig);
}

// else...
throw new IllegalStateException(String.format(
		"Neither %s nor %s was able to load an ApplicationContext from %s.", name(getXmlLoader()),
		name(getAnnotationConfigLoader()), mergedConfig));

[spring-projects-issues]

Sam Brannen commented

I actually assumed it was due to an auto-registered feature of Spring Boot Test (i.e., something like a ContextCustomizer).

So thank you for confirming my suspicions!

[spring-projects-issues]

Stefan Ludwig commented

FYI: I've extended the original bug example from the (GitHub issue) with "Boot - less" implementations of @Nested Spring REST Docs tests:
https://github.com/slu-it/bug-spring-restdocs-nested-tests/tree/master/src/test/java/withoutboot

The following test fails because the WebApplicationContext could not be provided by the Spring Extension (not found in context):

@SpringJUnitWebConfig(classes = Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    @BeforeEach
    void setup(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Stacktrace;

org.junit.jupiter.api.extension.ParameterResolutionException: Failed to resolve parameter [org.springframework.web.context.WebApplicationContext arg0] in executable [void withoutboot.FailingTest.setup(org.springframework.web.context.WebApplicationContext,org.springframework.restdocs.RestDocumentationContextProvider)]

	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameter(ExecutableInvoker.java:221)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameters(ExecutableInvoker.java:174)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameters(ExecutableInvoker.java:135)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:116)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeMethodInExtensionContext(ClassTestDescriptor.java:302)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$12(ClassTestDescriptor.java:290)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:135)
	at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:155)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:134)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:109)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:58)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:112)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:55)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:65)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.web.context.WebApplicationContext' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoMatchingBeanFound(DefaultListableBeanFactory.java:1509)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1104)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1065)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveDependency(AbstractAutowireCapableBeanFactory.java:344)
	at org.springframework.test.context.junit.jupiter.ParameterAutowireUtils.resolveDependency(ParameterAutowireUtils.java:98)
	at org.springframework.test.context.junit.jupiter.SpringExtension.resolveParameter(SpringExtension.java:177)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.resolveParameter(ExecutableInvoker.java:207)
	... 69 more

If the context configuration is duplicated by adding @SpringJUnitWebConfig(classes = Application.class) on the @Nested class it works (as previously mentioned by Andy Wilkinson). But this doesn't feel very intuitive.

(comment moved from SPR-16595)

[spring-projects-issues]

Sam Brannen commented

Stefan Ludwig,

I think the behavior you've described might actually be a bug in JUnit Jupiter instead of Spring.

Can you please tell me what happens if you move the setup for MockMvc into a constructor as follows?

@SpringJUnitWebConfig(Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    FailingTest(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Thanks in advance for feedback!

[spring-projects-issues]

Stefan Ludwig commented

If I do that, it works!
I added that code as an working example to the repository.

[spring-projects-issues]

Sam Brannen commented

If I do that, it works!

Awesome... and depressing... at the same time. ;-)

Awesome that it works for you!

Depressing (in a facetious way) for me because that means it is in fact an issue in JUnit Jupiter's handling of @BeforeEach methods with regard to the ExtensionContext supplied to a ParameterResolver registered for an enclosing test class when executing a test method within a @Nested test class.

And if that's too much of a mouthful for you, don't worry: I'll address the issue you've described within JUnit Jupiter.

Cheers,

Sam

[spring-projects-issues]

Stefan Ludwig commented

Glad to have helped! And sorry to depress you ;)

But are you sure it's not Springs management / caching of application contexts?

I thought it might be the same issue as, or at least very closely related to, #21136 because when I debug the following example:

@SpringJUnitWebConfig(classes = Application.class)
@ExtendWith(RestDocumentationExtension.class)
class FailingTest {

    MockMvc mockMvc;

    @BeforeEach
    void setup(WebApplicationContext wac, RestDocumentationContextProvider restDocumentation) {
        mockMvc = MockMvcBuilders.webAppContextSetup(wac) //
            .apply(documentationConfiguration(restDocumentation)) //
            .build();
    }

    @Test
    void testAndDocumentation() throws Exception {
        mockMvc.perform(get("/hello"))
            .andExpect(status().is2xxSuccessful())
            .andExpect(jsonPath("message", equalTo("Hello World!")))
            .andDo(document("hello-get-200"));
    }

    @Nested
    class NestedTests {

        @Test
        void testAndDocumentation() throws Exception {
            mockMvc.perform(get("/hello"))
                .andExpect(status().is2xxSuccessful())
                .andExpect(jsonPath("message", equalTo("Hello World!")))
                .andDo(document("hello-get-200"));
        }

    }

}

Adding a breakpoint in the SpringExtension within public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext).
I get an GenericWebApplicationContext with all the expected beans for the main test class.
But for the @Nested class I just get a GenericApplicationContext with none of the expected beans in it.

That sounds to me like what was described in #21136. Especially because adding @SpringJUnitWebConfig(classes = Application.class) to the @Nested class actually makes the test work and for both contexts (nested and enclosing class) the same application context is used in the parameter resolver.

[spring-projects-issues]

Sam Brannen commented

FYI: I have opened the following issue for JUnit Jupiter.

junit-team/junit-framework#1332

[spring-projects-issues]

Sam Brannen commented

Adding a breakpoint in the SpringExtension within public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext).
I get an GenericWebApplicationContext with all the expected beans for the main test class.
But for the @Nested class I just get a GenericApplicationContext with none of the expected beans in it.

Well, you have spring-boot-test in the classpath -- right?

Otherwise, Spring will not load any ApplicationContext for the @Nested class (if the nested class does not provide its own annotations).

[spring-projects-issues]

Sam Brannen commented

That sounds to me like what was described in #21136.

No. #21136 covers use cases where @Import is used on a test class.

[spring-projects-issues]

Stefan Ludwig commented

Oh, I did not think about the spring-boot-test dependency existing influencing the test. Thanks for clarifying!

[spring-projects-issues]

Sam Brannen commented

You're welcome.

And... yeah... I agree that having spring-boot-test change things just by being present in the classpath can be a bit confusing at times. ;-)

[spring-projects-issues]

Sam Brannen commented

I added that code as an working example to the repository.

Speaking of which, thanks for creating that!

It's a nice collection of examples.

[spring-projects-issues]

Andy Wilkinson commented

having spring-boot-test change things just by being present in the classpath

It doesn’t. You have to using SpringRunner or SpringExtension too. At that point we’re at the mercy of the registration mechanism for ContextCustomizerFactory and the like. Is there a way to opt those back out again?