From da193cb8620b7b7def474a563963b6dbde749e32 Mon Sep 17 00:00:00 2001 From: Tony Allevato Date: Thu, 1 Sep 2022 11:41:42 -0700 Subject: [PATCH] Add a `discover_tests` attribute to `swift_test` This attribute, which is `True` by default, controls whether `XCTest`-style tests are automatically discovered (either using the Objective-C runtime or manual discovery via symbol graphs). If set to `False`, the test is assumed to provide its own `main`. This will subsume the behavior of the `swift.bundled_xctests` feature, which will be removed in the future. PiperOrigin-RevId: 471589475 (cherry picked from commit 477b48a2390229fe7eb6faf056432ddc3a294eab) Signed-off-by: Brentley Jones --- doc/providers.md | 2 +- doc/rules.md | 39 ++------ swift/providers.bzl | 5 +- swift/swift_test.bzl | 104 ++++++++++++--------- swift/toolchains/swift_toolchain.bzl | 1 + swift/toolchains/xcode_swift_toolchain.bzl | 1 + 6 files changed, 77 insertions(+), 75 deletions(-) diff --git a/doc/providers.md b/doc/providers.md index b31a1e996..f9472df37 100644 --- a/doc/providers.md +++ b/doc/providers.md @@ -111,7 +111,7 @@ that use the toolchain. | requested_features | `List` of `string`s. Features that should be implicitly enabled by default for targets built using this toolchain, unless overridden by the user by listing their negation in the `features` attribute of a target/package or in the `--features` command line flag.

These features determine various compilation and debugging behaviors of the Swift build rules, and they are also passed to the C++ APIs used when linking (so features defined in CROSSTOOL may be used here). | | root_dir | `String`. The workspace-relative root directory of the toolchain. | | swift_worker | `File`. The executable representing the worker executable used to invoke the compiler and other Swift tools (for both incremental and non-incremental compiles). | -| test_configuration | `Struct` containing two fields:

* `env`: A `dict` of environment variables to be set when running tests that were built with this toolchain.

* `execution_requirements`: A `dict` of execution requirements for tests that were built with this toolchain.

This is used, for example, with Xcode-based toolchains to ensure that the `xctest` helper and coverage tools are found in the correct developer directory when running tests. | +| test_configuration | `Struct` containing the following fields:

* `env`: A `dict` of environment variables to be set when running tests that were built with this toolchain.

* `execution_requirements`: A `dict` of execution requirements for tests that were built with this toolchain.

* `uses_xctest_bundles`: A Boolean value indicating whether test targets should emit `.xctest` bundles that are launched with the `xctest` tool.

This is used, for example, with Xcode-based toolchains to ensure that the `xctest` helper and coverage tools are found in the correct developer directory when running tests. | | tool_configs | This field is an internal implementation detail of the build rules. | | unsupported_features | `List` of `string`s. Features that should be implicitly disabled by default for targets built using this toolchain, unless overridden by the user by listing them in the `features` attribute of a target/package or in the `--features` command line flag.

These features determine various compilation and debugging behaviors of the Swift build rules, and they are also passed to the C++ APIs used when linking (so features defined in CROSSTOOL may be used here). | diff --git a/doc/rules.md b/doc/rules.md index 610e2feb0..849f70706 100644 --- a/doc/rules.md +++ b/doc/rules.md @@ -678,41 +678,17 @@ swift_binary( ## swift_test 
-swift_test(name, deps, srcs, data, copts, defines, env, linkopts, malloc, module_name, package_name,
-           plugins, stamp, swiftc_inputs)
+swift_test(name, deps, srcs, data, copts, defines, discover_tests, env, linkopts, malloc,
+           module_name, package_name, plugins, stamp, swiftc_inputs)
Compiles and links Swift code into an executable test target. -The behavior of `swift_test` differs slightly for macOS targets, in order to -provide seamless integration with Apple's XCTest framework. The output of the -rule is still a binary, but one whose Mach-O type is `MH_BUNDLE` (a loadable -bundle). Thus, the binary cannot be launched directly. Instead, running -`bazel test` on the target will launch a test runner script that copies it into -an `.xctest` bundle directory and then launches the `xctest` helper tool from -Xcode, which uses Objective-C runtime reflection to locate the tests. - -On Linux, the output of a `swift_test` is a standard executable binary, because -the implementation of XCTest on that platform currently requires authors to -explicitly list the tests that are present and run them from their main program. - -Test bundling on macOS can be disabled on a per-target basis, if desired. You -may wish to do this if you are not using XCTest, but rather a different test -framework (or no framework at all) where the pass/fail outcome is represented as -a zero/non-zero exit code (as is the case with other Bazel test rules like -`cc_test`). To do so, disable the `"swift.bundled_xctests"` feature on the -target: - -```python -swift_test( - name = "MyTests", - srcs = [...], - features = ["-swift.bundled_xctests"], -) -``` - -You can also disable this feature for all the tests in a package by applying it -to your BUILD file's `package()` declaration instead of the individual targets. +By default, this rule performs _test discovery_ that finds tests written with +the `XCTest` framework and executes them automatically, without the user +providing their own `main` entry point. See the documentation of the +`discover_tests` attribute for more information about how this affects the rule +output and how to control this behavior. If integrating with Xcode, the relative paths in test binaries can prevent the Issue navigator from working for test failures. To work around this, you can @@ -738,6 +714,7 @@ bazel test //:Tests --test_filter=TestModuleName.TestClassName/testMethodName | data | The list of files needed by this target at runtime.

Files and targets named in the `data` attribute will appear in the `*.runfiles` area of this target, if it has one. This may include data files needed by a binary or library, or other programs needed by it. | List of labels | optional | `[]` | | copts | Additional compiler options that should be passed to `swiftc`. These strings are subject to `$(location ...)` and ["Make" variable](https://docs.bazel.build/versions/master/be/make-variables.html) expansion. | List of strings | optional | `[]` | | defines | A list of defines to add to the compilation command line.

Note that unlike C-family languages, Swift defines do not have values; they are simply identifiers that are either defined or undefined. So strings in this list should be simple identifiers, **not** `name=value` pairs.

Each string is prepended with `-D` and added to the command line. Unlike `copts`, these flags are added for the target and every target that depends on it, so use this attribute with caution. It is preferred that you add defines directly to `copts`, only using this feature in the rare case that a library needs to propagate a symbol up to those that depend on it. | List of strings | optional | `[]` | +| discover_tests | Determines whether or not tests are automatically discovered in the binary. The default value is `True`.

If tests are discovered, then you should not provide your own `main` entry point in the `swift_test` binary; the test runtime provides the entry point for you. If you set this attribute to `False`, then you are responsible for providing your own `main`. This allows you to write tests that use a framework other than Apple's `XCTest`. The only requirement of such a test is that it terminate with a zero exit code for success or a non-zero exit code for failure.

Additionally, on Apple platforms, test discovery is handled by the Objective-C runtime and the output of a `swift_test` rule is an `.xctest` bundle that is invoked using the `xctest` tool in Xcode. If this attribute is used to disable test discovery, then the output of the `swift_test` rule will instead be a standard executable binary that is invoked directly. | Boolean | optional | `True` | | env | Dictionary of environment variables that should be set during the test execution. | Dictionary: String -> String | optional | `{}` | | linkopts | Additional linker options that should be passed to `clang`. These strings are subject to `$(location ...)` expansion. | List of strings | optional | `[]` | | malloc | Override the default dependency on `malloc`.

By default, Swift binaries are linked against `@bazel_tools//tools/cpp:malloc"`, which is an empty library and the resulting binary will use libc's `malloc`. This label must refer to a `cc_library` rule. | Label | optional | `"@bazel_tools//tools/cpp:malloc"` | diff --git a/swift/providers.bzl b/swift/providers.bzl index 7ba6083fb..99f87768b 100644 --- a/swift/providers.bzl +++ b/swift/providers.bzl @@ -301,7 +301,7 @@ compiler and other Swift tools (for both incremental and non-incremental compiles). """, "test_configuration": """\ -`Struct` containing two fields: +`Struct` containing the following fields: * `env`: A `dict` of environment variables to be set when running tests that were built with this toolchain. @@ -309,6 +309,9 @@ compiles). * `execution_requirements`: A `dict` of execution requirements for tests that were built with this toolchain. +* `uses_xctest_bundles`: A Boolean value indicating whether test targets + should emit `.xctest` bundles that are launched with the `xctest` tool. + This is used, for example, with Xcode-based toolchains to ensure that the `xctest` helper and coverage tools are found in the correct developer directory when running tests. diff --git a/swift/swift_test.bzl b/swift/swift_test.bzl index 7573dacfc..01fab5bc4 100644 --- a/swift/swift_test.bzl +++ b/swift/swift_test.bzl @@ -224,9 +224,17 @@ def _swift_test_impl(ctx): unsupported_features = ctx.disabled_features, ) - is_bundled = swift_common.is_enabled( - feature_configuration = feature_configuration, - feature_name = SWIFT_FEATURE_BUNDLED_XCTESTS, + discover_tests = ctx.attr.discover_tests + + # TODO(b/220945250): Remove the `bundled_xctests` feature and use only the + # toolchain bit instead. + is_bundled = ( + discover_tests and + swift_toolchain.test_configuration.uses_xctest_bundles and + swift_common.is_enabled( + feature_configuration = feature_configuration, + feature_name = SWIFT_FEATURE_BUNDLED_XCTESTS, + ) ) # If we need to run the test in an .xctest bundle, the binary must have @@ -255,6 +263,7 @@ def _swift_test_impl(ctx): additional_linking_contexts.append(plugin_info.cc_info.linking_context) srcs = ctx.files.srcs + extra_copts = [] extra_deps = [] # If no sources were provided and we're not using `.xctest` bundling, assume @@ -263,16 +272,30 @@ def _swift_test_impl(ctx): # (a separate module) maps to its own `swift_library`. We'll need to modify # this approach if we want to support test discovery for simple `swift_test` # targets that just write XCTest-style tests in the `srcs` directly. - if not srcs and not is_bundled: - srcs = _generate_test_discovery_srcs( - actions = ctx.actions, - deps = ctx.attr.deps, - name = ctx.label.name, - test_discoverer = ctx.executable._test_discoverer, - ) - extra_deps = [ctx.attr._test_observer] - elif is_bundled: + if discover_tests: + if ( + not srcs and + not swift_toolchain.test_configuration.uses_xctest_bundles + ): + srcs = _generate_test_discovery_srcs( + actions = ctx.actions, + deps = ctx.attr.deps, + name = ctx.label.name, + test_discoverer = ctx.executable._test_discoverer, + ) + + # Discovered tests don't need an entry point; on Apple platforms, + # the binary is compiled as a bundle, and on non-Apple platforms, + # the generated sources above use `@main`. + # TODO(b/220945250): This should be moved out of this branch of the + # conditional, but it would break some tests that are already + # depending on this and need to be fixed. + extra_copts = ["-parse-as-library"] + + # Inject the test observer that prints the xUnit-style output for Bazel. extra_deps = [ctx.attr._test_observer] + else: + extra_copts = _maybe_parse_as_library_copts(srcs) module_contexts = [] all_supplemental_outputs = [] @@ -292,7 +315,7 @@ def _swift_test_impl(ctx): ctx, ctx.attr.copts, ctx.attr.swiftc_inputs, - ) + _maybe_parse_as_library_copts(srcs), + ) + extra_copts, defines = ctx.attr.defines, extra_swift_infos = extra_swift_infos, feature_configuration = feature_configuration, @@ -434,6 +457,27 @@ swift_test = rule( stamp_default = 0, ), { + "discover_tests": attr.bool( + default = True, + doc = """\ +Determines whether or not tests are automatically discovered in the binary. The +default value is `True`. + +If tests are discovered, then you should not provide your own `main` entry point +in the `swift_test` binary; the test runtime provides the entry point for you. +If you set this attribute to `False`, then you are responsible for providing +your own `main`. This allows you to write tests that use a framework other than +Apple's `XCTest`. The only requirement of such a test is that it terminate with +a zero exit code for success or a non-zero exit code for failure. + +Additionally, on Apple platforms, test discovery is handled by the Objective-C +runtime and the output of a `swift_test` rule is an `.xctest` bundle that is +invoked using the `xctest` tool in Xcode. If this attribute is used to disable +test discovery, then the output of the `swift_test` rule will instead be a +standard executable binary that is invoked directly. +""", + mandatory = False, + ), "env": attr.string_dict( doc = """ Dictionary of environment variables that should be set during the test execution. @@ -473,35 +517,11 @@ swift_test = rule( doc = """\ Compiles and links Swift code into an executable test target. -The behavior of `swift_test` differs slightly for macOS targets, in order to -provide seamless integration with Apple's XCTest framework. The output of the -rule is still a binary, but one whose Mach-O type is `MH_BUNDLE` (a loadable -bundle). Thus, the binary cannot be launched directly. Instead, running -`bazel test` on the target will launch a test runner script that copies it into -an `.xctest` bundle directory and then launches the `xctest` helper tool from -Xcode, which uses Objective-C runtime reflection to locate the tests. - -On Linux, the output of a `swift_test` is a standard executable binary, because -the implementation of XCTest on that platform currently requires authors to -explicitly list the tests that are present and run them from their main program. - -Test bundling on macOS can be disabled on a per-target basis, if desired. You -may wish to do this if you are not using XCTest, but rather a different test -framework (or no framework at all) where the pass/fail outcome is represented as -a zero/non-zero exit code (as is the case with other Bazel test rules like -`cc_test`). To do so, disable the `"swift.bundled_xctests"` feature on the -target: - -```python -swift_test( - name = "MyTests", - srcs = [...], - features = ["-swift.bundled_xctests"], -) -``` - -You can also disable this feature for all the tests in a package by applying it -to your BUILD file's `package()` declaration instead of the individual targets. +By default, this rule performs _test discovery_ that finds tests written with +the `XCTest` framework and executes them automatically, without the user +providing their own `main` entry point. See the documentation of the +`discover_tests` attribute for more information about how this affects the rule +output and how to control this behavior. If integrating with Xcode, the relative paths in test binaries can prevent the Issue navigator from working for test failures. To work around this, you can diff --git a/swift/toolchains/swift_toolchain.bzl b/swift/toolchains/swift_toolchain.bzl index fdadc860a..666cc9c0d 100644 --- a/swift/toolchains/swift_toolchain.bzl +++ b/swift/toolchains/swift_toolchain.bzl @@ -549,6 +549,7 @@ def _swift_toolchain_impl(ctx): test_configuration = struct( env = env, execution_requirements = {}, + uses_xctest_bundles = False, ), tool_configs = all_tool_configs, unsupported_features = ctx.disabled_features + [ diff --git a/swift/toolchains/xcode_swift_toolchain.bzl b/swift/toolchains/xcode_swift_toolchain.bzl index ab5c2f4c1..86989845d 100644 --- a/swift/toolchains/xcode_swift_toolchain.bzl +++ b/swift/toolchains/xcode_swift_toolchain.bzl @@ -806,6 +806,7 @@ def _xcode_swift_toolchain_impl(ctx): test_configuration = struct( env = env, execution_requirements = execution_requirements, + uses_xctest_bundles = True, ), tool_configs = all_tool_configs, unsupported_features = ctx.disabled_features + [