-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Clean up the documentation for the library testing #59401
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
6f3d24f
Clean up the doc for the library testing on Android
MaximLipnin a17fe25
Add a section about functional tests
MaximLipnin 6a10f2d
Update the doc for testing on apple mobile platforms
MaximLipnin 9a3f044
Initial note on testing various configurations on apple mobile platforms
MaximLipnin c550324
Trailing spaces
MaximLipnin 886789d
Fix links
MaximLipnin f3a23f4
Fix a link
MaximLipnin d045edb
Fix a typo
MaximLipnin d550d51
Trailing spaces
MaximLipnin 494d96e
Use a build parameter for configuration
MaximLipnin 819c852
Use a build parameter for configuration
MaximLipnin ea6d849
Add the functional tests section to the Mono general testing document
MaximLipnin c975413
Merge branch 'main' into cleanup_testing_doc
MaximLipnin 9af3c51
address the feedback
MaximLipnin 1863728
Trailing spaces
MaximLipnin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,34 +1,89 @@ | ||
| # Testing Libraries on iOS and tvOS | ||
| # Testing Libraries on iOS, tvOS, and MacCatalyst | ||
|
|
||
| In order to build libraries and tests for iOS or tvOS you need recent version of XCode installed (e.g. 11.3 or higher). | ||
| ## Prerequisites | ||
|
|
||
| Build Libraries for iOS Simulator: | ||
| - XCode 11.3 or higher | ||
MaximLipnin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Building Libs and Tests | ||
|
|
||
| You can build and run the library tests: | ||
| - on a simulator; | ||
| - on a device. | ||
|
|
||
| Run the following command in a terminal: | ||
| ``` | ||
| ./build.sh mono+libs -os <TARGET_OS> -arch x64 | ||
MaximLipnin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ``` | ||
| where `<TARGET_OS>` is one of the following: | ||
|
|
||
| | simulator | device| | ||
| |:--------------|:------| | ||
| | iOSSimulator | iOS | | ||
| | tvOSSimulator | tvOS | | ||
| | MacCatalyst | | | ||
MaximLipnin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
|
|
||
| e.g., to build for a iOS simulator, run: | ||
| ``` | ||
| ./build.sh mono+libs -os iOSSimulator -arch x64 | ||
| ``` | ||
|
|
||
| Run tests one by one for each test suite on a simulator: | ||
| ``` | ||
| ./build.sh libs.tests -os iOSSimulator -arch x64 -test | ||
| ``` | ||
|
|
||
| ### Building for a device | ||
|
|
||
| In order to run the tests on a device: | ||
| - Set the os to `iOS` instead of `iOSSimulator` | ||
| - Specify `DevTeamProvisioning` (see [developer.apple.com/account/#/membership](https://developer.apple.com/account/#/membership), scroll down to `Team ID`): | ||
| - Set the `-os` parameter to a device-related value (see above) | ||
| - Specify `DevTeamProvisioning` (see [developer.apple.com/account/#/membership](https://developer.apple.com/account/#/membership), scroll down to `Team ID`). | ||
|
|
||
| For example: | ||
| ``` | ||
| ./build.sh libs.tests -os iOS -arch x64 -test /p:DevTeamProvisioning=H1A2B3C4D5 | ||
| ``` | ||
| [AppleAppBuilder](https://github.com/dotnet/runtime/blob/main/src/mono/msbuild/AppleAppBuilder/AppleAppBuilder.cs) generates temp Xcode projects you can manually open and resolve provisioning issues there using native UI and deploy to your devices. | ||
| [AppleAppBuilder](https://github.com/dotnet/runtime/blob/main/src/tasks/AppleAppBuilder/AppleAppBuilder.cs) generates temp Xcode projects you can manually open and resolve provisioning issues there using native UI and deploy to your devices. | ||
|
|
||
| ### Running individual test suites | ||
| - The following shows how to run tests for a specific library: | ||
MaximLipnin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ``` | ||
| ./dotnet.sh build src/libraries/System.Numerics.Vectors/tests /t:Test /p:TargetOS=iOS /p:TargetArchitecture=x64 | ||
| ``` | ||
|
|
||
| ### Running the functional tests | ||
|
|
||
| There are [functional tests](https://github.com/dotnet/runtime/tree/main/src/tests/FunctionalTests/) which aim to test some specific features/configurations/modes on a target mobile platform. | ||
|
|
||
| A functional test can be run the same way as any library test suite, e.g.: | ||
| ``` | ||
| ./dotnet.sh build /t:Test /p:TargetOS=iOSSimulator /p:TargetArchitecture=x64 /p:Configuration=Release src/tests/FunctionalTests/iOS/Simulator/PInvoke/iOS.Simulator.PInvoke.Test.csproj | ||
MaximLipnin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ``` | ||
|
|
||
| Currently functional tests are expected to return `42` as a success code so please be careful when adding a new one. | ||
|
|
||
| ### Testing various configurations | ||
MaximLipnin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| It's possible to test various configurations by setting a combination of additional MSBuild properties such as `RunAOTCompilation`,`MonoEnableInterpreter`, and some more. | ||
|
|
||
| 1. Interpreter Only | ||
|
|
||
| This configuration is necessary for hot reload scenarios. | ||
|
|
||
| To enable the interpreter, add `/p:RunAOTCompilation=true /p:MonoEnableInterpreter=true` to a build command. | ||
|
|
||
| 2. AOT only | ||
|
|
||
| To build for AOT only mode, add `/p:RunAOTCompilation=true /p:MonoEnableInterpreter=false` to a build command. | ||
|
|
||
| 3. AOT-LLVM | ||
|
|
||
| To build for AOT-LLVM mode, add `/p:RunAOTCompilation=true /p:MonoEnableInterpreter=false /p:MonoEnableLLVM=true` to a build command. | ||
|
|
||
| ### Test App Design | ||
| iOS/tvOS `*.app` (or `*.ipa`) is basically a simple [ObjC app](https://github.com/dotnet/runtime/blob/main/src/mono/msbuild/AppleAppBuilder/Templates/main-console.m) that inits the Mono Runtime. This Mono Runtime starts a simple xunit test | ||
| iOS/tvOS `*.app` (or `*.ipa`) is basically a simple [ObjC app](https://github.com/dotnet/runtime/blob/main/src/tasks/AppleAppBuilder/Templates/main-console.m) that inits the Mono Runtime. This Mono Runtime starts a simple xunit test | ||
| runner called XHarness.TestRunner (see https://github.com/dotnet/xharness) which runs tests for all `*.Tests.dll` libs in the bundle. There is also XHarness.CLI tool to deploy `*.app` and `*.ipa` to a target (device or simulator) and listens for logs via network sockets. | ||
|
|
||
| ### Existing Limitations | ||
| - Most of the test suites crash on devices due to #35674 | ||
| - Simulator uses JIT mode only at the moment (to be extended with FullAOT and Interpreter) | ||
| - Interpreter is not enabled yet. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.