Runtime tests have issues finding the testing app on Android x64

[VincentBu]

Run: runtime 20210526.87

Failed tests:

Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.Open
- JIT/Methodical/*

Also failed in :

Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.Open
- JIT/Regression/JitBlue/DevDiv_461649/DevDiv_461649/DevDiv_461649.sh
- Interop/StructMarshalling/ReversePInvoke/MarshalSeqStruct/DelegatePInvoke/DelegatePInvokeTest/DelegatePInvokeTest.sh
- GC/Scenarios/Dynamo/dynamo/dynamo.sh

One of Error messages:

Return code:      78
Raw output file:      /datadisks/disk1/work/A0E608D3/w/B0710944/e/JIT/Methodical/Reports/JIT.Methodical/VT/callconv/_il_reljumper4/_il_reljumper4.output.txt
Raw output:
BEGIN EXECUTION
XHarness command issued: android test --instrumentation=net.dot.MonoRunner --package-name=net.dot.JIT_Methodical --app=/datadisks/disk1/work/A0E608D3/w/B0710944/e/JIT/Methodical/JIT_Methodical.apk --output-directory=/datadisks/disk1/work/A0E608D3/w/B0710944/e/JIT/Methodical/VT/callconv/_il_reljumper4 --arg=entrypoint:libname=_il_reljumper4.dll --expected-exit-code=100 -v
[23:00:51] dbug: Android Test command called: App = /datadisks/disk1/work/A0E608D3/w/B0710944/e/JIT/Methodical/JIT_Methodical.apk
Instrumentation Name = net.dot.MonoRunner
[23:00:51] dbug: Output Directory:/datadisks/disk1/work/A0E608D3/w/B0710944/e/JIT/Methodical/VT/callconv/_il_reljumper4
Timeout = 900 seconds.
[23:00:51] dbug: Arguments to instrumentation:
[23:00:51] dbug: ADBRunner using ADB.exe supplied from /datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/tools/net6.0/any/../../../runtimes/any/native/adb/linux/adb
[23:00:51] dbug: Full resolved path:'/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb'
[23:00:51] info: Will attempt to run device on detected architecture: 'x86_64'
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb  start-server'
[23:00:51] dbug:
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb  devices -l'
[23:00:51] dbug: Evaluating output line for device serial: emulator-5554          device product:sdk_phone_x86 model:Android_SDK_built_for_x86 device:generic_x86 transport_id:1
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb  -s emulator-5554 shell getprop ro.product.cpu.abilist'
[23:00:51] dbug: Evaluating output line for device serial: emulator-5556          device product:sdk_phone_x86_64 model:Android_SDK_built_for_x86_64 device:generic_x86_64 transport_id:2
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb  -s emulator-5556 shell getprop ro.product.cpu.abilist'
[23:00:51] dbug: Using first-found compatible device of 1 total- serial: 'emulator-5556' - architecture: x86_64,x86
[23:00:51] info: Active Android device set to serial 'emulator-5556'
[23:00:51] info: Waiting for device to be available (max 5 minutes)
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb -s emulator-5556 wait-for-device'
[23:00:51] dbug:
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb -s emulator-5556 shell getprop sys.boot_completed'
[23:00:51] dbug: Waited 0.0216012 seconds for device for sys.boot_completed to be 1.
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb -s emulator-5556 version'
[23:00:51] dbug: Working with Android Debug Bridge version 1.0.41
Version 29.0.6-6198805
Installed as /datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/any/native/adb/linux/adb

[23:00:51] info: Attempting to remove apk 'net.dot.JIT_Methodical':
[23:00:51] dbug: Executing command: '/datadisks/disk1/work/A0E608D3/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21271.1/runtimes/a


Stack trace
   at JIT_Methodical._VT_callconv__il_reljumper4__il_reljumper4_._VT_callconv__il_reljumper4__il_reljumper4_sh()

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[VincentBu]

Failed again in runtime 20210608.73 with JIT.Regression tests

One of Error message:

Return code:      85
Raw output file:      /datadisks/disk1/work/A9920943/w/97010892/uploads/Reports/JIT.Regression/JitBlue/Runtime_41100/Runtime_41100/Runtime_41100.output.txt
Raw output:
BEGIN EXECUTION
XHarness command issued: android test --instrumentation=net.dot.MonoRunner --package-name=net.dot.JIT_Regression --app=/datadisks/disk1/work/A9920943/w/97010892/e/JIT/Regression/JIT_Regression.apk --output-directory=/datadisks/disk1/work/A9920943/w/97010892/uploads/Reports/JIT.Regression/JitBlue/Runtime_41100/Runtime_41100 --arg=entrypoint:libname=Runtime_41100.dll --expected-exit-code=100 -v
[20:07:21] dbug: Android Test command called: App = /datadisks/disk1/work/A9920943/w/97010892/e/JIT/Regression/JIT_Regression.apk
Instrumentation Name = net.dot.MonoRunner
[20:07:21] dbug: Output Directory:/datadisks/disk1/work/A9920943/w/97010892/uploads/Reports/JIT.Regression/JitBlue/Runtime_41100/Runtime_41100
Timeout = 900 seconds.
[20:07:21] dbug: Arguments to instrumentation:
[20:07:21] dbug: ADBRunner using ADB.exe supplied from /datadisks/disk1/work/A9920943/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21307.1/tools/net6.0/any/../../../runtimes/any/native/adb/linux/adb
[20:07:21] dbug: Full resolved path:'/datadisks/disk1/work/A9920943/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21307.1/runtimes/any/native/adb/linux/adb'
[20:07:21] info: Will attempt to run device on detected architecture: 'x86_64'
[20:07:21] dbug: Executing command: '/datadisks/disk1/work/A9920943/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21307.1/runtimes/any/native/adb/linux/adb  start-server'
[20:07:21] dbug:
[20:07:21] dbug: Executing command: '/datadisks/disk1/work/A9920943/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21307.1/runtimes/any/native/adb/linux/adb  devices -l'
[20:07:21] dbug: Evaluating output line for device serial: emulator-5556          device product:sdk_phone_x86 model:Android_SDK_built_for_x86 device:generic_x86 transport_id:2
[20:07:21] dbug: Executing command: '/datadisks/disk1/work/A9920943/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21307.1/runtimes/any/native/adb/linux/adb  -s emulator-5556 shell getprop ro.product.cpu.abilist'
[20:07:21] fail: No devices with architecture 'x86_64' was found among attached and authorized devices.
[20:07:21] crit: Failed to find compatible device: x86_64
Microsoft.DotNet.XHarness.Common.CLI.NoDeviceFoundException: Failed to find compatible device: x86_64
at Microsoft.DotNet.XHarness.CLI.Commands.Android.AndroidInstallCommand.InvokeHelper(ILogger logger, String apkPackageName, String appPackagePath, IEnumerable`1 apkRequiredArchitecture, String deviceId, TimeSpan bootTimeoutSeconds, AdbRunner runner) in //src/Microsoft.DotNet.XHarness.CLI/Commands/Android/AndroidInstallCommand.cs:line 104
at Microsoft.DotNet.XHarness.CLI.Commands.Android.AndroidTestCommand.InvokeInternal(ILogger logger) in //src/Microsoft.DotNet.XHarness.CLI/Commands/Android/AndroidTestCommand.cs:line 78
XHarness exit code: 85 (ADB_DEVICE_ENUMERATION_FAILURE)
Test Harness Exitcode is : 85
To run the test:

set CORE_ROOT=/datadisks/disk1/work/A9920943/p
/datadisks/disk1/work/A9920943/w/97010892/e/JIT/Regression/JitBlue/Runtime_41100/Runtime_41100/Runtime_41100.sh
Expected: True
Actual:   False


Stack trace
   at JIT_Regression._JitBlue_Runtime_41100_Runtime_41100_Runtime_41100_._JitBlue_Runtime_41100_Runtime_41100_Runtime_41100_sh()

[SamMonoRT]

@fanyang-mono : another intermittent failure on Android "Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.Open" lane

[fanyang-mono]

Both of the log looks like issues related to the xharness test API. The first one stuck at uninstalling Android app. The second one showed that it couldn't find the Android emulator. @premun @greenEkatherine Could your team take a look at these failures?

[greenEkatherine]

I'm on it @fanyang-mono

[VincentBu]

Failed again in runtime 20210629.108 with JIT.jit64.* reflection.* Loader/regressions/GitHub_15544/main/main.sh CoreMangLib/system/environment/environment_version/environment_version.sh tests.

Failed tests:

Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.Open

- JIT/jit64/*
- reflection/regression/manyStackArgs/manyStackArgs/manyStackArgs.sh
- reflection/DefaultInterfaceMethods/GetInterfaceMapConsumer/GetInterfaceMapConsumer.sh
- Loader/regressions/GitHub_15544/main/main.sh
- CoreMangLib/system/environment/environment_version/environment_version.sh

One of Error messages:

Return code:      85
Raw output file:      /datadisks/disk1/work/A9CD0924/w/AF4909B5/uploads/Reports/JIT.jit64/opt/cg/cgstress/CgStress1_do/CgStress1_do.output.txt
Raw output:
BEGIN EXECUTION
XHarness command issued: android run --instrumentation=net.dot.MonoRunner --package-name=net.dot.JIT_jit64 --output-directory=/datadisks/disk1/work/A9CD0924/w/AF4909B5/uploads/Reports/JIT.jit64/opt/cg/cgstress/CgStress1_do --arg=entrypoint:libname=CgStress1_do.dll --expected-exit-code=100 -v
[23:48:11] dbug: Android Run command called: App = net.dot.JIT_jit64

[23:48:11] dbug: Timeout = 900 seconds.
[23:48:11] dbug: ADBRunner using ADB.exe supplied from /datadisks/disk1/work/A9CD0924/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21324.2/tools/net6.0/any/../../../runtimes/any/native/adb/linux/adb
[23:48:11] dbug: Full resolved path:'/datadisks/disk1/work/A9CD0924/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21324.2/runtimes/any/native/adb/linux/adb'
[23:48:11] dbug: Executing command: '/datadisks/disk1/work/A9CD0924/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21324.2/runtimes/any/native/adb/linux/adb  start-server'
[23:48:11] dbug:
[23:48:11] dbug: Executing command: '/datadisks/disk1/work/A9CD0924/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21324.2/runtimes/any/native/adb/linux/adb  devices -l'
[23:48:11] dbug: Evaluating output line for device serial: emulator-5554          device product:sdk_phone_x86_64 model:Android_SDK_built_for_x86_64 device:generic_x86_64 transport_id:3
[23:48:11] dbug: Executing command: '/datadisks/disk1/work/A9CD0924/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21324.2/runtimes/any/native/adb/linux/adb  -s emulator-5554 shell pm list packages -3'
[23:48:11] dbug: Evaluating output line for device serial: emulator-5556          device product:sdk_phone_x86 model:Android_SDK_built_for_x86 device:generic_x86 transport_id:2
[23:48:11] dbug: Executing command: '/datadisks/disk1/work/A9CD0924/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21324.2/runtimes/any/native/adb/linux/adb  -s emulator-5556 shell pm list packages -3'
[23:48:11] fail: No devices with app 'package:net.dot.JIT_jit64' was found among attached and authorized devices.
[23:48:11] fail: Cannot find a device with app=package:net.dot.JIT_jit64, please check that a device is attached
XHarness exit code: 85 (ADB_DEVICE_ENUMERATION_FAILURE)
Test Harness Exitcode is : 85
To run the test:

set CORE_ROOT=/datadisks/disk1/work/A9CD0924/p
/datadisks/disk1/work/A9CD0924/w/AF4909B5/e/JIT/jit64/opt/cg/cgstress/CgStress1_do/CgStress1_do.sh
Expected: True
Actual:   False


Stack trace
   at JIT_jit64._opt_cg_cgstress_CgStress1_do_CgStress1_do_._opt_cg_cgstress_CgStress1_do_CgStress1_do_sh() in JIT.jit64.XUnitWrapper.dll:token 0x6000007+0x2b2

[radical]

I'm not sure if this is the same failure. JIT.Methodical.a-dA-D seems to timeout on CoreCLR Pri0 Runtime Tests Run windows arm64 checked:

build: https://dev.azure.com/dnceng/public/_build/results?buildId=1213972&view=logs&jobId=bb94f37e-a09b-5b51-09b2-ab78e7aa5103&j=bb94f37e-a09b-5b51-09b2-ab78e7aa5103&t=506cadc9-3cc4-5d96-6847-011f75a33382
PR: dotnet/runtime#54936

The console log has only this line:
Console log: 'JIT.Methodical.a-dA-D' from job dc9b8409-2ae7-410d-81eb-24a7d8b741c2 workitem ae895520-db5a-4e94-8fdd-b661552c6c93 (windows.10.arm64v8.open) executed on machine DDARM64-119

[fanyang-mono]

@radical This issue tracks Android specific issues. The one you are seeing on windows should be tracked by a different issue.

[VincentBu]

Failed again in runtime 20210706.106
Failed tests

Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.Open

- JIT/Methodical/*
- JIT/Math/Functions/Functions_r/Functions_r.sh
- JIT/Math/Functions/Functions_ro/Functions_ro.sh
- Regressions/coreclr/*
- baseservices/*

One of error messages

Return code:      85
Raw output file:      /datadisks/disk1/work/AC000986/w/B8490A0C/uploads/Reports/JIT.Methodical/int64/signed/_rels_ldsfld_mulovf/_rels_ldsfld_mulovf.output.txt
Raw output:
BEGIN EXECUTION
XHarness command issued: android run --instrumentation=net.dot.MonoRunner --package-name=net.dot.JIT_Methodical --output-directory=/datadisks/disk1/work/AC000986/w/B8490A0C/uploads/Reports/JIT.Methodical/int64/signed/_rels_ldsfld_mulovf --arg=entrypoint:libname=_rels_ldsfld_mulovf.dll --expected-exit-code=100 -v
[03:27:56] dbug: Android Run command called: App = net.dot.JIT_Methodical

[03:27:56] dbug: Timeout = 900 seconds.
[03:27:56] dbug: ADBRunner using ADB.exe supplied from /datadisks/disk1/work/AC000986/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21330.2/tools/net6.0/any/../../../runtimes/any/native/adb/linux/adb
[03:27:56] dbug: Full resolved path:'/datadisks/disk1/work/AC000986/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21330.2/runtimes/any/native/adb/linux/adb'
[03:27:56] dbug: Executing command: '/datadisks/disk1/work/AC000986/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21330.2/runtimes/any/native/adb/linux/adb  start-server'
[03:27:57] dbug:
[03:27:57] dbug: Executing command: '/datadisks/disk1/work/AC000986/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21330.2/runtimes/any/native/adb/linux/adb  devices -l'
[03:27:57] dbug: Evaluating output line for device serial: emulator-5554          device product:sdk_phone_x86_64 model:Android_SDK_built_for_x86_64 device:generic_x86_64 transport_id:9
[03:27:57] dbug: Executing command: '/datadisks/disk1/work/AC000986/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21330.2/runtimes/any/native/adb/linux/adb  -s emulator-5554 shell pm list packages -3'
[03:27:57] dbug: Evaluating output line for device serial: emulator-5556          device product:sdk_phone_x86 model:Android_SDK_built_for_x86 device:generic_x86 transport_id:2
[03:27:57] dbug: Executing command: '/datadisks/disk1/work/AC000986/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21330.2/runtimes/any/native/adb/linux/adb  -s emulator-5556 shell pm list packages -3'
[03:27:57] fail: No devices with app 'package:net.dot.JIT_Methodical' was found among attached and authorized devices.
[03:27:57] fail: Cannot find a device with app=package:net.dot.JIT_Methodical, please check that a device is attached
XHarness exit code: 85 (ADB_DEVICE_ENUMERATION_FAILURE)
Test Harness Exitcode is : 85
To run the test:

set CORE_ROOT=/datadisks/disk1/work/AC000986/p
/datadisks/disk1/work/AC000986/w/B8490A0C/e/JIT/Methodical/int64/signed/_rels_ldsfld_mulovf/_rels_ldsfld_mulovf.sh
Expected: True
Actual:   False


Stack trace
   at JIT_Methodical._int64_signed__rels_ldsfld_mulovf__rels_ldsfld_mulovf_._int64_signed__rels_ldsfld_mulovf__rels_ldsfld_mulovf_sh() in JIT.Methodical.XUnitWrapper.dll:token 0x60003e5+0x2b2

[SamMonoRT]

@greenEkatherine @premun - should this be moved to the dotnet\xharness repo for tracking ?

[greenEkatherine]

@SamMonoRT nice idea, I'm going to transfer it there