Improve reliability of PyTorch test reporting

[Flamefire]

(created using eb --new-pr)

This fixes 2 blocking issues:

1. Test counts can be off when unittest.subTest is used. E.g.

<?xml version="1.0"?>
<testsuites>
  <testsuite name="pytest" errors="0" failures="1" skipped="0" tests="2" time="13.590" timestamp="2025-05-08T02:18:36.745086" hostname="c68">
    <testcase classname="MiscTests" name="test_pytree_tree_leaves" time="13.493" file="dynamo/test_misc.py">
      <failure message="torch._dynamo.exc.Unsupported: 'skip function isclass in file [snip]">Traceback (most recent call last):
[snip]</failure>
      <system-out>inline_call [snip]
</system-out>
    </testcase>
  </testsuite>
</testsuites>

--> tests=2 vs 1 <testcase>

2. A single test might have multiple <skipped> elements:

    <testcase classname="TestNestedTensorOpInfoCPU" name="test_compile_backward_xlogy_cpu_float32" time="9.881" file="test_nestedtensor.py">
      <skipped type="pytest.skip" message="Skipped!">PyTorch/2.7.0/foss-2024a/pytorch-v2.7.0/test/test_nestedtensor.py:8798: Skipped!</skipped>
      <skipped type="pytest.skip" message="Skipped!">PyTorch/2.7.0/foss-2024a/pytorch-v2.7.0/test/test_nestedtensor.py:8798: Skipped!</skipped>
    </testcase>

Additionally I added a try-catch in parse_test_result_file to report the failing file on error. So view the diff with ignored whitespace as except for added comments only if len(test_cases) != num_tests: was removed from that function

[lexming]

Thanks for the quick PR, testing it...

[boegel]

@Flamefire For which PyTorch versions is this "blocking"?

[Flamefire]

At least 2.6+, but IIRC I've seen it for 2.3 too in one occasion.

[boegel]

Test report by @boegel

Overview of tested easyconfigs (in order)

• SUCCESS PyTorch-2.1.2-foss-2023a.eb

Build succeeded (with --ignore-test-failure) for 1 out of 1 (1 easyconfigs in total)
node3505.doduo.os - Linux RHEL 9.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.9.18
See https://gist.github.com/boegel/5a64b169d935099ba7a1b7d2c7f1aab7 for a full test report.

[lexming]

LGTM

[lexming]

Merging, thanks @Flamefire !