Skip to content

Support surfacing failing tests on generic jobs #3841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
seemethere merged 11 commits into from
Mar 13, 2023
Merged

Support surfacing failing tests on generic jobs #3841

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c2ab5de
try upload test results
pmeier Mar 1, 2023
bc81299
always upload
pmeier Mar 1, 2023
cbad8f5
vendor action
pmeier Mar 1, 2023
45570c3
Revert "vendor action"
pmeier Mar 1, 2023
eca68ab
Merge branch 'main' into test-results
pmeier Mar 3, 2023
f9cb79e
conditionally upload test results
pmeier Mar 3, 2023
6147bcd
cleanup
pmeier Mar 6, 2023
dc8c9f9
Merge branch 'main' into test-results
pmeier Mar 6, 2023
2712344
use custom directory for test results
pmeier Mar 13, 2023
dac66e7
use released version
pmeier Mar 13, 2023
12d6ff0
add support for Windows
pmeier Mar 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/actions/setup-linux/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,11 @@ runs:
run: |
RUNNER_ARTIFACT_DIR="${RUNNER_TEMP}/artifacts"
mkdir -p "${RUNNER_ARTIFACT_DIR}"
echo "RUNNER_ARTIFACT_DIR=${RUNNER_TEMP}/artifacts" >> "${GITHUB_ENV}"
echo "RUNNER_ARTIFACT_DIR=${RUNNER_ARTIFACT_DIR}" >> "${GITHUB_ENV}"

RUNNER_TEST_RESULTS_DIR="${RUNNER_TEMP}/test-results"
mkdir -p "${RUNNER_TEST_RESULTS_DIR}"
echo "RUNNER_TEST_RESULTS_DIR=${RUNNER_TEST_RESULTS_DIR}" >> "${GITHUB_ENV}"

- name: Needs nvidia driver?
shell: bash
Expand Down
6 changes: 5 additions & 1 deletion .github/actions/setup-windows/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,8 @@ runs:
run: |
RUNNER_ARTIFACT_DIR="$(cygpath ${RUNNER_TEMP})/artifacts"
mkdir -p "${RUNNER_ARTIFACT_DIR}"
echo "RUNNER_ARTIFACT_DIR=$(cygpath ${RUNNER_TEMP})/artifacts" >> "${GITHUB_ENV}"
echo "RUNNER_ARTIFACT_DIR=${RUNNER_ARTIFACT_DIR}" >> "${GITHUB_ENV}"

RUNNER_TEST_RESULTS_DIR="${RUNNER_TEMP}/test-results"
mkdir -p "${RUNNER_TEST_RESULTS_DIR}"
echo "RUNNER_TEST_RESULTS_DIR=${RUNNER_TEST_RESULTS_DIR}" >> "${GITHUB_ENV}"
9 changes: 9 additions & 0 deletions .github/workflows/linux_job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ jobs:
# shellcheck disable=SC2086,SC2090
container_name=$(docker run \
-e RUNNER_ARTIFACT_DIR=/artifacts \
-e RUNNER_TEST_RESULTS_DIR=/test-results \
--env-file="${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}" \
`# It is unknown why the container sees a different value for this.` \
--env=GITHUB_STEP_SUMMARY \
Expand All @@ -169,13 +170,21 @@ jobs:
-v "${GITHUB_WORKSPACE}/${REPOSITORY}:/work" \
-v "${GITHUB_WORKSPACE}/test-infra:/test-infra" \
-v "${RUNNER_ARTIFACT_DIR}:/artifacts" \
-v "${RUNNER_TEST_RESULTS_DIR}:/test-results" \
-v "${RUNNER_TEMP}/exec_script:/exec" \
-v "${GITHUB_STEP_SUMMARY}":"${GITHUB_STEP_SUMMARY}" \
-w /work \
"${DOCKER_IMAGE}"
)
docker exec -t "${container_name}" bash /exec

- name: Surface failing tests
if: always()
uses: pmeier/[email protected]
with:
path: ${{ env.RUNNER_TEST_RESULTS_DIR }}
fail-on-empty: false

- name: Chown repository directory
if: ${{ always() }}
uses: ./test-infra/.github/actions/chown-directory
Expand Down
13 changes: 12 additions & 1 deletion .github/workflows/macos_job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,11 @@ jobs:
run: |
RUNNER_ARTIFACT_DIR="${RUNNER_TEMP}/artifacts"
mkdir -p "${RUNNER_ARTIFACT_DIR}"
echo "RUNNER_ARTIFACT_DIR=${RUNNER_TEMP}/artifacts" >> "${GITHUB_ENV}"
echo "RUNNER_ARTIFACT_DIR=${RUNNER_ARTIFACT_DIR}" >> "${GITHUB_ENV}"

RUNNER_TEST_RESULTS_DIR="${RUNNER_TEMP}/test-results"
mkdir -p "${RUNNER_TEST_RESULTS_DIR}"
echo "RUNNER_TEST_RESULTS_DIR=${RUNNER_TEST_RESULTS_DIR}" >> "${GITHUB_ENV}"

- name: Download artifacts (if any)
uses: actions/download-artifact@v3
Expand Down Expand Up @@ -131,6 +135,13 @@ jobs:
done < "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}"
bash "${RUNNER_TEMP}/exec_script"

- name: Surface failing tests
if: always()
uses: pmeier/[email protected]
with:
path: ${{ env.RUNNER_TEST_RESULTS_DIR }}
fail-on-empty: false

- name: Check if there are potential artifacts and move them to the correct artifact location
shell: bash -l {0}
working-directory: ${{ inputs.repository }}
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/windows_job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,13 @@ jobs:
done < "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}"
bash "${RUNNER_TEMP}/exec_script"

- name: Surface failing tests
if: always()
uses: pmeier/[email protected]
with:
path: ${{ env.RUNNER_TEST_RESULTS_DIR }}
fail-on-empty: false

- name: Check if there are potential artifacts and move them to the correct artifact location
shell: bash -l {0}
working-directory: ${{ inputs.repository }}
Expand Down