Goal
Make integration-test selection configurable and uniform across the product repos via a standard Makefile interface:
make test TEST_TYPE=<smoke|core|full|...>
TEST_TYPE (a flag/variable) controls which tests run. The implementation is repo-independent — this repo maps TEST_TYPE to its own underlying selection (a -k/-m expression, a model-key list, a marker set, etc.) behind the same make test interface. Every CI caller and local developer then selects scope identically across torch-spyre / hf-adapters / spyre-inference.
Why
Today this repo's test scope is hard-coded in the workflow (a fixed matrix), so callers can't independently pick a subset. Standardizing on make test TEST_TYPE=... (a) gives one knob for scope, (b) makes the Makefile uniform across the product repos, and (c) decouples what to run (the variable) from how it's run (repo-internal).
Proposal
- Makefile: add/extend a
test target that takes TEST_TYPE (default full, back-compatible) and maps each level to this repo's selection mechanism internally. Keep the target name and TEST_TYPE vocabulary consistent with the other product repos.
- Workflow: add an
.github/workflows/integration-tests.yaml that exposes TEST_TYPE as a workflow_dispatch input and runs make test TEST_TYPE=${{ inputs.test_type }} instead of an inline hard-coded matrix. (A stub of this workflow is being added separately so the integration pipeline can start driving it; this issue tracks making it real + the Makefile.)
- Define the level vocabulary (
smoke, core, full, …) consistently with torch-spyre.
Back-compat
TEST_TYPE defaults to full (current behavior). No change to test code required.
Scope
Makefile test target + TEST_TYPE plumbing + a real integration-tests.yaml consuming it. Companion issues: torch-spyre (label/tag-driven selection) and the other product repo.
Goal
Make integration-test selection configurable and uniform across the product repos via a standard Makefile interface:
TEST_TYPE(a flag/variable) controls which tests run. The implementation is repo-independent — this repo mapsTEST_TYPEto its own underlying selection (a-k/-mexpression, a model-key list, a marker set, etc.) behind the samemake testinterface. Every CI caller and local developer then selects scope identically across torch-spyre / hf-adapters / spyre-inference.Why
Today this repo's test scope is hard-coded in the workflow (a fixed
matrix), so callers can't independently pick a subset. Standardizing onmake test TEST_TYPE=...(a) gives one knob for scope, (b) makes the Makefile uniform across the product repos, and (c) decouples what to run (the variable) from how it's run (repo-internal).Proposal
testtarget that takesTEST_TYPE(defaultfull, back-compatible) and maps each level to this repo's selection mechanism internally. Keep the target name andTEST_TYPEvocabulary consistent with the other product repos..github/workflows/integration-tests.yamlthat exposesTEST_TYPEas aworkflow_dispatchinput and runsmake test TEST_TYPE=${{ inputs.test_type }}instead of an inline hard-coded matrix. (A stub of this workflow is being added separately so the integration pipeline can start driving it; this issue tracks making it real + the Makefile.)smoke,core,full, …) consistently with torch-spyre.Back-compat
TEST_TYPEdefaults tofull(current behavior). No change to test code required.Scope
Makefile
testtarget +TEST_TYPEplumbing + a realintegration-tests.yamlconsuming it. Companion issues: torch-spyre (label/tag-driven selection) and the other product repo.