Skip to content

Commit 6028739

Browse files
slimfrkhahuangjunyi.0
authored andcommitted
[env] fix: Improve License Check Hook Flexibility (volcengine#3202)
### What does this PR do? Solve volcengine#3201 #### Problem The existing license check hook scans all directories recursively from a single root directory, which causes issues in local development environments: * Virtual environments (`.venv`, `venv/`) get scanned and fail license checks * No easy way to exclude common build/cache directories without hardcoding exclusions * Different behavior between local development (with venvs) and CI/CD (clean environment) #### Solution Modified the `check_license.py` script to accept multiple target directories instead of a single root directory with exclusions. ### Design & Code Changes Changed argument from `--directory` to `--directories` * Now accepts multiple `Path` arguments using `nargs="+"` * Allows specifying exactly which directories to scan * in local mode: `--directories examples recipe scripts tests verl setup.py` * in github workflow: `--directories .`
1 parent 8380463 commit 6028739

File tree

3 files changed

+31
-6
lines changed

3 files changed

+31
-6
lines changed

.github/workflows/sanity.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
# - `special_sanity`: a suite of quick sanity tests
1313
# - `special_standalone`: a set of test that are designed to run in dedicated environments
1414

15-
# Accelerators for tests
15+
# Accelerators for tests
1616
# - By default tests are run with GPU available, except for the ones under `special_npu`, and any test script whose name ends with `on_cpu.py`.
1717
# - For test scripts with `on_cpu.py` name suffix would be tested on CPU resources in linux environment.
1818

@@ -78,7 +78,7 @@ jobs:
7878
pytest -s -x tests/special_sanity
7979
- name: Run license test
8080
run: |
81-
python3 tests/special_sanity/check_license.py --directory .
81+
python3 tests/special_sanity/check_license.py --directories .
8282
- name: Assert naming convention
8383
run: |
8484
if grep -rIn --exclude-dir=.git --exclude-dir=.github --exclude-dir=venv --exclude-dir=__pycache__ 'veRL' .; then

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,6 @@ repos:
3232
hooks:
3333
- id: check-license
3434
name: Check license
35-
entry: python3 tests/special_sanity/check_license.py --directory .
35+
entry: python3 tests/special_sanity/check_license.py --directories examples recipe scripts tests verl setup.py
3636
language: python
3737
pass_filenames: false

tests/special_sanity/check_license.py

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
# limitations under the License.
1414
from argparse import ArgumentParser
1515
from pathlib import Path
16+
from typing import Iterable
1617

1718
license_head_bytedance = "Copyright 2024 Bytedance Ltd. and/or its affiliates"
1819
license_head_bytedance_25 = "Copyright 2025 Bytedance Ltd. and/or its affiliates"
@@ -35,13 +36,37 @@
3536
]
3637

3738

39+
def get_py_files(path_arg: Path) -> Iterable[Path]:
40+
"""get py files under a dir. if already py file return it
41+
42+
Args:
43+
path_arg (Path): path to scan for py files
44+
45+
Returns:
46+
Iterable[Path]: list of py files
47+
"""
48+
if path_arg.is_dir():
49+
return path_arg.glob("**/*.py")
50+
elif path_arg.is_file() and path_arg.suffix == ".py":
51+
return [path_arg]
52+
return []
53+
54+
3855
if __name__ == "__main__":
3956
parser = ArgumentParser()
40-
parser.add_argument("--directory", "-d", required=True, type=str)
57+
parser.add_argument(
58+
"--directories",
59+
"-d",
60+
required=True,
61+
type=Path,
62+
nargs="+",
63+
help="List of directories to check for license headers",
64+
)
4165
args = parser.parse_args()
42-
directory_in_str = args.directory
4366

44-
pathlist = Path(directory_in_str).glob("**/*.py")
67+
# Collect all Python files from specified directories
68+
pathlist = set(path for path_arg in args.directories for path in get_py_files(path_arg))
69+
4570
for path in pathlist:
4671
# because path is object not string
4772
path_in_str = str(path.absolute())

0 commit comments

Comments
 (0)