Skip to content

Rename openrlhf to marti, add HF cache under repo, update deps (torch 2.9, vllm)#20

Open
deema-A wants to merge 1 commit intoTsinghuaC3I:mainfrom
deema-A:marti-openrlhf-rename-and-deps
Open

Rename openrlhf to marti, add HF cache under repo, update deps (torch 2.9, vllm)#20
deema-A wants to merge 1 commit intoTsinghuaC3I:mainfrom
deema-A:marti-openrlhf-rename-and-deps

Conversation

@deema-A
Copy link
Copy Markdown

@deema-A deema-A commented Mar 17, 2026

Description

Updates the codebase to use the marti package name instead of openrlhf, improves the multi-agent training script for local/simple runs, and updates dependencies.

Changes

Package rename (openrlhf → marti)

  • examples/multi-agent/run_train_chain.sh: Switched CLI and workflow paths from openrlhf to marti (e.g. openrlhf.cli.multi_agent_train_ppo_raymarti.cli.multi_agent_train_ppo_ray, workflow/processor paths under marti/agent_workflows/).
  • marti/trainer/ray/vllm_engine.py: Updated worker_extension_cls from openrlhf.trainer.ray.vllm_worker_wrap.WorkerWrap to marti.trainer.ray.vllm_worker_wrap.WorkerWrap.

Script / environment

  • ROOT_DIR: If unset, derived from the script path so the script works from any working directory.
  • Hugging Face cache: Set HF_DATASETS_CACHE and HF_HOME under the repo (e.g. ${ROOT_DIR}/.cache/huggingface) to avoid PermissionError when ~/.cache/huggingface isn’t writable (e.g. on shared systems).
  • Defaults: MODEL_DIR="Qwen", MAX_LEN=30000 for the example run.

Dependencies (requirements.txt)

  • Pinned torch to 2.9; added vllm >0.8.5.post1.
  • Added: json5, latex2sympy2, pylatexenc, srsly, word2number.

Testing

  • Multi-agent chain training script runs with the new paths and env.
  • No remaining references to openrlhf in the updated code paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants