-
-
Notifications
You must be signed in to change notification settings - Fork 307
Description
Orchestrator LTS — Version 2.0.0
This is the tracking issue for Orchestrator's Long-Term Support release. LTS represents a fully production-ready, stable orchestrator that replaces the legacy cloud-runner approach with a clean, extensible, well-tested foundation.
The Journey: Cloud Runner to Orchestrator
Cloud Runner started as a way to run Unity builds on remote infrastructure — AWS, Kubernetes, or Docker — when GitHub-hosted runners were not enough. It worked, but it was tightly coupled to specific cloud providers, had limited extensibility, and carried assumptions about infrastructure that did not hold for every team.
Orchestrator was the rewrite. The core insight was that the build execution system should be provider-agnostic — a clean interface that any backend can implement, whether that is AWS, a local machine, a Kubernetes cluster, or an entirely custom system. The rename was not cosmetic; it reflected a fundamentally different architecture:
- Cloud Runner was a cloud-specific build system with hardcoded provider logic
- Orchestrator is a generic build orchestration layer with a pluggable provider model
The migration path from Cloud Runner to Orchestrator is incremental — existing Cloud Runner configurations continue to work, and the provider interface is backward compatible.
With 1.0.0, the new architecture proved itself: the provider model worked, builds were reliable, and the system was stable enough for production use. LTS now takes that foundation and makes it complete.
Why LTS, not just 1.0.0?
1.0.0 meant the system was stable, production-ready, and validated through real-world usage. The architecture worked and the provider model proved itself.
LTS sets a higher bar. LTS means the system now feels complete:
- The architecture has proven itself in real usage
- Rough edges discovered during the 1.x cycle have been addressed
- Convenience and operational workflows are in place
- The reasonable expansions of the system have been implemented
In other words, Orchestrator now covers its domain cleanly and confidently.
The system has been regularly dogfooded in real CI workflows, which has helped flush out the remaining gaps and polish the ergonomics around real CI usage.
We are now past the point where obvious new capabilities belong inside the core. Even for advanced enterprise workflows, the necessary primitives are already there. Future ideas are more likely to live around the ecosystem or as separate tools, rather than expanding the orchestrator core itself.
What LTS Means
- Stable API — Inputs, provider interface, and hook system are locked. Breaking changes require a major version bump.
- Production-tested — All core features validated in real CI pipelines.
- Fully documented — Every feature has documentation with examples.
- Comprehensive tests — Unit and integration test coverage for all services.
- Backward compatible — Existing cloud-runner users can migrate incrementally.
After LTS
From here the expectation is that the project requires much less active maintenance, because the infrastructure and convenience features needed for real-world use are already built in.
Future changes should mostly consist of:
- Bug fixes
- Experimental features graduating to stable
- Ecosystem tooling around the core
Companion Milestone: Unity Editor Tooling
Reaching readiness alongside LTS is the Unity Editor tooling. A new editor package (game-ci-editor-tools) provides a GUI inside Unity for configuring workflows, triggering builds, inspecting caches, and viewing CI status — allowing teams to use Game CI conveniently by quickly installing template workflows to their repo and configuring workflows as needed without even needing to work directly with YAML.
LTS Feature Set
Core Infrastructure
| Feature | Issue | PR | Branch | Status |
|---|---|---|---|---|
| CLI Provider Protocol — any executable as provider | #777 | #777 | feature/orchestrator-enterprise-support |
Complete — 19 inputs, 6 services, 130+ tests |
| Provider load balancing — multi-provider failover | — | #783 | feature/provider-load-balancing |
Complete — 6 inputs, 2 outputs, 11 tests |
| GCP Cloud Run + Azure ACI providers | — | #778 | feature/cloud-run-azure-providers |
Draft — scaffold, being implemented |
Secure git token handling via http.extraHeader |
#785 | #786 | fix/secure-git-token-usage |
Complete — 1 input, 12 tests |
| CI Platform Providers — PowerShell, GitHub Actions, GitLab CI, Ansible | #805 | #806 | feature/ci-platform-providers |
Complete — 4 providers, 15 inputs, 91 tests |
Enterprise & Monorepo Features (PR #777)
All enterprise features are part of the orchestrator enterprise support PR. Together, these provide first-class support for large monorepos and multi-product Unity projects — the kind of repositories where dozens of submodules, massive Library folders, custom LFS backends, and pre-commit hooks are part of daily life:
| Feature | Description |
|---|---|
| Submodule Profiles | YAML-based selective git submodule update --init with glob pattern matching and variant overlays. Critical for monorepos with many submodules — only initialize what each build actually needs. Supports per-product profiles and build-type variants (e.g., client vs server). See docs: #802, #803. |
| Local Build Caching | Filesystem-based caching with atomic folder moves (O(1) on NTFS/ext4) for massive Library folders. For projects with 50GB+ Library folders, move-based caching is orders of magnitude faster than file-by-file copy. |
| Child Workspace Isolation | Create isolated child workspaces from a parent project for multi-product CI builds. Each workspace gets its own Library folder with optional cache root, git config preservation, and separate Library folder support. Prevents concurrent builds from interfering with each other. |
| Custom LFS Transfer Agents | Auto-configure elastic-git-storage, rclone-based agents, or any custom LFS transfer agent. Essential for projects that have outgrown GitHub LFS bandwidth limits or need storage backends like S3, GCS, or Azure Blob. |
| Git Hooks Integration | Lefthook/husky detection, install, skip-list, and Unity Git Hooks (UPM) support with CI-friendly env configuration. Ensures the same pre-commit quality checks that run locally also run in CI. |
| CLI Provider Protocol | Spawn any executable as a build provider — JSON on stdin/stdout, supports streaming output. Allows teams to implement custom providers in any language (Go, Python, Rust, shell) without modifying unity-builder itself. |
Build Reliability
| Feature | Issue | PR | Branch | Status |
|---|---|---|---|---|
| Git integrity, reserved filename cleanup, build archival | #807 | #808 | feature/build-reliability |
Complete — 6 inputs, 5 tests |
Build reliability features harden the pipeline for self-hosted runners and enterprise environments:
| Feature | Description |
|---|---|
| Git Corruption Detection & Recovery | Pre-checkout git fsck, stale lock file cleanup, submodule backing store validation, automatic recovery from corrupted repos. |
| Reserved Filename Cleanup | Scan and remove Windows reserved device names (nul, con, prn, aux, com1-com9, lpt1-lpt9) from Assets/ before Unity opens the project. |
| Build Output Archival | Post-build archival with configurable retention policies to prevent disk exhaustion on self-hosted runners. |
Secret Management
| Feature | Issue | PR | Branch | Status |
|---|---|---|---|---|
| Premade secret sources — AWS SSM, GCP, Azure, HashiCorp Vault | #776 | #787 | feature/premade-secret-sources |
Complete — 1 input, 7 premade sources, 34 tests |
Quality & Testing
| Feature | Issue | PR | Branch | Status |
|---|---|---|---|---|
| Orchestrator unit test suite | — | #784 | feature/orchestrator-unit-tests |
Complete — 65 new mock-based unit tests across 4 files |
Documentation
All documentation lives in the game-ci/documentation repository:
| Topic | Docs PR | Branch | Covers |
|---|---|---|---|
| Provider plugins, jobs, LFS agents, architecture | #532 | document-provider-plugins |
Core orchestrator docs, provider interface, hook system |
| Cloud provider guides (GCP, Azure) | #533 | docs/cloud-providers |
Cloud Run, ACI setup and configuration |
| Build services reference | #535 | docs/build-services |
CLI providers, caching, LFS, hooks reference |
| Load balancing API | #536 | docs/load-balancing-api |
Multi-provider routing and failover |
| Secret sources | #537 | docs/secret-sources |
AWS SSM, GCP SM, Azure KV, Vault integration guides |
| Next-gen features + advanced topics | #538 | docs/test-workflow-hot-runners |
Test workflow, hot runners, artifacts, sync, massive projects, monorepo, build reliability, 4 provider pages |
All LTS PRs
| PR | Branch | Description | Status |
|---|---|---|---|
| #777 | feature/orchestrator-enterprise-support |
Enterprise features — CLI providers, caching, LFS, hooks, submodule profiles, child workspaces | Complete — 19 inputs, 6 services, 130+ tests |
| #778 | feature/cloud-run-azure-providers |
GCP Cloud Run and Azure ACI providers | Draft — scaffold, being implemented |
| #783 | feature/provider-load-balancing |
Provider load balancing and failover | Complete — 6 inputs, 2 outputs, 11 tests |
| #784 | feature/orchestrator-unit-tests |
Orchestrator unit test coverage | Complete — 65 new tests across 4 files |
| #786 | fix/secure-git-token-usage |
Secure git token handling via http.extraHeader | Complete — 1 input, 12 tests |
| #787 | feature/premade-secret-sources |
Premade secret sources (AWS SSM, GCP, Azure, Vault) | Complete — 1 input, 7 premade sources, 34 tests |
| #806 | feature/ci-platform-providers |
Automation & CI dispatch providers (PowerShell, GitHub Actions, GitLab CI, Ansible) | Complete — 4 providers, 15 inputs, 91 tests |
| #808 | feature/build-reliability |
Build reliability — git integrity, reserved filenames, archival | Complete — 6 inputs, 5 tests |
Implementation Summary
New Action Inputs (63 total across all LTS PRs)
Enterprise features (PR #777) — 19 inputs:
- Submodule:
submoduleProfilePath,submoduleVariantPath,submoduleToken - Caching:
localCacheEnabled,localCacheRoot,localCacheLibrary,localCacheLfs - Child workspaces:
childWorkspacesEnabled,childWorkspaceName,childWorkspaceCacheRoot,childWorkspacePreserveGit,childWorkspaceSeparateLibrary - LFS:
lfsTransferAgent,lfsTransferAgentArgs,lfsStoragePaths - Hooks:
gitHooksEnabled,gitHooksSkipList,gitHooksRunBeforeBuild - Provider:
providerExecutable
Load balancing (PR #783) — 6 inputs:
fallbackProviderStrategy,runnerCheckEnabled,runnerCheckLabels,runnerCheckMinAvailable,retryOnFallback,providerInitTimeout
Secure git token (PR #786) — 1 input:
gitAuthMode
Secret sources (PR #787) — 1 input:
secretSource
CI providers (PR #806) — 15 inputs:
- PowerShell:
remotePowershellHost,remotePowershellCredential,remotePowershellTransport - GitHub Actions:
githubActionsRepo,githubActionsWorkflow,githubActionsToken,githubActionsRef - GitLab CI:
gitlabProjectId,gitlabTriggerToken,gitlabApiUrl,gitlabRef - Ansible:
ansibleInventory,ansiblePlaybook,ansibleExtraVars,ansibleVaultPassword
Build reliability (PR #808) — 6 inputs:
gitIntegrityCheck,gitAutoRecover,cleanReservedFilenames,buildArchiveEnabled,buildArchivePath,buildArchiveRetention
New Services
| Service | PR | Location | Purpose |
|---|---|---|---|
SubmoduleProfileService |
#777 | services/submodule/ |
Parse YAML profiles, match .gitmodules, generate init plans, execute selective submodule init |
LocalCacheService |
#777 | services/cache/ |
Cache key generation, tar-based save/restore, garbage collection, atomic moves |
ChildWorkspaceService |
#777 | services/workspace/ |
Create isolated child workspaces from parent project, manage per-workspace Library folders |
LfsAgentService |
#777 | services/lfs/ |
Register custom transfer agents via git config, validate executables, configure storage paths |
GitHooksService |
#777 | services/hooks/ |
Detect lefthook/husky, install hooks, configure skip lists, Unity Git Hooks init, CI env vars |
CliProvider |
#777 | providers/cli/ |
Bridge class implementing ProviderInterface — spawns external executable per method call |
SecretSourceService |
#787 | services/secrets/ |
7 premade secret sources, YAML definitions, custom commands |
BuildReliabilityService |
#808 | services/reliability/ |
Git integrity checks, stale lock cleanup, reserved filename scanning, build archival with retention |
Total test count across LTS PRs
| PR | Tests |
|---|---|
| #777 | 130+ |
| #783 | 11 |
| #784 | 65 |
| #786 | 12 |
| #787 | 34 |
| #806 | 91 |
| #808 | 5 |
| Total | 348+ |
Related Tracking Issues
- Next-Gen Experimental Features (tracking: Orchestrator LTS Next-Gen — experimental features built on LTS foundation #795) — Test Workflow Engine, Hot Runner Protocol, Incremental Sync, Generic Artifacts
- Unity Editor Tooling (feat: Unity Editor Tooling — in-editor GUI for orchestrator job management and workflow configuration #797) — game-ci/game-ci-editor-tools
- Community Plugin Validation (feat: Community Plugin Validation — scheduled compile and build checks for Unity packages #800) — Scheduled CI for community package compatibility
- Orchestrator Isolation (feat: Orchestrator Isolation — clean internal boundaries for extraction readiness #796) — Clean internal boundaries for extraction readiness
Release Criteria
- All LTS PRs merged and CI green
- Full test suite passing (542+ existing + 348+ new service tests)
- Documentation complete for all features with real-world examples
- Migration guide from cloud-runner to orchestrator
- At least one production user validated (GameClient CI)
- Changelog and breaking changes documented
Versioning
LTS will be released as v2.0.0 of unity-builder. The @v4 action tag (GitHub Actions versioning) continues — this is the internal orchestrator version.
Merge Queue
PRs should be reviewed and merged in this order. Each group can be merged in any order within the group, but groups should be completed roughly in sequence since later groups may depend on infrastructure established by earlier ones.
Group 1: Core Infrastructure (merge first)
These establish foundational capabilities that other features build upon.
| PR | Title | Status | Notes |
|---|---|---|---|
| #777 | feat(orchestrator): QoL for large projects — CLI providers, caching, LFS, hooks | Draft | Largest PR — 19 inputs, 6 services, 130+ tests. Foundation for enterprise features. |
| #786 | fix(orchestrator): secure git authentication via http.extraHeader | Ready | Secure token handling. Small, self-contained fix. |
| #784 | test(orchestrator): further unit test coverage | Ready | 65 new tests. Validates core systems before adding more features. |
| #808 | feat(orchestrator): build reliability features — git integrity, reserved filenames, archival | Draft | 6 inputs, 5 tests. Hardens CI for self-hosted runners. |
Group 2: Provider & Orchestrator Enhancements
Build on core infrastructure to add new providers and orchestration capabilities.
| PR | Title | Status | Notes |
|---|---|---|---|
| #783 | feat(orchestrator): built-in load balancing across providers | Draft | 6 inputs, 11 tests. Multi-provider failover. |
| #787 | feat(orchestrator): premade secret sources and YAML definitions | Ready | 7 premade sources, 34 tests. |
| #778 | feat(orchestrator): GCP Cloud Run and Azure ACI providers | Draft | New cloud providers. Scaffold stage. |
| #806 | feat(orchestrator): automation & CI dispatch providers — PowerShell, GitHub Actions, GitLab CI, Ansible | Draft | 4 providers, 15 inputs, 91 tests. |
Group 3: Tooling & CLI
| PR | Title | Status | Notes |
|---|---|---|---|
| #813 | feat: Official CLI Support — game-ci | Draft | CLI distribution of unity-builder. |
Group 4: CI & Testing
| PR | Title | Status | Notes |
|---|---|---|---|
| #809 | ci: split orchestrator integrity into parallel jobs for faster validation | Draft | CI speedup — no feature changes. |
Group 5: Next-Gen Features
These are experimental features tracked under #795.
| PR | Title | Status | Notes |
|---|---|---|---|
| #790 | feat(orchestrator): test workflow engine | Draft | 43 tests. YAML suite definitions + taxonomy filters. |
| #791 | feat(orchestrator): hot runner protocol | Draft | 35 tests. Persistent editor providers. |
| #798 | feat(orchestrator): generic artifact system | Draft | 36+ tests. Multi-type output management. |
| #799 | feat(orchestrator): incremental sync protocol | Draft | 42 tests. 4 sync strategies. |
| #804 | feat(orchestrator): community plugin validation | Draft | Scheduled CI workflow. |
Group 6: Documentation (game-ci/documentation)
These should be merged after the corresponding unity-builder PRs land.
| Docs PR | Title | Status | Covers |
|---|---|---|---|
| #539 | docs(orchestrator): Fix typos and formatting in documentation | Ready | Typo/formatting fixes — can merge anytime |
| #532 | docs(orchestrator): comprehensive documentation update for LTS version | Ready | Core orchestrator docs, provider interface, hook system |
| #535 | docs(orchestrator): QoL build services — CLI providers, caching, LFS, hooks | Draft | CLI providers, caching, LFS, hooks reference |
| #533 | docs(providers): GCP Cloud Run and Azure ACI | Draft | Cloud Run, ACI setup |
| #536 | docs(orchestrator): built-in load balancing across providers | Draft | Load balancing API |
| #537 | docs(secrets): premade secret sources documentation | Draft | AWS SSM, GCP SM, Azure KV, Vault |
| #538 | docs(orchestrator): next-gen features and advanced topics | Draft | Test workflow, hot runners, artifacts, sync, massive projects, monorepo, build reliability |
| #540 | docs(cli): game-ci CLI documentation | Draft | CLI installation, usage, commands |
Community & External PRs (no dependency order)
These are independent community contributions that can be merged at any time:
| PR | Title | Status |
|---|---|---|
| #765 | Windows: add retry logic and explicit failure for Unity license activation | Ready |
| #764 | Add support for parallel linking in IL2CPP builds | Ready |
| #726 | Add parameter linux64RemoveExecutableExtension |
Ready |
| #686 | Fixed AWS to work when no secrets specified | Ready |
Snyk Security PRs (merge as convenient)
| PR | Title |
|---|---|
| #774 | [Snyk] Security upgrade @kubernetes/client-node from 0.16.3 to 1.0.0 |
| #769 | [Snyk] Security upgrade @kubernetes/client-node from 0.16.3 to 1.0.0 |
| #719 | [Snyk] Security upgrade @kubernetes/client-node from 0.16.3 to 1.0.0 |
| #707 | [Snyk] Security upgrade unity-changeset from 2.1.0 to 2.3.0 |
| #696 | [Snyk] Security upgrade @kubernetes/client-node from 0.16.3 to 1.1.0 |
| #693 | [Snyk] Fix for 3 vulnerabilities |
| #684 | [Snyk] Security upgrade unity-changeset from 2.1.0 to 2.3.0 |
| #683 | [Snyk] Upgrade reflect-metadata from 0.1.13 to 0.2.2 |
| #682 | [Snyk] Upgrade unity-changeset from 2.1.0 to 2.3.0 |
| #681 | [Snyk] Upgrade @kubernetes/client-node from 0.16.3 to 0.22.3 |
| #680 | [Snyk] Upgrade yaml from 2.3.1 to 2.6.1 |
Related Issues
| Issue | Title | Status |
|---|---|---|
| #810 | Orchestrator: The Endgame — top-level overview | Open |
| #795 | Orchestrator LTS Next-Gen — experimental features | Open |
| #776 | Premade secret sources | Open |
| #785 | Secure git token handling | Open |
| #805 | Automation & CI Dispatch Providers | Open |
| #807 | Build Reliability Features | Open |
| #797 | Unity Editor Tooling | Open |
| #800 | Community Plugin Validation | Open |
| #796 | Orchestrator Isolation | Open |
| #801 | Claude for Unity-Builder | Open |
| #811 | Help Bot | Open |
| #812 | Official CLI Support | Open |
Hierarchy
#810 (Orchestrator: The Endgame — top-level overview)
├── #794 (LTS v2.0.0) <-- you are here
│ └── #795 (Next-Gen Final Features)
│ ├── #788 (Test Workflow Engine)
│ ├── #789 (Hot Runner Protocol)
│ ├── #792 (Generic Artifact System)
│ ├── #793 (Incremental Sync Protocol)
│ ├── #796 (Orchestrator Isolation)
│ ├── #797 (Unity Editor Tooling)
│ ├── #800 (Community Plugin Validation)
│ ├── #802 (docs: massive projects)
│ ├── #803 (docs: monorepo support)
│ └── #812 (CLI support)
├── #811 (Help Bot)
└── #801 (Claude for Unity-Builder)
Integration PR
All LTS PRs (infrastructure + next-gen) are combined into a single reviewable integration branch:
| PR | Branch | Component PRs |
|---|---|---|
| #815 | release/lts-2.0.0 |
#777, #778, #783, #784, #786, #787, #806, #808, #809, #790, #791, #798, #799, #804, #813 |
This integration branch includes delete-me-update-this-integration-branch.ps1 and delete-me-update-all-integration-branches.ps1 scripts to keep it in sync with component branches.
The infrastructure subset (without next-gen features) is available as #816 on release/lts-infrastructure.
The next-gen subset is also available separately as #814 on release/next-gen.