[WIP] Enable async scheduling by default

[njhill]

Currently just doing full CI test to flush out any issues.

[gemini-code-assist]

Code Review

This pull request enables asynchronous scheduling by default, which is a significant and beneficial change for performance. The implementation involves refactoring the configuration handling to introduce a disable_async_scheduling flag and automatically disabling async scheduling for incompatible features like pipeline parallelism and speculative decoding. The core execution flow is also refactored into a two-step process (execute_model and sample_tokens) to support async structured outputs. The changes are extensive and touch many parts of the codebase, including configuration, core engine logic, executors, and tests.

My review has identified a critical issue with the configuration validation that could lead to unexpected hard failures instead of the intended automatic disabling of async scheduling. I've also found a potential issue in the KV output aggregation logic that might not handle None outputs from all workers correctly. I've provided suggestions to address these issues.

The rest of the changes, including test updates, seem correct and consistent with the goal of this PR.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".