Skip to content

[gRPC] gRPC-in-gRPC-out for Generate API #2166

Description

@zetxqx

Change EPP to implement gRPC-to-gRPC for Generate API (Non-streaming & Streaming)

  • Import and manage vLLM gRPC proto definitions (e.g., in pkg/epp/proto).
  • Implement gRPC message pass-through/parsing for the Generate API.
  • Ensure support for both unary and server-side streaming gRPC calls.

Metadata

Metadata

Assignees

Labels

needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions