Skip to content

Commit d4e114b

Browse files
HecarimVCH3CHO
HecarimV
and
CH3CHO
authored
feat(ai-proxy): support Google Cloud Vertex (#2119)
Co-authored-by: Kent Dong <[email protected]>
1 parent e674c78 commit d4e114b

File tree

4 files changed

+840
-1
lines changed

4 files changed

+840
-1
lines changed

‎plugins/wasm-go/extensions/ai-proxy/README.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,19 @@ Dify 所对应的 `type` 为 `dify`。它特有的配置字段如下:
262262
| `inputVariable` | string | 非必填 | - | dify 中应用类型为 workflow 时需要设置输入变量，当 botType 为 workflow 时一起使用 |
263263
| `outputVariable` | string | 非必填 | - | dify 中应用类型为 workflow 时需要设置输出变量，当 botType 为 workflow 时一起使用 |
264264

265+
#### Google Vertex AI
266+
267+
Google Vertex AI 所对应的 type 为 vertex。它特有的配置字段如下：
268+
269+
| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
270+
|-----------------------------|---------------|--------|--------|-------------------------------------------------------------------------------|
271+
| `vertexAuthKey` | string | 必填 | - | 用于认证的 Google Service Account JSON Key，格式为 PEM 编码的 PKCS#8 私钥和 client_email 等信息 |
272+
| `vertexRegion` | string | 必填 | - | Google Cloud 区域（如 us-central1, europe-west4 等），用于构建 Vertex API 地址 |
273+
| `vertexProjectId` | string | 必填 | - | Google Cloud 项目 ID，用于标识目标 GCP 项目 |
274+
| `vertexAuthServiceName` | string | 必填 | - | 用于 OAuth2 认证的服务名称，该服务为了访问oauth2.googleapis.com |
275+
| `vertexGeminiSafetySetting` | map of string | 非必填 | - | Gemini 模型的内容安全过滤设置。 |
276+
| `vertexTokenRefreshAhead` | number | 非必填 | - | Vertex access token刷新提前时间(单位秒) |
277+
265278
## 用法示例
266279

267280
### 使用 OpenAI 协议代理 Azure OpenAI 服务
@@ -1629,6 +1642,69 @@ provider:
16291642
}
16301643
```
16311644

1645+
### 使用 OpenAI 协议代理 Google Vertex 服务
1646+
1647+
**配置信息**
1648+
1649+
```yaml
1650+
provider:
1651+
type: vertex
1652+
vertexAuthKey: |
1653+
{
1654+
"type": "service_account",
1655+
"project_id": "your-project-id",
1656+
"private_key_id": "your-private-key-id",
1657+
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
1658+
"client_email": "[email protected]",
1659+
"token_uri": "https://oauth2.googleapis.com/token"
1660+
}
1661+
vertexRegion: us-central1
1662+
vertexProjectId: your-project-id
1663+
vertexAuthServiceName: your-auth-service-name
1664+
```
1665+
1666+
**请求示例**
1667+
1668+
```json
1669+
{
1670+
"model": "gemini-2.0-flash-001",
1671+
"messages": [
1672+
{
1673+
"role": "user",
1674+
"content": "你好，你是谁？"
1675+
}
1676+
],
1677+
"stream": false
1678+
}
1679+
```
1680+
1681+
**响应示例**
1682+
1683+
```json
1684+
{
1685+
"id": "chatcmpl-0000000000000",
1686+
"choices": [
1687+
{
1688+
"index": 0,
1689+
"message": {
1690+
"role": "assistant",
1691+
"content": "你好！我是 Vertex AI 提供的 Gemini 模型，由 Google 开发的人工智能助手。我可以回答问题、提供信息和帮助完成各种任务。有什么我可以帮您的吗？"
1692+
},
1693+
"finish_reason": "stop"
1694+
}
1695+
],
1696+
"created": 1729986750,
1697+
"model": "gemini-2.0-flash-001",
1698+
"object": "chat.completion",
1699+
"usage": {
1700+
"prompt_tokens": 15,
1701+
"completion_tokens": 43,
1702+
"total_tokens": 58
1703+
}
1704+
}
1705+
```
1706+
1707+
16321708
## 完整配置示例
16331709

16341710
### Kubernetes 示例

‎plugins/wasm-go/extensions/ai-proxy/README_EN.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,18 @@ For DeepL, the corresponding `type` is `deepl`. Its unique configuration field i
208208
| ------------ | --------- | ----------- | ------- | ------------------------------------ |
209209
| `targetLang` | string | Required | - | The target language required by the DeepL translation service |
210210

211+
#### Google Vertex AI
212+
For Vertex, the corresponding `type` is `vertex`. Its unique configuration field is:
213+
214+
| Name | Data Type | Requirement | Default | Description |
215+
|-----------------------------|---------------|---------------| ------ |-------------------------------------------------------------------------------------------------------------------------------------------------------------|
216+
| `vertexAuthKey` | string | Required | - | Google Service Account JSON Key used for authentication. The format should be PEM encoded PKCS#8 private key along with client_email and other information |
217+
| `vertexRegion` | string | Required | - | Google Cloud region (e.g., us-central1, europe-west4) used to build the Vertex API address |
218+
| `vertexProjectId` | string | Required | - | Google Cloud Project ID, used to identify the target GCP project |
219+
| `vertexAuthServiceName` | string | Required | - | Service name for OAuth2 authentication, used to access oauth2.googleapis.com |
220+
| `vertexGeminiSafetySetting` | map of string | Optional | - | Gemini model content safety filtering settings. |
221+
| `vertexTokenRefreshAhead` | number | Optional | - | Vertex access token refresh ahead time in seconds |
222+
211223
## Usage Examples
212224

213225
### Using OpenAI Protocol Proxy for Azure OpenAI Service
@@ -1411,6 +1423,64 @@ provider:
14111423
}
14121424
```
14131425

1426+
### Utilizing OpenAI Protocol Proxy for Google Vertex Services
1427+
**Configuration Information**
1428+
```yaml
1429+
provider:
1430+
type: vertex
1431+
vertexAuthKey: |
1432+
{
1433+
"type": "service_account",
1434+
"project_id": "your-project-id",
1435+
"private_key_id": "your-private-key-id",
1436+
"private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
1437+
"client_email": "[email protected]",
1438+
"token_uri": "https://oauth2.googleapis.com/token"
1439+
}
1440+
vertexRegion: us-central1
1441+
vertexProjectId: your-project-id
1442+
vertexAuthServiceName: your-auth-service-name
1443+
```
1444+
1445+
**Request Example**
1446+
```json
1447+
{
1448+
"model": "gemini-2.0-flash-001",
1449+
"messages": [
1450+
{
1451+
"role": "user",
1452+
"content": "Who are you?"
1453+
}
1454+
],
1455+
"stream": false
1456+
}
1457+
```
1458+
1459+
**Response Example**
1460+
```json
1461+
{
1462+
"id": "chatcmpl-0000000000000",
1463+
"choices": [
1464+
{
1465+
"index": 0,
1466+
"message": {
1467+
"role": "assistant",
1468+
"content": "Hello! I am the Gemini model provided by Vertex AI, developed by Google. I can answer questions, provide information, and assist in completing various tasks. How can I help you today?"
1469+
},
1470+
"finish_reason": "stop"
1471+
}
1472+
],
1473+
"created": 1729986750,
1474+
"model": "gemini-2.0-flash-001",
1475+
"object": "chat.completion",
1476+
"usage": {
1477+
"prompt_tokens": 15,
1478+
"completion_tokens": 43,
1479+
"total_tokens": 58
1480+
}
1481+
}
1482+
```
1483+
14141484
## Full Configuration Example
14151485

14161486
### Kubernetes Example

‎plugins/wasm-go/extensions/ai-proxy/provider/provider.go‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@ const (
9090
providerTypeTogetherAI = "together-ai"
9191
providerTypeDify = "dify"
9292
providerTypeBedrock = "bedrock"
93+
providerTypeVertex = "vertex"
9394

9495
protocolOpenAI = "openai"
9596
protocolOriginal = "original"
@@ -161,6 +162,7 @@ var (
161162
providerTypeTogetherAI: &togetherAIProviderInitializer{},
162163
providerTypeDify: &difyProviderInitializer{},
163164
providerTypeBedrock: &bedrockProviderInitializer{},
165+
providerTypeVertex: &vertexProviderInitializer{},
164166
}
165167
)
166168

@@ -298,6 +300,21 @@ type ProviderConfig struct {
298300
// @Title zh-CN Gemini AI内容过滤和安全级别设定
299301
// @Description zh-CN 仅适用于 Gemini AI 服务。参考：https://ai.google.dev/gemini-api/docs/safety-settings
300302
geminiSafetySetting map[string]string `required:"false" yaml:"geminiSafetySetting" json:"geminiSafetySetting"`
303+
// @Title zh-CN Vertex AI访问区域
304+
// @Description zh-CN 仅适用于Vertex AI服务。如需查看支持的区域的完整列表，请参阅https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations?hl=zh-cn#available-regions
305+
vertexRegion string `required:"false" yaml:"vertexRegion" json:"vertexRegion"`
306+
// @Title zh-CN Vertex AI项目Id
307+
// @Description zh-CN 仅适用于Vertex AI服务。创建和管理项目请参阅https://cloud.google.com/resource-manager/docs/creating-managing-projects?hl=zh-cn#identifiers
308+
vertexProjectId string `required:"false" yaml:"vertexProjectId" json:"vertexProjectId"`
309+
// @Title zh-CN Vertex 认证秘钥
310+
// @Description zh-CN 用于Google服务账号认证的完整JSON密钥文件内容，获取可参考https://cloud.google.com/iam/docs/keys-create-delete?hl=zh-cn#iam-service-account-keys-create-console
311+
vertexAuthKey string `required:"false" yaml:"vertexAuthKey" json:"vertexAuthKey"`
312+
// @Title zh-CN Vertex 认证服务名
313+
// @Description zh-CN 用于Google服务账号认证的服务,DNS类型的服务名
314+
vertexAuthServiceName string `required:"false" yaml:"vertexAuthServiceName" json:"vertexAuthServiceName"`
315+
// @Title zh-CN Vertex token刷新提前时间
316+
// @Description zh-CN 用于Google服务账号认证，access token过期时间判定提前刷新，单位为秒，默认值为60秒
317+
vertexTokenRefreshAhead int64 `required:"false" yaml:"vertexTokenRefreshAhead" json:"vertexTokenRefreshAhead"`
301318
// @Title zh-CN 翻译服务需指定的目标语种
302319
// @Description zh-CN 翻译结果的语种，目前仅适用于DeepL服务。
303320
targetLang string `required:"false" yaml:"targetLang" json:"targetLang"`
@@ -390,12 +407,20 @@ func (c *ProviderConfig) FromJson(json gjson.Result) {
390407
c.minimaxApiType = json.Get("minimaxApiType").String()
391408
c.minimaxGroupId = json.Get("minimaxGroupId").String()
392409
c.cloudflareAccountId = json.Get("cloudflareAccountId").String()
393-
if c.typ == providerTypeGemini {
410+
if c.typ == providerTypeGemini || c.typ == providerTypeVertex {
394411
c.geminiSafetySetting = make(map[string]string)
395412
for k, v := range json.Get("geminiSafetySetting").Map() {
396413
c.geminiSafetySetting[k] = v.String()
397414
}
398415
}
416+
c.vertexRegion = json.Get("vertexRegion").String()
417+
c.vertexProjectId = json.Get("vertexProjectId").String()
418+
c.vertexAuthKey = json.Get("vertexAuthKey").String()
419+
c.vertexAuthServiceName = json.Get("vertexAuthServiceName").String()
420+
c.vertexTokenRefreshAhead = json.Get("vertexTokenRefreshAhead").Int()
421+
if c.vertexTokenRefreshAhead == 0 {
422+
c.vertexTokenRefreshAhead = 60
423+
}
399424
c.targetLang = json.Get("targetLang").String()
400425

401426
if schemaValue, ok := json.Get("responseJsonSchema").Value().(map[string]interface{}); ok {

0 commit comments

Comments
 (0)