diff --git a/.github/styles/base/Dictionary.txt b/.github/styles/base/Dictionary.txt index efd8fb2957..14ff1f3a09 100644 --- a/.github/styles/base/Dictionary.txt +++ b/.github/styles/base/Dictionary.txt @@ -13,6 +13,7 @@ agentic Agno Agno's Alertmanager +Alibaba allow_terminated allowlist allowlisting diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-anthropic.md b/app/_how-tos/use-claude-code-with-ai-gateway-anthropic.md new file mode 100644 index 0000000000..b23ad73490 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-anthropic.md @@ -0,0 +1,230 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and Anthropic +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy Advanced + url: /plugins/ai-proxy-advanced/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - ai-proxy-advanced + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + - anthropic + +tldr: + q: How do I run Claude CLI through Kong AI Gateway? + a: Install Claude CLI, configure its API key helper, create a Gateway Service and Route, attach the AI Proxy plugin to forward requests to Claude, enable file-log to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + inline: + - title: Anthropic + icon_url: /assets/icons/anthropic.svg + include_content: prereqs/anthropic + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- + +## Configure the AI Proxy plugin + +First, configure the AI Proxy plugin for the [Anthropic provider](/ai-gateway/ai-providers/#anthropic). +* This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. +* The configuration also raises the maximum request body size to 512 KB to support larger prompts. + +Set `llm_format: anthropic` to tell Kong AI Gateway that requests and responses use Claude's native API format. This parameter controls schema validation and prevents format mismatches between Claude Code and the gateway. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy + config: + logging: + log_statistics: true + log_payloads: false + auth: + header_name: x-api-key + header_value: ${key} + model: + name: claude-sonnet-4-5-20250929 + provider: anthropic + options: + anthropic_version: '2023-06-01' + llm_format: anthropic + logging: + log_statistics: true + max_request_body_size: 524288 + route_type: llm/v1/chat +variables: + key: + value: $ANTHROPIC_API_KEY + description: The API key to use to connect to Anthropic. +{% endentity_examples %} + +## Configure the File Log plugin + +Now, let's enable the [File Log](/plugins/file-log/) plugin on the Service, to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through Kong + +Now, we can start a Claude Code session that points it to the local AI Gateway endpoint: + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=claude-sonnet-4-5-20250929 \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + +```text +Tell me about Madrid Skylitzes manuscript. +``` + +Claude Code might prompt you approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +The Madrid Skylitzes is a remarkable 12th-century illuminated Byzantine +manuscript that represents one of the most important surviving examples +of medieval historical documentation. Here are the key details: + +What it is + +The Madrid Skylitzes is the only surviving illustrated manuscript of John +Skylitzes' "Synopsis of Histories" (Σύνοψις Ἱστοριῶν), which chronicles +Byzantine history from 811 to 1057 CE - covering the period from the death +of Emperor Nicephorus I to the deposition of Michael VI. + +Artistic Significance + +- 574 miniature paintings (with about 100 lost over time) +- Lavishly decorated with gold leaf, vibrant pigments, and intricate +detailing +- Depicts everything from imperial coronations and battles to daily life +in Byzantium +- The only surviving Byzantine illuminated chronicle written in Greek + +Unique Collaboration + +The manuscript is believed to be the work of 7 different artists from +various backgrounds: +- 4 Italian artists +- 1 English or French artist +- 2 Byzantine artists +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + "...": "...", + "headers": { + ... + "user-agent": "claude-cli/2.0.37 (external, cli)", + "content-type": "application/json", + ... + }, + "method": "POST", + ... + "ai": { + "proxy": { + "usage": { + "prompt_tokens": 1, + "completion_tokens_details": {}, + "completion_tokens": 85, + "total_tokens": 86, + "cost": 0, + "time_per_token": 38.941176470588, + "time_to_first_token": 2583, + "prompt_tokens_details": {} + }, + "meta": { + "request_model": "claude-sonnet-4-20250514", + "response_model": "claude-sonnet-4-20250514", + "llm_latency": 3310, + "plugin_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", + "request_mode": "stream", + "provider_name": "anthropic" + } + } + }, + ... +} +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using the `claude-sonnet-4-5-20250929` model we selected while starting the Claude Code session. diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-azure.md b/app/_how-tos/use-claude-code-with-ai-gateway-azure.md new file mode 100644 index 0000000000..723459dd04 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-azure.md @@ -0,0 +1,221 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and Azure +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy + url: /plugins/ai-proxy/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic using Azure OpenAI models + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - ai-proxy-advanced + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + - openai + +tldr: + q: How do I run Claude CLI through Kong AI Gateway for Azure OpenAI models? + a: Install Claude CLI, configure its API key helper, create a Gateway Service and Route, attach the AI Proxy plugin to forward requests to Claude, enable the File Log plugin to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + inline: + - title: Azure + include_content: prereqs/azure-ai + icon_url: /assets/icons/azure.svg + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- +## Configure the AI Proxy plugin + +First, configure the AI Proxy plugin for the [Azure AI provider](/ai-gateway/ai-providers/#azure-ai): +* This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. +* The configuration also raises the maximum request body size to 512 KB to support larger prompts. + +The `llm_format: anthropic` parameter tells Kong AI Gateway to expect request and response payloads that match Claude's native API format. Without this setting, the Gateway would default to OpenAI's format, which would cause request failures when Claude Code communicates with the Azure endpoint. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy + config: + logging: + log_statistics: true + log_payloads: true + route_type: llm/v1/chat + llm_format: anthropic + auth: + header_name: Authorization + header_value: Bearer ${azure_key} + model: + provider: azure + options: + azure_api_version: "2025-01-01-preview" + azure_instance: ${azure_instance} + azure_deployment_id: ${azure_deployment} +variables: + azure_key: + value: "$AZURE_OPENAI_API_KEY" + azure_instance: + value: "$AZURE_INSTANCE_NAME" + azure_deployment: + value: "$AZURE_DEPLOYMENT_ID" +{% endentity_examples %} + +## Configure the File Log plugin + +Now, let's enable the [File Log](/plugins/file-log/) plugin on the Service, to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through AI Gateway + +Now, we can start a Claude Code session that points it to the local AI Gateway endpoint: + +{:.warning} +> Ensure that `ANTHROPIC_MODEL` matches the model you deployed in Azure. + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=YOUR_AZURE_MODEL \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + +```text +Tell me about Vienna Oribasius manuscript. +``` + +Claude Code might prompt you approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +The "Vienna Oribasius manuscript" refers to a famous illustrated medical +codex that preserves the works of Oribasius of Pergamon, a noted Greek +physician who lived in the 4th century CE. Oribasius was a compiler of +earlier medical knowledge, and his writings form an important link in the +transmission of Greco-Roman medical science to the Byzantine, Islamic, and +later European worlds. +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + "...": "...", + "headers": { + ... + "user-agent": "claude-cli/2.0.37 (external, cli)", + "content-type": "application/json", + ... + }, + "method": "POST", + ... + "ai": { + "meta": { + "request_mode": "oneshot", + "response_model": "gpt-4.1-2025-04-14", + "request_model": "gpt-4.1", + "llm_latency": 4606, + "provider_name": "azure", + "azure_deployment_id": "gpt-4.1", + "plugin_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", + "azure_api_version": "2024-12-01-preview", + "azure_instance_id": "example-azure-openai" + }, + "usage": { + "completion_tokens": 414, + "completion_tokens_details": { + "accepted_prediction_tokens": 0, + "audio_tokens": 0, + "rejected_prediction_tokens": 0, + "reasoning_tokens": 0 + }, + "total_tokens": 11559, + "cost": 0, + "time_per_token": 11.125603864734, + "time_to_first_token": 4605, + "prompt_tokens": 11145, + "prompt_tokens_details": { + "audio_tokens": 0, + "cached_tokens": 11008, + "cached_tokens_details": {} + } + } + } + }, +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using the `gpt-4.1` Azure AI model we selected while starting the Claude Code session. diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-bedrock.md b/app/_how-tos/use-claude-code-with-ai-gateway-bedrock.md new file mode 100644 index 0000000000..72af2e1e83 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-bedrock.md @@ -0,0 +1,245 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and AWS Bedrock +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy + url: /plugins/ai-proxy/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic using AWS Bedrock models + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - ai-proxy + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + +tldr: + q: How do I run Claude CLI through Kong AI Gateway with AWS Bedrock? + a: Install Claude CLI, configure its API key helper, create a Gateway Service and Route, attach the AI Proxy plugin to forward requests to AWS Bedrock, enable file-log to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + prereqs: + inline: + - title: AWS Bedrock + icon_url: /assets/icons/bedrock.svg + content: | + 1. Enable model access in AWS Bedrock: + - Sign in to the AWS Management Console + - Navigate to Amazon Bedrock + - Select **Model access** in the left navigation + - Request access to Claude models (for example, `us.anthropic.claude-haiku-4-5-20251001-v1:0`) + - Wait for access approval (typically immediate for most models) + + 2. Create an IAM user with Bedrock permissions: + - Navigate to IAM in the AWS Console + - Create a new user or select an existing user + - Attach the `AmazonBedrockFullAccess` policy or create a custom policy with `bedrock:InvokeModel` permissions + - Create access keys for the user + + 3. Export the Access Key ID, Secret Access Key and AWS region to your environment: + ```sh + export DECK_AWS_ACCESS_KEY_ID='YOUR AWS ACCESS KEY ID' + export DECK_AWS_SECRET_ACCESS_KEY='YOUR AWS SECRET ACCESS KEY' + export DECK_AWS_REGION='YOUR AWS REGION' + ``` + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- + +## Configure the AI Proxy plugin + +Configure the AI Proxy plugin for the [AWS Bedrock provider](/ai-gateway/ai-providers/#bedrock). + +* This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. +* The configuration also raises the maximum token count to 8192 KB to support larger prompts. + +The `llm_format: anthropic` parameter tells Kong AI Gateway to expect request and response payloads that match Claude's native API format. Without this setting, the gateway would default to OpenAI's format, which would cause request failures when Claude Code communicates with the Bedrock endpoint. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy + config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + allow_override: false + aws_access_key_id: ${aws_access_key_id} + aws_secret_access_key: ${aws_secret_access_key} + model: + provider: bedrock + name: us.anthropic.claude-haiku-4-5-20251001-v1:0 + options: + anthropic_version: bedrock-2023-05-31 + bedrock: + aws_region: ${aws_region} + max_tokens: 8192 +variables: + aws_access_key_id: + value: $AWS_ACCESS_KEY_ID + aws_secret_access_key: + value: $AWS_SECRET_ACCESS_KEY + aws_region: + value: $AWS_REGION +{% endentity_examples %} + +## Configure the File Log plugin + +Enable the [File Log](/plugins/file-log/) plugin on the service to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through Kong + +Start a Claude Code session that points to the local AI Gateway endpoint: + +{:.warning} +> Ensure that `ANTHROPIC_MODEL` matches the model you configured in the AI Proxy plugin (for example, `us.anthropic.claude-haiku-4-5-20251001-v1:0`). + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=us.anthropic.claude-haiku-4-5-20251001-v1:0 \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + +```text +Tell me about Anna Komnene's Alexiad. +``` + +Claude Code might prompt you to approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +Anna Komnene (1083-1153?) was a Byzantine princess, scholar, physician, +hospital administrator, and historian. She is known for writing the +Alexiad, a historical account of the reign of her father, Emperor Alexios +I Komnenos (r. 1081-1118). The Alexiad is a valuable primary source for +understanding Byzantine history and the First Crusade. +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + ... + "method": "POST", + "headers": { + "user-agent": "claude-cli/2.0.37 (external, cli)", + "content-type": "application/json" + }, + ... + "ai": { + "proxy": { + "tried_targets": [ + { + "provider": "bedrock", + "model": "us.anthropic.claude-haiku-4-5-20251001-v1:0", + "port": 443, + "upstream_scheme": "https", + "host": "bedrock-runtime.us-west-2.amazonaws.com", + "upstream_uri": "/model/us.anthropic.claude-haiku-4-5-20251001-v1:0/invoke", + "route_type": "llm/v1/chat", + "ip": "xxx.xxx.xxx.xxx" + } + ], + "meta": { + "request_model": "us.anthropic.claude-haiku-4-5-20251001-v1:0", + "request_mode": "oneshot", + "response_model": "us.anthropic.claude-haiku-4-5-20251001-v1:0", + "provider_name": "bedrock", + "llm_latency": 1542, + "plugin_id": "13f5c57a-77b2-4c1f-9492-9048566db7cf" + }, + "usage": { + "completion_tokens": 124, + "completion_tokens_details": {}, + "total_tokens": 11308, + "cost": 0, + "time_per_token": 12.435483870968, + "time_to_first_token": 1542, + "prompt_tokens": 11184, + "prompt_tokens_details": {} + } + } + } + ... +} +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using AWS Bedrock with the `us.anthropic.claude-haiku-4-5-20251001-v1:0` model. \ No newline at end of file diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-dashscope.md b/app/_how-tos/use-claude-code-with-ai-gateway-dashscope.md new file mode 100644 index 0000000000..cf6d531685 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-dashscope.md @@ -0,0 +1,231 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and DashScope +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy + url: /plugins/ai-proxy/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic using Alibaba Cloud DashScope models + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - ai-proxy + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + +tldr: + q: How do I run Claude CLI through Kong AI Gateway with DashScope? + a: Install Claude CLI, configure its API key helper, create a Gateway Service and Route, attach the AI Proxy plugin to forward requests to DashScope, enable file-log to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + prereqs: + inline: + - title: DashScope + icon_url: /assets/icons/dashscope.svg + content: | + You need an active DashScope account with API access. Sign up at the [Alibaba Cloud DashScope platform](https://dashscope.aliyuncs.com/), obtain your API key from the API-KEY interface, and export it to your environment: + ```sh + export DECK_DASHSCOPE_API_KEY='YOUR DASHSCOPE API KEY' + ``` + + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- + +## Configure the AI Proxy plugin + +Configure the AI Proxy plugin for the DashScope provider. +* This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. +* The configuration also raises the maximum token count size to 8192 to support larger prompts. + +The `llm_format: anthropic` parameter tells Kong AI Gateway to expect request and response payloads that match Claude's native API format. Without this setting, the gateway would default to OpenAI's format, which would cause request failures when Claude Code communicates with the DashScope endpoint. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy + config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${dashscope_api_key} + model: + provider: dashscope + name: qwen-plus + options: + max_tokens: 8192 + temperature: 1.0 +variables: + dashscope_api_key: + value: $DASHSCOPE_API_KEY +{% endentity_examples %} + +## Configure the File Log plugin + +Enable the [File Log](/plugins/file-log/) plugin on the service to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through Kong + +Start a Claude Code session that points to the local AI Gateway endpoint: + +{:.warning} +> Ensure that `ANTHROPIC_MODEL` matches the model you configured in the AI Proxy plugin (for example, `qwen-plus`). + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=qwen-plus \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + +```text +Tell me who Niketas Choniates was. +``` + +Claude Code might prompt you to approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +Niketas Choniates was a Byzantine Greek historian and government official +who lived from around 1155 to 1217. He is best known for his historical +work "Historia" (also called "Chronike Diegesis"), which chronicles the +reigns of the Byzantine emperors from 1118 to 1207, covering the period of + the Komnenos and Angelos dynasties. + +Choniates served as a high-ranking official in the Byzantine Empire, +eventually becoming the governor of Athens. His historical writings are +particularly valuable because they provide a detailed eyewitness account +of the Fourth Crusade and the subsequent sack of Constantinople in 1204, +an event he personally experienced and fled from. His account is +considered one of the most important sources for understanding this +pivotal moment in Byzantine history. +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + ... + "upstream_uri": "/compatible-mode/v1/chat/completions?beta=true", + "request": { + "method": "POST", + "headers": { + "user-agent": "claude-cli/2.0.57 (external, cli)", + "content-type": "application/json", + "anthropic-version": "2023-06-01" + } + }, + ... + "ai": { + "proxy": { + "usage": { + "completion_tokens": 493, + "completion_tokens_details": {}, + "total_tokens": 13979, + "cost": 0, + "time_per_token": 34.539553752535, + "time_to_first_token": 17027, + "prompt_tokens": 13486, + "prompt_tokens_details": { + "cached_tokens": 0 + } + }, + "meta": { + "response_model": "qwen-plus", + "plugin_id": "63199335-6c5a-4798-a0ad-f2cbf13cc497", + "request_model": "qwen-plus", + "request_mode": "oneshot", + "provider_name": "dashscope", + "llm_latency": 17028 + } + } + }, + "response": { + "headers": { + "x-kong-llm-model": "dashscope/qwen-plus", + "x-dashscope-call-gateway": "true" + } + } + ... +} +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using DashScope with the `qwen-plus` model. \ No newline at end of file diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-gemini.md b/app/_how-tos/use-claude-code-with-ai-gateway-gemini.md new file mode 100644 index 0000000000..895192b5b2 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-gemini.md @@ -0,0 +1,243 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and Gemini +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy Advanced + url: /plugins/ai-proxy-advanced/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic using Gemini models + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - ai-proxy-advanced + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + +tldr: + q: How do I run Claude CLI through Kong AI Gateway? + a: Install Claude CLI, configure its API key helper, create a Gateway Service and Route, attach the AI Proxy plugin to forward requests to Claude, enable the File Log plugin to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + prereqs: + inline: + - title: Gemini + content: | + Before you begin, you must get the following credentials from Google Cloud: + + - **Service Account Key**: A JSON key file for a service account with Vertex AI permissions + - **Project ID**: Your Google Cloud project identifier + - **Location ID**: The region where your Vertex AI endpoint is deployed (for example, `us-central1`) + - **API Endpoint**: The Vertex AI API endpoint URL (typically `https://{location}-aiplatform.googleapis.com`) + + Export these values as environment variables: + ```sh + export GEMINI_API_KEY="" + export GCP_PROJECT_ID="" + export GEMINI_LOCATION_ID="" + export GEMINI_API_ENDPOINT="" + ``` + icon_url: /assets/icons/gcp.svg + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- + +## Configure the AI Proxy plugin + +First, configure the AI Proxy plugin for the [Gemini provider](/ai-gateway/ai-providers/#gemini): +* This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. +* The configuration also raises the maximum request body size to 512 KB to support larger prompts. + +The `llm_format: anthropic` parameter tells Kong AI Gateway to expect request and response payloads that match Claude's native API format. Without this setting, the Gateway would default to OpenAI's format, which would cause request failures when Claude Code communicates with the Gemini endpoint. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy-advanced + config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + allow_override: false + gcp_use_service_account: true + gcp_service_account_json: ${gcp_service_account_key} + model: + provider: gemini + name: gemini-2.0-flash + options: + gemini: + api_endpoint: ${gcp_api_endpoint} + project_id: ${gcp_project_id} + location_id: ${gcp_location_id} + max_tokens: 8192 +variables: + gcp_service_account_key: + value: $GEMINI_API_KEY + gcp_api_endpoint: + value: $GEMINI_API_ENDPOINT + gcp_project_id: + value: $GCP_PROJECT_ID + gcp_location_id: + value: $GEMINI_LOCATION_ID +{% endentity_examples %} + +## Configure the File Log plugin + +Now, let's enable the [File Log](/plugins/file-log/) plugin on the Service, to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through AI Gateway + +Now, we can start a Claude Code session that points it to the local AI Gateway endpoint: + +{:.warning} +> Ensure that `ANTHROPIC_MODEL` matches the model you deployed in Gemini. + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=YOUR_GEMINI_MODEL \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + +```text +Tell me about Anna Komnene's Alexiad. +``` + +Claude Code might prompt you approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +Anna Komnene (1083-1153?) was a Byzantine princess, scholar, physician, +hospital administrator, and historian. She is known for writing the +Alexiad, a historical account of the reign of her father, Emperor Alexios +I Komnenos (r. 1081-1118). The Alexiad is a valuable primary source for +understanding Byzantine history and the First Crusade. +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + ... + "method": "POST", + "headers": { + "user-agent": "claude-cli/2.0.37 (external, cli)", + "content-type": "application/json" + }, + ... + "ai": { + "proxy": { + "tried_targets": [ + { + "provider": "gemini", + "model": "gemini-2.0-flash", + "port": 443, + "upstream_scheme": "https", + "host": "us-central1-aiplatform.googleapis.com", + "upstream_uri": "/v1/projects/example-project-id/locations/us-central1/publishers/google/models/gemini-2.0-flash:generateContent", + "route_type": "llm/v1/chat", + "ip": "xxx.xxx.xxx.xxx" + } + ], + "meta": { + "request_model": "gemini-2.0-flash", + "request_mode": "oneshot", + "response_model": "gemini-2.0-flash", + "provider_name": "gemini", + "llm_latency": 1694, + "plugin_id": "13f5c57a-77b2-4c1f-9492-9048566db7cf" + }, + "usage": { + "completion_tokens": 19, + "completion_tokens_details": {}, + "total_tokens": 11203, + "cost": 0, + "time_per_token": 89.157894736842, + "time_to_first_token": 1694, + "prompt_tokens": 11184, + "prompt_tokens_details": {} + } + } + } + ... +} +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using the `gemini-2.0-flash` model we selected while starting the Claude Code session. diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-huggingface.md b/app/_how-tos/use-claude-code-with-ai-gateway-huggingface.md new file mode 100644 index 0000000000..6fdc5666d2 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-huggingface.md @@ -0,0 +1,247 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and HuggingFace +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy + url: /plugins/ai-proxy/ + - text: Pre-function + url: /plugins/pre-function/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic using HuggingFace Inference API models + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - pre-function + - ai-proxy + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + +tldr: + q: How do I run Claude CLI through Kong AI Gateway with HuggingFace? + a: Install Claude CLI, configure a pre-function plugin to remove the model field from requests, attach the AI Proxy plugin to forward requests to HuggingFace, enable file-log to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + inline: + - title: HuggingFace + icon_url: /assets/icons/huggingface.svg + content: | + You need an active HuggingFace account with API access. Sign up at [HuggingFace](https://huggingface.co/) and obtain your API token from the [Access Tokens page](https://huggingface.co/settings/tokens). Ensure you have access to the HuggingFace Inference API, and export your token to your environment: + ```sh + export DECK_HUGGINGFACE_API_TOKEN='YOUR HUGGINGFACE API TOKEN' + ``` + + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- + +## Configure the Pre-function plugin + +Claude CLI automatically includes a `model` field in its request payload. However, when the AI Proxy plugin is configured with HuggingFace provider and specific model in its settings, this creates a conflict. The pre-function plugin removes the `model` field from incoming requests before they reach the AI Proxy plugin, ensuring the gateway uses the model you configured rather than the one Claude CLI sends. + +{% entity_examples %} +entities: + plugins: + - name: pre-function + config: + access: + - | + local body = kong.request.get_body("application/json", nil, 10485760) + if not body or body == "" then + return + end + body.model = nil + kong.service.request.set_body(body, "application/json") +{% endentity_examples %} + +## Configure the AI Proxy plugin + +Configure the AI Proxy plugin for the [HuggingFace provider](/ai-gateway/ai-providers/#huggingface). This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. + +The `llm_format: anthropic` parameter tells Kong AI Gateway to expect request and response payloads that match Claude's native API format. Without this setting, the gateway would default to OpenAI's format, which would cause request failures when Claude Code communicates with the HuggingFace endpoint. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy + config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${key} + model: + provider: huggingface + name: meta-llama/Llama-3.3-70B-Instruct +variables: + key: + value: $HUGGINGFACE_API_TOKEN + description: The API token to use to connect to HuggingFace Inference API. +{% endentity_examples %} + +## Configure the File Log plugin + +Enable the [File Log](/plugins/file-log/) plugin on the service to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through Kong + +Start a Claude Code session that points to the local AI Gateway endpoint: + +{:.warning} +> The `ANTHROPIC_MODEL` value can be any string since the pre-function plugin removes it. The actual model used is `meta-llama/Llama-3.3-70B-Instruct` as configured in the AI Proxy plugin. + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=any-model-name \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + +```text +Try creating a logging.py that logs simple http logs. +``` + +Claude Code might prompt you to approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +Create file +╭───────────────────────────────────────────────────────────────────────────────────────────────────────╮ +│ logging.py │ +│ │ +│ import logging │ +│ │ +│ logging.basicConfig(filename='app.log', filemode='a', format='%(name)s - %(levelname)s - │ +│ %(message)s') │ +│ │ +│ def log_info(message): │ +│ logging.info(message) │ +│ │ +│ def log_warning(message): │ +│ logging.warning(message) │ +│ │ +│ def log_error(message): │ +│ logging.error(message) │ +╰───────────────────────────────────────────────────────────────────────────────────────────────────────╯ + Do you want to create logging.py? + ❯ 1. Yes +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + ... + "upstream_uri": "/v1/chat/completions?beta=true", + "request": { + "method": "POST", + "headers": { + "user-agent": "claude-cli/2.0.58 (external, cli)", + "content-type": "application/json", + "anthropic-version": "2023-06-01" + } + }, + ... + "ai": { + "proxy": { + "usage": { + "completion_tokens": 26, + "completion_tokens_details": {}, + "total_tokens": 178, + "cost": 0, + "time_per_token": 52.538461538462, + "time_to_first_token": 1365, + "prompt_tokens": 152, + "prompt_tokens_details": {} + }, + "meta": { + "llm_latency": 1366, + "request_mode": "oneshot", + "plugin_id": "0000b82c-5826-4abf-93b0-2fa230f5e030", + "provider_name": "huggingface", + "response_model": "meta-llama/Llama-3.3-70B-Instruct", + "request_model": "meta-llama/Llama-3.3-70B-Instruct" + } + } + } + ... +} +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using HuggingFace with the `meta-llama/Llama-3.3-70B-Instruct` model. \ No newline at end of file diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-openai.md b/app/_how-tos/use-claude-code-with-ai-gateway-openai.md new file mode 100644 index 0000000000..8ca4bf2f04 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-openai.md @@ -0,0 +1,211 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and OpenAI +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy + url: /plugins/ai-proxy/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic using OpenAI models + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - ai-proxy-advanced + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + - openai + +tldr: + q: How do I run Claude CLI through Kong AI Gateway? + a: Install Claude CLI, configure its API key helper, create a Gateway Service and Route, attach the AI Proxy plugin to forward requests to Claude, enable the File Log plugin to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + inline: + - title: OpenAI + include_content: prereqs/openai + icon_url: /assets/icons/openai.svg + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- + +## Configure the AI Proxy plugin + +First, configure the AI Proxy plugin for the [OpenAI provider](/ai-gateway/ai-providers/#openai): + * This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. + * The configuration also raises the maximum request body size to 512 KB to support larger prompts. + +The `llm_format: anthropic` parameter tells Kong AI Gateway to expect request and response payloads that match Claude's native API format. Without this setting, the Gateway would default to OpenAI's format, which would cause request failures when Claude Code communicates with the OpenAI endpoint. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy + config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${openai_key} + allow_override: false + model: + provider: openai + name: gpt-5-mini + max_request_body_size: 524288 +variables: + openai_key: + value: "$OPENAI_API_KEY" +{% endentity_examples %} + +## Configure the File Log plugin + +Now, let's enable the [File Log](/plugins/file-log/) plugin on the Service, to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through AI Gateway + +Now, we can start a Claude Code session that points it to the local AI Gateway endpoint: + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=gpt-5-mini \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + + +```text +Tell me about Procopius' Secret History. +``` + +Claude Code might prompt you approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +Procopius’ Secret History (Greek: Ἀνέκδοτα, Anekdota) is a fascinating and +notorious work of Byzantine literature written in the 6th century by the +court historian Procopius of Caesarea. Unlike his official histories +(“Wars” and “Buildings”), which paint the Byzantine Emperor Justinian I +and his wife Theodora in a generally positive and conventional manner, the +Secret History offers a scandalous, behind-the-scenes account that +sharply criticizes and even vilifies the emperor, the empress, and other +key figures of the time. +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + ... + "method": "POST", + "headers": { + "user-agent": "claude-cli/2.0.37 (external, cli)", + "content-type": "application/json" + }, + "ai": { + "meta": { + "request_model": "gpt-5-mini", + "request_mode": "oneshot", + "response_model": "gpt-5-mini-2025-08-07", + "provider_name": "openai", + "llm_latency": 6786, + "plugin_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" + }, + "usage": { + "completion_tokens": 456, + "completion_tokens_details": { + "accepted_prediction_tokens": 0, + "audio_tokens": 0, + "rejected_prediction_tokens": 0, + "reasoning_tokens": 256 + }, + "total_tokens": 481, + "cost": 0, + "time_per_token": 14.881578947368, + "time_to_first_token": 6785, + "prompt_tokens": 25, + "prompt_tokens_details": { + "cached_tokens": 0, + "audio_tokens": 0 + } + } + } + ... +} +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using the `gpt-5-mini` model we selected while starting the Claude Code session. diff --git a/app/_how-tos/use-claude-code-with-ai-gateway-vertex.md b/app/_how-tos/use-claude-code-with-ai-gateway-vertex.md new file mode 100644 index 0000000000..7a94de63d0 --- /dev/null +++ b/app/_how-tos/use-claude-code-with-ai-gateway-vertex.md @@ -0,0 +1,243 @@ +--- +title: Route Claude CLI traffic through Kong AI Gateway and Vertex AI +content_type: how_to + +related_resources: + - text: AI Gateway + url: /ai-gateway/ + - text: AI Proxy Advanced + url: /plugins/ai-proxy-advanced/ + - text: File Log + url: /plugins/file-log/ + +description: Configure AI Gateway to proxy Claude CLI traffic using Google Vertex AI models + +products: + - gateway + - ai-gateway + +works_on: + - on-prem + - konnect + +min_version: + gateway: '3.13' + +plugins: + - ai-proxy-advanced + - file-log + +entities: + - service + - route + - plugin + +tags: + - ai + +tldr: + q: How do I run Claude CLI through Kong AI Gateway? + a: Install Claude CLI, configure its API key helper, create a Gateway Service and Route, attach the AI Proxy plugin to forward requests to Claude, enable file-log to inspect traffic, and point Claude CLI to the local proxy endpoint so all LLM requests pass through the AI Gateway for monitoring and control. + +tools: + - deck + +prereqs: + prereqs: + inline: + - title: Vertex + content: | + Before you begin, you must get the following credentials from Google Cloud: + + - **Service Account Key**: A JSON key file for a service account with Vertex AI permissions + - **Project ID**: Your Google Cloud project identifier + - **Location ID**: The region where your Vertex AI endpoint is deployed (for example, `us-central1`) + - **API Endpoint**: The Vertex AI API endpoint URL (typically `https://{location}-aiplatform.googleapis.com`) + + Export these values as environment variables: + ```sh + export GEMINI_API_KEY="" + export GCP_PROJECT_ID="" + export GEMINI_LOCATION_ID="" + export GEMINI_API_ENDPOINT="" + ``` + icon_url: /assets/icons/vertex.svg + - title: Claude Code CLI + icon_url: /assets/icons/third-party/claude.svg + include_content: prereqs/claude-code + entities: + services: + - example-service + routes: + - example-route + +cleanup: + inline: + - title: Clean up Konnect environment + include_content: cleanup/platform/konnect + icon_url: /assets/icons/gateway.svg + - title: Destroy the {{site.base_gateway}} container + include_content: cleanup/products/gateway + icon_url: /assets/icons/gateway.svg +--- + +## Configure the AI Proxy plugin + +First, configure the AI Proxy plugin for the Gemini provider. +* This setup uses the default `llm/v1/chat` route. Claude Code sends its requests to this route. +* The configuration also raises the maximum tokens count size to 8192 to support larger prompts. + +The `llm_format: anthropic` parameter tells Kong AI Gateway to expect request and response payloads that match Claude's native API format. Without this setting, the Gateway would default to OpenAI's format, which would cause request failures when Claude Code communicates with the Gemini endpoint. + +{% entity_examples %} +entities: + plugins: + - name: ai-proxy-advanced + config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + allow_override: false + gcp_use_service_account: true + gcp_service_account_json: ${gcp_service_account_key} + model: + provider: gemini + name: gemini-2.5-flash + options: + gemini: + api_endpoint: ${gcp_api_endpoint} + project_id: ${gcp_project_id} + location_id: ${gcp_location_id} + max_tokens: 8192 +variables: + gcp_service_account_key: + value: $GEMINI_API_KEY + gcp_api_endpoint: + value: $GEMINI_API_ENDPOINT + gcp_project_id: + value: $GCP_PROJECT_ID + gcp_location_id: + value: $GEMINI_LOCATION_ID +{% endentity_examples %} + +## Configure the File Log plugin + +Now, let's enable the [File Log](/plugins/file-log/) plugin on the Service, to inspect the LLM traffic between Claude and the AI Gateway. This creates a local `claude.json` file on your machine. The file records each request and response so you can review what Claude sends through the AI Gateway. + +{% entity_examples %} +entities: + plugins: + - name: file-log + config: + path: "/tmp/claude.json" +{% endentity_examples %} + +## Verify traffic through Kong + +Now, we can start a Claude Code session that points it to the local AI Gateway endpoint: + +{:.warning} +> Ensure that `ANTHROPIC_MODEL` matches the model you deployed in Gemini. + +```sh +ANTHROPIC_BASE_URL=http://localhost:8000/anything \ +ANTHROPIC_MODEL=YOUR_VERTEX_MODEL \ +claude +``` + +Claude Code asks for permission before it runs tools or interacts with files: + +```text +I'll need permission to work with your files. + +This means I can: +- Read any file in this folder +- Create, edit, or delete files +- Run commands (like npm, git, tests, ls, rm) +- Use tools defined in .mcp.json + +Learn more ( https://docs.claude.com/s/claude-code-security ) + +❯ 1. Yes, continue +2. No, exit +``` +{:.no-copy-code} + +Select **Yes, continue**. The session starts. Ask a simple question to confirm that requests reach Kong AI Gateway. + +```text +Tell me about Anna Komnene's Alexiad. +``` + +Claude Code might prompt you approve its web search for answering the question. When you select **Yes**, Claude will produce a full-length response to your request: + +```text +Anna Komnene (1083-1153?) was a Byzantine princess, scholar, physician, +hospital administrator, and historian. She is known for writing the +Alexiad, a historical account of the reign of her father, Emperor Alexios +I Komnenos (r. 1081-1118). The Alexiad is a valuable primary source for +understanding Byzantine history and the First Crusade. +``` +{:.no-copy-code} + +Next, inspect the Kong AI Gateway logs to verify that the traffic was proxied through it: + +```sh +docker exec kong-quickstart-gateway cat /tmp/claude.json | jq +``` + +You should find an entry that shows the upstream request made by Claude Code. A typical log record looks like this: + +```json +{ + ... + "method": "POST", + "headers": { + "user-agent": "claude-cli/2.0.37 (external, cli)", + "content-type": "application/json" + }, + ... + "ai": { + "proxy": { + "tried_targets": [ + { + "provider": "gemini", + "model": "gemini-2.0-flash", + "port": 443, + "upstream_scheme": "https", + "host": "us-central1-aiplatform.googleapis.com", + "upstream_uri": "/v1/projects/example-project-id/locations/us-central1/publishers/google/models/gemini-2.0-flash:generateContent", + "route_type": "llm/v1/chat", + "ip": "xxx.xxx.xxx.xxx" + } + ], + "meta": { + "request_model": "gemini-2.5-flash", + "request_mode": "oneshot", + "response_model": "gemini-2.5-flash", + "provider_name": "gemini", + "llm_latency": 1694, + "plugin_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" + }, + "usage": { + "completion_tokens": 19, + "completion_tokens_details": {}, + "total_tokens": 11203, + "cost": 0, + "time_per_token": 85.157894736842, + "time_to_first_token": 2546, + "prompt_tokens": 11184, + "prompt_tokens_details": {} + } + } + } + ... +} +``` +{:.no-copy-code} + +This output confirms that Claude Code routed the request through Kong AI Gateway using the `gemini-2.5-flash` model we selected while starting the Claude Code session. diff --git a/app/_includes/prereqs/claude-code.md b/app/_includes/prereqs/claude-code.md new file mode 100644 index 0000000000..eb3bec6b33 --- /dev/null +++ b/app/_includes/prereqs/claude-code.md @@ -0,0 +1,46 @@ +1. Install Claude: + + ```sh + curl -fsSL https://claude.ai/install.sh | bash + ``` + +2. Create or edit the Claude settings file: + + ```sh + mkdir -p ~/.claude + nano ~/.claude/settings.json + ``` + + Put this exact content in the file: + + ```json + { + "apiKeyHelper": "~/.claude/anthropic_key.sh" + } + ``` + +3. Create the API key helper script: + + ```sh + nano ~/.claude/anthropic_key.sh + ``` + + Inside, put a dummy API key: + + ```sh + echo "x" + ``` + +4. Make the script executable: + + ```sh + chmod +x ~/.claude/anthropic_key.sh + ``` + +5. Verify it works by running the script: + + ```sh + ~/.claude/anthropic_key.sh + ``` + + You should see only your API key printed. \ No newline at end of file diff --git a/app/_indices/ai-gateway.yaml b/app/_indices/ai-gateway.yaml index 2cd7911e96..539a639421 100644 --- a/app/_indices/ai-gateway.yaml +++ b/app/_indices/ai-gateway.yaml @@ -44,6 +44,9 @@ sections: - title: AI Gateway resource sizing guidelines description: Review Kong's AI Gateway recommended resource allocation sizing guidelines for Kong AI Gateway based on configuration and traffic patterns. url: /ai-gateway/resource-sizing-guidelines-ai/ + - title: Proxy AI CLI tools through Kong AI Gateway" + description: onfigure Kong AI Gateway to proxy requests from AI command-line tools to LLM providers. + url: /ai-gateway/ai-clis/ - title: Gen AI OpenTelemetry attributes reference description: Reference for OpenTelemetry span attributes emitted by Kong AI Gateway for generative AI requests, including model parameters, token usage, and tool-call metadata. url: /ai-gateway/llm-open-telemetry/ diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-anthropic.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-anthropic.yaml new file mode 100644 index 0000000000..d67fa86f1f --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-anthropic.yaml @@ -0,0 +1,41 @@ +title: 'Configure AI Proxy Advanced for Claude Code with Anthropic' +description: 'Set up the AI Proxy Advanced plugin to work with Claude Code, using Anthropic as the LLM provider.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy Advanced plugin to work with Claude Code, using Anthropic as the LLM provider. + For a detailed guide on how to use Anthropic with Claude Code see [this-guide](/how-to/use-claude-code-with-ai-gateway-anthropic/) + +show_in_api: true +weight: 905 + +requirements: +- Anthropic subscription + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_statistics: true + auth: + header_name: x-api-key + header_value: ${key} + model: + name: claude-sonnet-4-5-20250929 + provider: anthropic + options: + anthropic_version: '2023-06-01' + +variables: + key: + value: $ANTHROPIC_API_KEY + description: The API key to use to connect to Anthropic. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-azure.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-azure.yaml new file mode 100644 index 0000000000..a2ff7970a4 --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-azure.yaml @@ -0,0 +1,51 @@ +title: 'Configure AI Proxy for Claude Code with Azure OpenAI' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Azure OpenAI as the LLM provider with GPT-4.1 model.' + +extended_description: | + {% new_in 3.13 %}Set up the AI Proxy plugin to work with Claude Code, using Azure OpenAI as the LLM provider with GPT-4.1 model + For a detailed guide on how to use Azure OpenAI with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-azure/) + +show_in_api: true +weight: 901 + +requirements: +- Azure OpenAI subscription +- Azure deployment configured + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_payloads: true + log_statistics: true + auth: + header_name: "Authorization" + header_value: "Bearer ${azure_api_key}" + model: + provider: azure + name: gpt-4.1 + options: + azure_api_version: "2024-12-01-preview" + azure_instance: "${azure_instance}" + azure_deployment_id: "${azure_deployment_id}" + +variables: + azure_api_key: + value: $AZURE_API_KEY + description: The API key to use to connect to Azure OpenAI. + azure_instance: + value: $AZURE_INSTANCE + description: The Azure OpenAI instance name. + azure_deployment_id: + value: $AZURE_DEPLOYMENT_ID + description: The Azure OpenAI deployment ID for the model. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-bedrock.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-bedrock.yaml new file mode 100644 index 0000000000..e643ded382 --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-bedrock.yaml @@ -0,0 +1,51 @@ +title: 'Configure AI Proxy for Claude Code with AWS Bedrock' +description: 'Set up the AI Proxy plugin to work with Claude Code, using AWS Bedrock with Claude Haiku 4.5 model.' + +extended_description: | + {% new_in 3.13 %}Set up the AI Proxy plugin to work with Claude Code, using AWS Bedrock with Claude Haiku 4.5 model. + For a detailed guide on how to use AWS Bedrock with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-bedrock/) + +show_in_api: true +weight: 903 + +requirements: +- AWS account +- AWS Bedrock access enabled +- AWS IAM credentials configured + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + auth: + allow_override: false + aws_access_key_id: "${aws_access_key_id}" + aws_secret_access_key: "${aws_secret_access_key}" + model: + provider: bedrock + name: us.anthropic.claude-haiku-4-5-20251001-v1:0 + options: + anthropic_version: bedrock-2023-05-31 + bedrock: + aws_region: "${aws_region}" + max_tokens: 8192 + +variables: + aws_access_key_id: + value: $AWS_ACCESS_KEY_ID + description: The AWS access key ID for authentication. + aws_secret_access_key: + value: $AWS_SECRET_ACCESS_KEY + description: The AWS secret access key for authentication. + aws_region: + value: $AWS_REGION + description: The AWS region for Bedrock service (for example, us-west-2). + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-dashscope.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-dashscope.yaml new file mode 100644 index 0000000000..727f7a508a --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-dashscope.yaml @@ -0,0 +1,43 @@ +title: 'Configure AI Proxy Advanced for Claude Code with DashScope' +description: 'Set up the AI Proxy Advanced plugin to work with Claude Code, using Alibaba Cloud DashScope as the LLM provider.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy Advanced plugin to work with Claude Code, using Alibaba Cloud DashScope as the LLM provider with Qwen models. + For a detailed guide on how to use DashScope with Claude Code see [this-guide](/how-to/use-claude-code-with-ai-gateway-dashscope/) + +show_in_api: true +weight: 905 + +requirements: +- DashScope subscription (Alibaba Cloud Model Studio) + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${key} + model: + provider: dashscope + name: qwen-plus + options: + max_tokens: 8192 + temperature: 1.0 + +variables: + key: + value: $DASHSCOPE_API_KEY + description: The API key to use to connect to DashScope. Obtain this from the Alibaba Cloud DashScope platform. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-gemini.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-gemini.yaml new file mode 100644 index 0000000000..b5621f3486 --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-gemini.yaml @@ -0,0 +1,58 @@ +title: 'Configure AI Proxy for Claude Code with Google Gemini' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Google Gemini 2.0 Flash with GCP service account authentication.' + +extended_description: | + {% new_in 3.13%} Set up the AI Proxy plugin to work with Claude Code, using Google Gemini 2.0 Flash with GCP service account authentication. + For a detailed guide on how to use Google Gemini with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-gemini/) + +show_in_api: true +weight: 902 + +requirements: +- Google Cloud Platform account +- Vertex AI API enabled +- Service account with appropriate permissions + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + allow_override: false + gcp_use_service_account: true + gcp_service_account_json: "${gcp_service_account_json}" + model: + provider: gemini + name: gemini-2.0-flash + options: + gemini: + api_endpoint: "${gemini_api_endpoint}" + project_id: "${gcp_project_id}" + location_id: "${gcp_location_id}" + max_tokens: 8192 + +variables: + gcp_service_account_json: + value: $GCP_SERVICE_ACCOUNT_JSON + description: The GCP service account JSON credentials for authentication. + gemini_api_endpoint: + value: $GEMINI_API_ENDPOINT + description: The Gemini API endpoint (for example us-central1-aiplatform.googleapis.com). + gcp_project_id: + value: $GCP_PROJECT_ID + description: The GCP project ID. + gcp_location_id: + value: $GCP_LOCATION_ID + description: The GCP location/region ID (for example us-central1). + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-huggingface.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-huggingface.yaml new file mode 100644 index 0000000000..6ec94760be --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-huggingface.yaml @@ -0,0 +1,41 @@ +title: 'Configure AI Proxy Advanced for Claude Code with HuggingFace' +description: 'Set up the AI Proxy Advanced plugin to work with Claude Code, using HuggingFace Inference API as the LLM provider.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy Advanced plugin to work with Claude Code, using HuggingFace Inference API as the LLM provider with Llama 3.3 70B model. + For a detailed guide on how to use HuggingFace with Claude Code see [this-guide](/how-to/use-claude-code-with-ai-gateway-huggingface/) + +show_in_api: true +weight: 906 + +requirements: +- HuggingFace account with API access +- HuggingFace API token + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${huggingface_token} + model: + provider: huggingface + name: meta-llama/Llama-3.3-70B-Instruct + +variables: + huggingface_token: + value: $HUGGINGFACE_API_TOKEN + description: The API token to use to connect to HuggingFace Inference API. Obtain this from your HuggingFace account settings. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-openai.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-openai.yaml new file mode 100644 index 0000000000..0a01323df3 --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-openai.yaml @@ -0,0 +1,41 @@ +title: 'Configure AI Proxy Advanced for Claude Code with OpenAI' +description: 'Set up the AI Proxy plugin to work with Claude Code, using OpenAI as the LLM provider.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy Advanced plugin to work with Claude Code, using OpenAI as the LLM provider. + For a detailed guide on how to use OpenAI with Claude Code see this [how to guide](/how-to/use-claude-code-with-ai-gateway-openai/) + +show_in_api: true +weight: 900 + +requirements: +- OpenAI subscription + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${key} + allow_override: false + model: + name: gpt-5-mini + provider: openai + +variables: + key: + value: $OPENAI_API_KEY + description: The API key to use to connect to OpenAI. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-vertex.yaml b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-vertex.yaml new file mode 100644 index 0000000000..94016738fd --- /dev/null +++ b/app/_kong_plugins/ai-proxy-advanced/examples/claude-code-vertex.yaml @@ -0,0 +1,58 @@ +title: 'Configure AI Proxy for Claude Code with Google Vertex AI' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Google Vertex AI with Gemini 2.5 Flash model.' + +extended_description: | + {% new_in 3.13 %}Set up the AI Proxy plugin to work with Claude Code, using Google Vertex AI with Gemini 2.5 Flash model. + For a detailed guide on how to use Google Vertex AI with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-vertex/) + +show_in_api: true +weight: 904 + +requirements: +- Google Cloud Platform account +- Vertex AI API enabled +- Service account with Vertex AI permissions + +config: + llm_format: anthropic + targets: + - route_type: llm/v1/chat + logging: + log_payloads: false + log_statistics: true + auth: + allow_override: false + gcp_use_service_account: true + gcp_service_account_json: "${gcp_service_account_json}" + model: + provider: gemini + name: gemini-2.5-flash + options: + anthropic_version: vertex-2023-10-16 + gemini: + api_endpoint: "${vertex_api_endpoint}" + project_id: "${gcp_project_id}" + location_id: "${gcp_location_id}" + +variables: + gcp_service_account_json: + value: $GCP_SERVICE_ACCOUNT_JSON + description: The GCP service account JSON credentials for authentication. + vertex_api_endpoint: + value: $VERTEX_API_ENDPOINT + description: The Vertex AI API endpoint (e.g., us-east5-aiplatform.googleapis.com). + gcp_project_id: + value: $GCP_PROJECT_ID + description: The GCP project ID. + gcp_location_id: + value: $GCP_LOCATION_ID + description: The GCP location/region ID (e.g., us-east5). + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy-advanced/index.md b/app/_kong_plugins/ai-proxy-advanced/index.md index b5f1e46681..8c7b5d1dbc 100644 --- a/app/_kong_plugins/ai-proxy-advanced/index.md +++ b/app/_kong_plugins/ai-proxy-advanced/index.md @@ -68,6 +68,8 @@ examples_groups: text: Azure processing routes - slug: native-routes text: Native routes + - slug: claude-code + text: claude-code faqs: - q: Can I authenticate to Azure AI with Azure Identity? diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-anthropic.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-anthropic.yaml new file mode 100644 index 0000000000..833923eea5 --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-anthropic.yaml @@ -0,0 +1,41 @@ +title: 'Configure AI Proxy for Claude Code with Anthropic' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Anthropic as the LLM provider.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy plugin to work with Claude Code, using Anthropic as the LLM provider. + For a detailed guide on how to use Anthropic with Claude Code see this [how to guide](/how-to/use-claude-code-with-ai-gateway-anthropic/) + +show_in_api: true +weight: 900 + +requirements: +- Anthropic subscription + +config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + auth: + header_name: x-api-key + header_value: ${key} + max_request_body_size: 524288 + model: + name: claude-sonnet-4-5-20250929 + provider: anthropic + options: + anthropic_version: '2023-06-01' + +variables: + key: + value: $ANTHROPIC_API_KEY + description: The API key to use to connect to Anthropic. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-azure.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-azure.yaml new file mode 100644 index 0000000000..81c342aca7 --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-azure.yaml @@ -0,0 +1,50 @@ +title: 'Configure AI Proxy for Claude Code with Azure OpenAI' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Azure OpenAI as the LLM provider with GPT-4.1 model.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy plugin to work with Claude Code, using Azure OpenAI as the LLM provider with GPT-4.1 model. + For a detailed guide on how to use Azure OpenAI with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-azure/) + +show_in_api: true +weight: 901 + +requirements: +- Azure OpenAI subscription +- Azure deployment configured + +config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_payloads: true + log_statistics: true + auth: + header_name: "Authorization" + header_value: "Bearer ${azure_api_key}" + model: + provider: azure + name: gpt-4.1 + options: + azure_api_version: "2024-12-01-preview" + azure_instance: "${azure_instance}" + azure_deployment_id: "${azure_deployment_id}" + +variables: + azure_api_key: + value: $AZURE_API_KEY + description: The API key to use to connect to Azure OpenAI. + azure_instance: + value: $AZURE_INSTANCE + description: The Azure OpenAI instance name. + azure_deployment_id: + value: $AZURE_DEPLOYMENT_ID + description: The Azure OpenAI deployment ID for the model. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-bedrock.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-bedrock.yaml new file mode 100644 index 0000000000..cdbd7d25fe --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-bedrock.yaml @@ -0,0 +1,50 @@ +title: 'Configure AI Proxy for Claude Code with AWS Bedrock' +description: 'Set up the AI Proxy plugin to work with Claude Code, using AWS Bedrock with Claude Haiku 4.5 and API version bedrock-2023-05-31.' + +extended_description: | + {% new_in %}Set up the AI Proxy plugin to work with Claude Code, using AWS Bedrock with Claude Haiku 4.5 and API version bedrock-2023-05-31. + For a detailed guide on how to use AWS Bedrock with Claude Code see [/how-to/use-claude-code-with-ai-gateway-bedrock](/how-to/use-claude-code-with-ai-gateway-bedrock/) + +show_in_api: true +weight: 903 + +requirements: +- AWS account +- AWS Bedrock access enabled +- AWS IAM credentials configured + +config: + llm_format: anthropic + route_type: llm/v1/chat + auth: + allow_override: false + aws_access_key_id: "${aws_access_key_id}" + aws_secret_access_key: "${aws_secret_access_key}" + model: + provider: bedrock + name: us.anthropic.claude-haiku-4-5-20251001-v1:0 + options: + anthropic_version: bedrock-2023-05-31 + bedrock: + aws_region: "${aws_region}" + max_tokens: 8192 + +variables: + aws_access_key_id: + value: $AWS_ACCESS_KEY_ID + description: The AWS access key ID for authentication. + aws_secret_access_key: + value: $AWS_SECRET_ACCESS_KEY + description: The AWS secret access key for authentication. + aws_region: + value: $AWS_REGION + description: The AWS region for Bedrock service (for example, us-west-2). + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-dashscope.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-dashscope.yaml new file mode 100644 index 0000000000..068026bd2a --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-dashscope.yaml @@ -0,0 +1,43 @@ +title: 'Configure AI Proxy for Claude Code with DashScope (Alibaba Cloud)' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Alibaba Cloud DashScope as the LLM provider with Qwen models.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy plugin to work with Claude Code, using Alibaba Cloud DashScope as the LLM provider with Qwen models. + For a detailed guide on how to use DashScope with Claude Code see this [how to guide](/how-to/use-claude-code-with-ai-gateway-dashscope/) + +show_in_api: true +weight: 900 + +requirements: +- DashScope subscription (Alibaba Cloud Model Studio) + +config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${key} + max_request_body_size: 524288 + model: + provider: dashscope + name: qwen-plus + options: + max_tokens: 8192 + temperature: 1.0 + +variables: + key: + value: $DASHSCOPE_API_KEY + description: The API key to use to connect to DashScope. Obtain this from the Alibaba Cloud DashScope platform. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-gemini.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-gemini.yaml new file mode 100644 index 0000000000..d791d9359a --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-gemini.yaml @@ -0,0 +1,57 @@ +title: 'Configure AI Proxy for Claude Code with Google Gemini' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Google Gemini 2.0 Flash with GCP service account authentication.' + +extended_description: | + {% new_in 3.13 %}Set up the AI Proxy plugin to work with Claude Code, using Google Gemini 2.0 Flash with GCP service account authentication. + For a detailed guide on how to use Google Gemini with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-gemini/) + +show_in_api: true +weight: 902 + +requirements: +- Google Cloud Platform account +- Vertex AI API enabled +- Service account with appropriate permissions + +config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + allow_override: false + gcp_use_service_account: true + gcp_service_account_json: "${gcp_service_account_json}" + model: + provider: gemini + name: gemini-2.0-flash + options: + gemini: + api_endpoint: "${gemini_api_endpoint}" + project_id: "${gcp_project_id}" + location_id: "${gcp_location_id}" + max_tokens: 8192 + +variables: + gcp_service_account_json: + value: $GCP_SERVICE_ACCOUNT_JSON + description: The GCP service account JSON credentials for authentication. + gemini_api_endpoint: + value: $GEMINI_API_ENDPOINT + description: The Gemini API endpoint (for example, us-central1-aiplatform.googleapis.com). + gcp_project_id: + value: $GCP_PROJECT_ID + description: The GCP project ID. + gcp_location_id: + value: $GCP_LOCATION_ID + description: The GCP location/region ID (for example, us-central1). + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-huggingface.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-huggingface.yaml new file mode 100644 index 0000000000..af69cfd256 --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-huggingface.yaml @@ -0,0 +1,40 @@ +title: 'Configure AI Proxy for Claude Code with HuggingFace' +description: 'Set up the AI Proxy plugin to work with Claude Code, using HuggingFace Inference API as the LLM provider with Llama models.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy plugin to work with Claude Code, using HuggingFace Inference API as the LLM provider with Llama 3.3 70B model. + For a detailed guide on how to use HuggingFace with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-huggingface/) + +show_in_api: true +weight: 901 + +requirements: +- HuggingFace account with API access +- HuggingFace API token + +config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_payloads: false + log_statistics: true + auth: + header_name: Authorization + header_value: Bearer ${huggingface_token} + model: + provider: huggingface + name: meta-llama/Llama-3.3-70B-Instruct + +variables: + huggingface_token: + value: $HUGGINGFACE_API_TOKEN + description: The API token to use to connect to HuggingFace Inference API. Obtain this from your HuggingFace account settings. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-openai.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-openai.yaml new file mode 100644 index 0000000000..3e3f4ffb9a --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-openai.yaml @@ -0,0 +1,40 @@ +title: 'Configure AI Proxy for Claude Code with OpenAI' +description: 'Set up the AI Proxy plugin to work with Claude Code, using OpenAI as the LLM provider.' + +extended_description: | + {% new_in 3.13 %} Set up the AI Proxy plugin to work with Claude Code, using OpenAI as the LLM provider. + For a detailed guide on how to use OpenAI with Claude Code see this [how to guide](/how-to/use-claude-code-with-ai-gateway-openai/) + +show_in_api: true +weight: 900 + +requirements: +- OpenAI subscription + +config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_statistics: true + log_payloads: false + auth: + header_name: Authorization + header_value: Bearer ${key} + allow_override: false + model: + name: gpt-5-mini + provider: openai + +variables: + key: + value: $OPENAI_API_KEY + description: The API key to use to connect to OpenAI. + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/examples/claude-code-vertex.yaml b/app/_kong_plugins/ai-proxy/examples/claude-code-vertex.yaml new file mode 100644 index 0000000000..4ef1d54dfb --- /dev/null +++ b/app/_kong_plugins/ai-proxy/examples/claude-code-vertex.yaml @@ -0,0 +1,57 @@ +title: 'Configure AI Proxy for Claude Code with Google Vertex AI' +description: 'Set up the AI Proxy plugin to work with Claude Code, using Google Vertex AI with Gemini 2.5 Flash model.' + +extended_description: | + {% new_in 3.13 %}Set up the AI Proxy plugin to work with Claude Code, using Google Vertex AI with Gemini 2.5 Flash model. + For a detailed guide on how to use Google Vertex AI with Claude Code see [this guide](/how-to/use-claude-code-with-ai-gateway-vertex/) + +show_in_api: true +weight: 904 + +requirements: +- Google Cloud Platform account +- Vertex AI API enabled +- Service account with Vertex AI permissions + +config: + llm_format: anthropic + route_type: llm/v1/chat + logging: + log_payloads: false + log_statistics: true + auth: + allow_override: false + gcp_use_service_account: true + gcp_service_account_json: "${gcp_service_account_json}" + model: + provider: gemini + name: gemini-2.5-flash + options: + anthropic_version: vertex-2023-10-16 + gemini: + api_endpoint: "${vertex_api_endpoint}" + project_id: "${gcp_project_id}" + location_id: "${gcp_location_id}" + +variables: + gcp_service_account_json: + value: $GCP_SERVICE_ACCOUNT_JSON + description: The GCP service account JSON credentials for authentication. + vertex_api_endpoint: + value: $VERTEX_API_ENDPOINT + description: The Vertex AI API endpoint (for example, us-east5-aiplatform.googleapis.com). + gcp_project_id: + value: $GCP_PROJECT_ID + description: The GCP project ID. + gcp_location_id: + value: $GCP_LOCATION_ID + description: The GCP location/region ID (for example, us-east5). + +tools: + - deck + - admin-api + - konnect-api + - kic + - terraform + +group: claude-code \ No newline at end of file diff --git a/app/_kong_plugins/ai-proxy/index.md b/app/_kong_plugins/ai-proxy/index.md index 91cd537fd7..6c3617a9a8 100644 --- a/app/_kong_plugins/ai-proxy/index.md +++ b/app/_kong_plugins/ai-proxy/index.md @@ -65,6 +65,8 @@ examples_groups: text: Azure processing routes - slug: native-routes text: Native routes + - slug: claude-code + text: Claude Code use cases faqs: - q: Can I authenticate to Azure AI with Azure Identity? diff --git a/app/_landing_pages/ai-gateway.yaml b/app/_landing_pages/ai-gateway.yaml index bbca227718..60ddbc64b8 100644 --- a/app/_landing_pages/ai-gateway.yaml +++ b/app/_landing_pages/ai-gateway.yaml @@ -158,40 +158,40 @@ rows: text: | You can enable the AI Gateway features through a set of modern and specialized plugins, using the same model you use for any other {{site.base_gateway}} plugin. When deployed alongside existing {{site.base_gateway}} plugins, {{site.base_gateway}} users can quickly assemble a sophisticated AI management platform without custom code or deploying new and unfamiliar tools. - - columns: - - blocks: - - type: card - config: - title: Universal API - description: Route client requests to various AI providers. - icon: /assets/icons/plugins/universal-api.svg - cta: - url: ./#universal-api - align: end - - blocks: - - type: card - config: - title: Rate limiting - description: Manage traffic to your LLM API. - icon: /assets/icons/plugins/ai-rate-limiting-advanced.png - cta: - url: /plugins/ai-rate-limiting-advanced/ - align: end - - blocks: - - type: card - config: - title: Semantic caching - description: Semantically cache responses from LLMs. - icon: /assets/icons/plugins/ai-semantic-cache.png - cta: - url: /plugins/ai-semantic-cache/ - align: end - - columns: + - column_count: 3 + columns: + - blocks: + - type: card + config: + title: Universal API + description: Route client requests to various AI providers + icon: /assets/icons/plugins/universal-api.svg + cta: + url: ./#universal-api + align: end + - blocks: + - type: card + config: + title: Rate limiting + description: Manage traffic to your LLM API + icon: /assets/icons/plugins/ai-rate-limiting-advanced.png + cta: + url: /plugins/ai-rate-limiting-advanced/ + align: end + - blocks: + - type: card + config: + title: Semantic caching + description: Semantically cache responses from LLMs + icon: /assets/icons/plugins/ai-semantic-cache.png + cta: + url: /plugins/ai-semantic-cache/ + align: end - blocks: - type: card config: title: Semantic routing - description: Semantically distribute requests to different LLM models. + description: Semantically distribute requests to different LLM models icon: /assets/icons/plugins/ai-proxy-advanced.png cta: url: /plugins/ai-proxy-advanced/examples/semantic/ @@ -209,17 +209,16 @@ rows: - type: card config: title: Automated RAG injection - description: Automatically embed RAG logic into your workflows. + description: Automatically embed RAG logic into your workflows icon: /assets/icons/plugins/ai-rag-injector.png cta: url: ./#automated-rag align: end - - columns: - blocks: - type: card config: title: Data governance - description: Use AI plugins to control AI data and usage. + description: Use AI plugins to control AI data and usage icon: /assets/icons/security.svg cta: url: ./#data-governance @@ -228,7 +227,7 @@ rows: - type: card config: title: Guardrails - description: Inspect requests and configure content safety and moderation. + description: Inspect requests and configure content safety and moderation icon: /assets/icons/lock.svg cta: url: ./#guardrails-and-content-safety @@ -237,17 +236,16 @@ rows: - type: card config: title: Prompt engineering - description: Create prompt templates and manipulate client prompts. + description: Create prompt templates and manipulate client prompts icon: /assets/icons/code.svg cta: url: ./#prompt-engineering align: end - - columns: - blocks: - type: card config: title: Load balancing - description: Learn about the load balancing algorithms available for AI Gateway. + description: Learn about the load balancing algorithms available for AI Gateway icon: /assets/icons/load-balance.svg cta: url: ./#load-balancing @@ -255,12 +253,30 @@ rows: - blocks: - type: card config: - title: Observability and metrics - description: Learn about AI Gateway observability capabilities. - icon: /assets/icons/analytics.svg + title: Audit log + description: Learn about AI Gateway logging capabilities + icon: /assets/icons/audit.svg + cta: + url: /ai-gateway/ai-audit-log-reference/ + align: end + - blocks: + - type: card + config: + title: LLM metrics + description: Expose and visualize LLM metrics + icon: /assets/icons/monitor.svg cta: url: ./#observability-and-metrics align: end + - blocks: + - type: card + config: + title: '{{site.konnect_short_name}} Advanced Analytics' + description: Visualize LLM metrics in {{site.konnect_short_name}} + icon: /assets/icons/analytics.svg + cta: + url: /advanced-analytics/llm-reporting/ + align: end - blocks: - type: card config: @@ -270,7 +286,6 @@ rows: cta: url: /ai-gateway/streaming/ align: end - - columns: - blocks: - type: card config: @@ -293,11 +308,20 @@ rows: - type: card config: title: Request transformations - description: Use AI to transform requests and responses. + description: Use AI to transform requests and responses icon: /assets/icons/plugins/ai-request-transformer.png cta: url: ./#request-transformations align: end + - blocks: + - type: card + config: + title: Proxy AI CLI tools through Kong AI Gateway + description: Configure Kong AI Gateway to proxy requests from AI command-line tools to LLM providers + icon: /assets/icons/terminal.svg + cta: + url: /ai-gateway/ai-clis/ + align: end - header: @@ -451,7 +475,7 @@ rows: - type: plugin config: slug: ai-lakera-guard - icon: ai-lakera.png + icon: ai-lakera.png - blocks: - type: card config: diff --git a/app/_landing_pages/ai-gateway/ai-clis.yaml b/app/_landing_pages/ai-gateway/ai-clis.yaml new file mode 100644 index 0000000000..8cdcfc4d05 --- /dev/null +++ b/app/_landing_pages/ai-gateway/ai-clis.yaml @@ -0,0 +1,145 @@ +metadata: + title: "Proxy AI CLI tools through Kong AI Gateway" + content_type: landing_page + description: Configure Kong AI Gateway to proxy requests from AI command-line tools to LLM providers for logging, cost tracking, and rate limiting. + products: + - ai-gateway + works_on: + - on-prem + - konnect + breadcrumbs: + - /ai-gateway/ + tags: + - ai +rows: + - header: + type: h1 + text: "Proxy AI CLI tools through Kong AI Gateway" + columns: + - blocks: + - type: structured_text + config: + blocks: + - type: text + text: | + Kong AI Gateway can proxy requests from AI command-line tools to LLM providers. This gives you centralized control over AI traffic: log all requests, track costs across teams, enforce rate limits, or apply security policies and guardrails. + + Supported AI CLI tools: + + - [**Claude Code**](#claude-code): Anthropic, OpenAI, Azure OpenAI, Google Gemini, Google Vertex, AWS Bedrock, and Alibaba Cloud (Dashscope) + - [**Codex CLI**](#codex-cli): OpenAI + + + {:.info} + > **Current limitations:** + > * Load balancing or failover features currently only work if all providers share the same model identifier. + > * Streaming is not supported when using non-Claude models with the following providers: Azure OpenAI, Google Gemini, and AWS Bedrock. Token usage might be reported as 0, but otherwise functionality is not affected. + + - header: + type: h3 + text: "Claude Code" + columns: + - blocks: + - type: structured_text + config: + blocks: + - type: text + text: | + Claude Code is Anthropic's command-line tool that delegates coding tasks to Claude AI. Route Claude Code requests through Kong AI Gateway to monitor usage, control costs, and enforce rate limits across your development team. + - column_count: 4 + columns: + - blocks: + - type: card + config: + title: Claude Code with Anthropic + description: Use Claude Code with Anthropic provider + icon: /assets/icons/anthropic.svg + cta: + url: /how-to/use-claude-code-with-ai-gateway-anthropic/ + align: end + - blocks: + - type: card + config: + title: Claude Code with OpenAI + icon: /assets/icons/openai.svg + description: Use Claude Code with OpenAI provider + cta: + url: /how-to/use-claude-code-with-ai-gateway-openai/ + align: end + - blocks: + - type: card + config: + title: Claude Code with Azure AI + icon: /assets/icons/azure.svg + description: Use Claude Code with Azure AI provider + cta: + url: /how-to/use-claude-code-with-ai-gateway-azure/ + align: end + - blocks: + - type: card + config: + title: Claude Code with Gemini + icon: /assets/icons/gcp.svg + description: Use Claude Code with Gemini provider + cta: + url: /how-to/use-claude-code-with-ai-gateway-gemini/ + align: end + - blocks: + - type: card + config: + title: Claude Code with Vertex AI + icon: /assets/icons/vertex.svg + description: Use Claude Code with Vertex AI provider + cta: + url: /how-to/use-claude-code-with-ai-gateway-vertex/ + align: end + - blocks: + - type: card + config: + title: Claude Code with Bedrock + icon: /assets/icons/bedrock.svg + description: Use Claude Code with Bedrock provider + cta: + url: /how-to/use-claude-code-with-ai-gateway-bedrock/ + align: end + - blocks: + - type: card + config: + title: Claude Code with Alibaba Cloud + icon: /assets/icons/alibaba-cloud.svg + description: Use Claude Code with Alibaba Cloud (Dashscope) provider + cta: + url: /how-to/use-claude-code-with-ai-gateway-dashscope/ + align: end + - blocks: + - type: card + config: + title: Claude Code with HuggingFace + icon: /assets/icons/huggingface.svg + description: Use Claude Code with HuggingFace provider + cta: + url: /how-to/use-claude-code-with-ai-gateway-bedrock/ + align: end + - header: + type: h3 + text: "Codex CLI" + columns: + - blocks: + - type: structured_text + config: + blocks: + - type: text + text: | + Codex CLI is OpenAI's command-line tool for code generation and assistance. Proxy Codex CLI requests through Kong AI Gateway to gain visibility into API usage, implement rate limiting, and centralize credential management. + + - column_count: 4 + columns: + - blocks: + - type: card + config: + title: Codex CLI with OpenAI + description: Use Codex CLI with OpenAI models + icon: /assets/icons/openai.svg + cta: + url: /how-to/use-codex-with-ai-gateway/ + align: end \ No newline at end of file diff --git a/app/assets/icons/alibaba-cloud.svg b/app/assets/icons/alibaba-cloud.svg new file mode 100644 index 0000000000..503a55b26b --- /dev/null +++ b/app/assets/icons/alibaba-cloud.svg @@ -0,0 +1,4 @@ + + + + diff --git a/app/assets/icons/vertex.svg b/app/assets/icons/vertex.svg new file mode 100644 index 0000000000..1a6a483ab5 --- /dev/null +++ b/app/assets/icons/vertex.svg @@ -0,0 +1,51 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/tools/broken-link-checker/config/ignored_targets.json b/tools/broken-link-checker/config/ignored_targets.json index bf874b49f5..f385302d49 100644 --- a/tools/broken-link-checker/config/ignored_targets.json +++ b/tools/broken-link-checker/config/ignored_targets.json @@ -110,5 +110,7 @@ "https://docs.splunk.com/*", "https://konghq.com/compliance", "https://developer.hashicorp.com/*", - "https://www.meetup.com/topics/kong/all" + "https://www.meetup.com/topics/kong/all", + "https://dashscope.aliyuncs.com/", + "https://huggingface.co/settings/tokens" ]