Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ length of 512 tokens:
- [Distributed Tracing](#distributed-tracing)
- [gRPC](#grpc)
- [Local Install](#local-install)
- [Apple Silicon (Homebrew)](#apple-silicon-homebrew)
- [Docker Build](#docker-build)
- [Apple M1/M2 Arm](#apple-m1m2-arm64-architectures)
- [Examples](#examples)
Expand Down Expand Up @@ -492,6 +493,22 @@ grpcurl -d '{"inputs": "What is Deep Learning"}' -plaintext 0.0.0.0:8080 tei.v1.

## Local install

### Apple Silicon (Homebrew)

On Apple Silicon (M1/M2/M3/M4), you can install a prebuilt binary via Homebrew:

```shell
brew install text-embeddings-inference
```

Then launch Text Embeddings Inference with Metal acceleration:

```shell
model=Qwen/Qwen3-Embedding-0.6B

text-embeddings-router --model-id $model --port 8080
```

### CPU

You can also opt to install `text-embeddings-inference` locally.
Expand Down
21 changes: 20 additions & 1 deletion docs/source/en/local_metal.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,26 @@ rendered properly in your Markdown viewer.
# Using TEI locally with Metal

You can install `text-embeddings-inference` locally to run it on your own Mac with Metal support.
Here are the step-by-step instructions for installation:

## Homebrew (Apple Silicon)

On Apple Silicon (M1/M2/M3/M4), you can install a prebuilt binary via Homebrew:

```shell
brew install text-embeddings-inference
```

Then launch Text Embeddings Inference:

```shell
model=Qwen/Qwen3-Embedding-0.6B

text-embeddings-router --model-id $model --port 8080
```

## Build from source

Alternatively, you can build from source. Here are the step-by-step instructions:

## Step 1: Install Rust

Expand Down