Skip to content

Update docs for Couchbase #316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update docs for Couchbase #316

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
213 changes: 163 additions & 50 deletions ...ncepts/vector-store-connectors/out-of-the-box-connectors/couchbase-connector.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ zone_pivot_groups: programming-languages
author: azaddhirajkumar
ms.topic: conceptual
ms.author: westey
ms.date: 01/14/2025
ms.date: 11/03/2025
ms.service: semantic-kernel
---

Expand All @@ -14,27 +14,26 @@ ms.service: semantic-kernel
> [!WARNING]
> The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release.

::: zone pivot="programming-language-csharp"

## Overview

The Couchbase Vector Store connector can be used to access and manage data in Couchbase. The connector has the
following characteristics.

| Feature Area | Support |
|---------------------------------------|-------------------------------------------------------------------------------------------------------------------|
| Collection maps to | Couchbase collection |
| Supported key property types | string |
| Supported data property types | All types that are supported by System.Text.Json (either built-in or by using a custom converter) |
| Supported vector property types | <ul><li>ReadOnlyMemory\<float\></li></ul> |
| Supported index types | N/A |
| Supported distance functions | <ul><li>CosineSimilarity</li><li>DotProductSimilarity</li><li>EuclideanDistance</li></ul> |
| Supported filter clauses | <ul><li>AnyTagEqualTo</li><li>EqualTo</li></ul> |
| Supports multiple vectors in a record | Yes |
| IsFilterable supported? | No |
| IsFullTextSearchable supported? | No |
| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) |
| HybridSearch supported? | No |
| Feature Area | Support |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Collection maps to | Couchbase collection + index |
| Supported key property types | <ul><li>`string`</li></ul> |
| Supported data property types | All types that are supported by `System.Text.Json` (either built-in or by using a custom converter) |
| Supported vector property types | <ul><li>`ReadOnlyMemory<float>`</li><li>`Embedding<float>`</li><li>`float[]`</li></ul> |
| Supported distance functions | <ul><li>CosineSimilarity</li><li>DotProductSimilarity</li><li>EuclideanDistance</li></ul> |
| Supported filter clauses | <ul><li>`AnyTagEqualTo`</li><li>`EqualTo`</li></ul> |
| Supports multiple vectors in a record | Yes |
| IsIndexed supported? | Yes |
| IsFullTextIndexed supported? | Yes |
| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) |
| HybridSearch supported? | Yes |

::: zone pivot="programming-language-csharp"

## Getting Started

Expand All @@ -44,14 +43,15 @@ Add the Couchbase Vector Store connector NuGet package to your project.
dotnet add package CouchbaseConnector.SemanticKernel --prerelease
```

You can add the vector store to the dependency injection container available on the `KernelBuilder` or to
the `IServiceCollection` dependency injection container using extension methods provided by Semantic Kernel.
You can add the vector store to the dependency injection container available on the `KernelBuilder` or to the `IServiceCollection` dependency injection container using extension methods provided by Semantic Kernel.

```csharp
using Microsoft.SemanticKernel;
using Couchbase.SemanticKernel;

// Using Kernel Builder.
var kernelBuilder = Kernel.CreateBuilder()
var kernelBuilder = Kernel
.CreateBuilder()
.AddCouchbaseVectorStore(
connectionString: "couchbases://your-cluster-address",
username: "username",
Expand All @@ -61,7 +61,8 @@ var kernelBuilder = Kernel.CreateBuilder()
```

```csharp
using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Couchbase.SemanticKernel;

// Using IServiceCollection with ASP.NET Core.
var builder = WebApplication.CreateBuilder(args);
Expand All @@ -73,6 +74,53 @@ builder.Services.AddCouchbaseVectorStore(
scopeName: "scope-name");
```

**Configuring Index Type**

The vector store defaults to using Hyperscale indexes. You can specify a different index type by passing `CouchbaseVectorStoreOptions`:

```csharp
using Microsoft.SemanticKernel;
using Couchbase.SemanticKernel;

var builder = WebApplication.CreateBuilder(args);

// Option 1: Use Hyperscale index
builder.Services.AddCouchbaseVectorStore(
connectionString: "couchbases://your-cluster-address",
username: "username",
password: "password",
bucketName: "bucket-name",
scopeName: "scope-name",
options: new CouchbaseVectorStoreOptions
{
IndexType = CouchbaseIndexType.Hyperscale
});

// Option 2: Use Search/FTS index
builder.Services.AddCouchbaseVectorStore(
connectionString: "couchbases://your-cluster-address",
username: "username",
password: "password",
bucketName: "bucket-name",
scopeName: "scope-name",
options: new CouchbaseVectorStoreOptions
{
IndexType = CouchbaseIndexType.Search
});

// Option 3: Use Composite index
builder.Services.AddCouchbaseVectorStore(
connectionString: "couchbases://your-cluster-address",
username: "username",
password: "password",
bucketName: "bucket-name",
scopeName: "scope-name",
options: new CouchbaseVectorStoreOptions
{
IndexType = CouchbaseIndexType.Composite
});
```

Extension methods that take no parameters are also provided. These require an instance of the `IScope` class to be
separately registered with the dependency injection container.

Expand Down Expand Up @@ -154,7 +202,6 @@ var clusterOptions = new ClusterOptions
};

var cluster = await Cluster.ConnectAsync(clusterOptions);

var bucket = await cluster.BucketAsync("bucket-name");
var scope = bucket.Scope("scope-name");

Expand All @@ -163,80 +210,146 @@ var vectorStore = new CouchbaseVectorStore(scope);

It is possible to construct a direct reference to a named collection.

### Using Search Collection (FTS Index)

For Full-Text Search indexes and hybrid search scenarios:

```csharp
using Couchbase.SemanticKernel;
using Couchbase;
using Couchbase.KeyValue;
using Couchbase.SemanticKernel;

var clusterOptions = new ClusterOptions
{
ConnectionString = "couchbases://your-cluster-address",
UserName = "username",
Password = "password"
};
var cluster = await Cluster.ConnectAsync(clusterOptions);
var bucket = await cluster.BucketAsync("bucket-name");
var scope = bucket.Scope("scope-name");

var collection = new CouchbaseSearchCollection<string, Hotel>(
scope,
"skhotels");
```

### Using Query Collection (Hyperscale or Composite Index)

For high-performance vector search with Hyperscale indexes:

```csharp
using Couchbase.SemanticKernel;
using Couchbase;
using Couchbase.KeyValue;

var cluster = await Cluster.ConnectAsync(clusterOptions);
var bucket = await cluster.BucketAsync("bucket-name");
var scope = bucket.Scope("scope-name");

var collection = new CouchbaseFtsVectorStoreRecordCollection<Hotel>(
// Using Hyperscale index (default)
var collection = new CouchbaseQueryCollection<string, Hotel>(
scope,
"skhotels",
indexType: CouchbaseIndexType.Hyperscale);

// Or using Composite index
var collectionComposite = new CouchbaseQueryCollection<string, Hotel>(
scope,
"hotelCollection");
"skhotels",
indexType: CouchbaseIndexType.Composite);
```

### Index Type Comparison

Couchbase offers three types of indexes for vector search:

**Search (FTS) - Full-Text Search Index**

- Best for hybrid searches combining full-text search with vector similarity
- Allows semantic search alongside traditional keyword matching
- Supports geospatial searches in addition to vector and text
- **Use when:** You need to combine traditional keyword search with vector similarity search in the same query
- **Ideal for:** E-commerce product search, travel recommendations, content discovery with multiple search criteria
- **Requires:** Couchbase Server 7.6+ or Capella

**Hyperscale Vector Index**

- Best for pure vector searches - content discovery, recommendations, semantic search
- High performance with low memory footprint - designed to scale to billions of vectors
- Optimized for concurrent operations - supports simultaneous searches and inserts
- **Use when:** You primarily perform vector-only queries without complex scalar filtering
- **Ideal for:** Large-scale semantic search, recommendation systems, content discovery
- **Requires:** Couchbase Server 8.0+ or Capella

**Composite Vector Index**

- Best for filtered vector searches - combines vector search with scalar value filtering
- Efficient pre-filtering - scalar attributes reduce the vector comparison scope
- **Use when:** Your queries combine vector similarity with scalar filters that eliminate large portions of data
- **Ideal for:** Compliance-based filtering, user-specific searches, time-bounded queries
- **Requires:** Couchbase Server 8.0+ or Capella

**Choosing the Right Index Type:**

- Start with **Hyperscale** for pure vector searches and large datasets (scales to billions)
- Use **Search (FTS)** for hybrid search combining text and vectors
- Choose **Composite** when scalar filters significantly reduce your search space (works well for tens of millions to billions of vectors)

[Detailed comparison of vector index types](https://docs.couchbase.com/server/current/vector-index/use-vector-indexes.html)
## Data mapping

The Couchbase connector uses `System.Text.Json.JsonSerializer` for data mapping. Properties in the data model are serialized into a JSON object and mapped to Couchbase storage.
The Couchbase connector will use `System.Text.Json.JsonSerializer` to do mapping. Properties in the data model are serialized into a JSON object and stored as the document value in Couchbase.

Usage of the `JsonPropertyNameAttribute` is supported if a different storage name to the data model property name is required. It is also possible to use a custom `JsonSerializerOptions` instance with a customized property naming policy.

Use the `JsonPropertyName` attribute to map a property to a different name in Couchbase storage. Alternatively, you can configure `JsonSerializerOptions` for advanced customization.
```csharp
using Couchbase.SemanticKernel;
using Couchbase.KeyValue;
using System.Text.Json;

var jsonSerializerOptions = new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseUpper
};

var options = new CouchbaseFtsVectorStoreRecordCollectionOptions<Hotel>
var options = new CouchbaseQueryCollectionOptions
{
JsonSerializerOptions = jsonSerializerOptions
};

var collection = new CouchbaseFtsVectorStoreRecordCollection<Hotel>(scope, "hotels", options);
var collection = new CouchbaseQueryCollection<string, Hotel>(scope, "skhotelsjson", options);
```
Using the above custom `JsonSerializerOptions` which is using `CamelCase`, the following data model will be mapped to the below json.

Since a naming policy of snake case upper was chosen, here is an example of how this data type will be stored in Couchbase. Also note the use of `JsonPropertyNameAttribute` on the `Description` property to further customize the storage naming.

```csharp
using System.Text.Json.Serialization;
using Microsoft.Extensions.VectorData;

public class Hotel
{
[JsonPropertyName("hotelId")]
[VectorStoreRecordKey]
[VectorStoreKey]
public string HotelId { get; set; }

[JsonPropertyName("hotelName")]
[VectorStoreRecordData]
[VectorStoreData(IsIndexed = true)]
public string HotelName { get; set; }

[JsonPropertyName("description")]
[VectorStoreRecordData]
[JsonPropertyName("HOTEL_DESCRIPTION")]
[VectorStoreData(IsFullTextIndexed = true)]
public string Description { get; set; }

[JsonPropertyName("descriptionEmbedding")]
[VectorStoreRecordVector(Dimensions: 4, DistanceFunction.DotProductSimilarity)]
public ReadOnlyMemory<float> DescriptionEmbedding { get; set; }
[VectorStoreVector(Dimensions: 4, DistanceFunction.CosineSimilarity)]
public ReadOnlyMemory<float>? DescriptionEmbedding { get; set; }
}
```

```json
{
"hotelId": "h1",
"hotelName": "Hotel Happy",
"description": "A place where everyone can be happy",
"descriptionEmbedding": [0.9, 0.1, 0.1, 0.1]
"_id" : "h1",
"HOTEL_ID" : "h1",
"HOTEL_NAME" : "Hotel Happy",
"HOTEL_DESCRIPTION" : "A place where everyone can be happy.",
"DESCRIPTION_EMBEDDING" : [
0.9,
0.1,
0.1,
0.1
]
}
```

Expand Down