Neighbor Redis

Nearest neighbor search for Ruby and Redis

Supports Redis 8 vector sets and RediSearch vector indexes

Installation

First, install Redis. With Docker, use:

docker run -p 6379:6379 redis:8

Add this line to your application’s Gemfile:

gem "neighbor-redis"

And set the Redis client:

Neighbor::Redis.client = RedisClient.config.new_pool

Getting Started

Create an index

index = Neighbor::Redis::VectorSet.new("items")

Add vectors

index.add(1, [1, 1, 1])
index.add(2, [2, 2, 2])
index.add(3, [1, 1, 2])

Search for nearest neighbors to a vector

index.search([1, 1, 1], count: 5)

Search for nearest neighbors to a vector in the index

index.search_id(1, count: 5)

IDs are treated as strings by default, but can also be treated as integers

Neighbor::Redis::VectorSet.new("items", id_type: "integer")

Operations

Add or update a vector

index.add(id, vector)

Add or update multiple vectors

index.add_all(ids, vectors)

Get a vector

index.find(id)

Remove a vector

index.remove(id)

Remove multiple vectors

index.remove_all(ids)

Count vectors

index.count

Metadata

Add a vector with metadata

index.add(id, vector, metadata: {category: "A"})

Add multiple vectors with metadata

index.add_all(ids, vectors, metadata: [{category: "A"}, {category: "B"}, ...])

Get metadata for a vector

index.metadata(id)

Get metadata with search results

index.search(vector, with_metadata: true)

Set metadata

index.set_metadata(id, {category: "B"})

Remove metadata

index.remove_metadata(id)

Index Types

Vector sets

• use cosine distance
• use single-precision floats
• support exact and approximate search
• support quantization and dimensionality reduction

Vector indexes

• support L2, inner product, and cosine distance
• support single or double-precision floats
• support either exact (flat) or approximate (HNSW and SVS Vamana) search
• can support quantization and dimensionality reduction (SVS Vamana)
• require calling create before searching

Vector Sets

Create a vector set

Neighbor::Redis::VectorSet.new(name)

Specify parameters

Neighbor::Redis::VectorSet.new(name, m: 16, ef_construction: 200, ef_search: 10)

Use quantization (int8 or binary)

Neighbor::Redis::VectorSet.new(name, quantization: "int8")

Use dimensionality reduction

Neighbor::Redis::VectorSet.new(name, reduce: 2)

Perform exact search

index.search(vector, exact: true)

Vector Indexes

Create a vector index (l2, inner_product, or cosine distance)

index = Neighbor::Redis::HnswIndex.new(name, dimensions: 3, distance: "cosine")
index.create

Store vectors as double precision (instead of single precision)

Neighbor::Redis::HnswIndex.new(name, type: "float64")

Store vectors as JSON (instead of a hash/blob)

Neighbor::Redis::HnswIndex.new(name, redis_type: "json")

Index Options

HNSW

Neighbor::Redis::HnswIndex.new(name, m: 16, ef_construction: 200, ef_search: 10)

SVS Vamana - Redis 8.2+

Neighbor::Redis::SvsVamanaIndex.new(
  name,
  compression: nil,
  construction_window_size: 200,
  graph_max_degree: 32,
  search_window_size: 10,
  training_threshold: nil,
  reduce: nil
)

Flat

Neighbor::Redis::FlatIndex.new(name)

Example

You can use Neighbor Redis for online item-based recommendations with Disco. We’ll use MovieLens data for this example.

Create an index

index = Neighbor::Redis::VectorSet.new("movies")

Fit the recommender

data = Disco.load_movielens
recommender = Disco::Recommender.new(factors: 20)
recommender.fit(data)

Store the item factors

index.add_all(recommender.item_ids, recommender.item_factors)

And get similar movies

index.search_id("Star Wars (1977)").map { |v| v[:id] }

See the complete code for vector sets and vector indexes

Reference

Get index info

index.info

Check if an index exists

index.exists?

Drop an index

index.drop

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development:

git clone https://github.com/ankane/neighbor-redis.git
cd neighbor-redis
bundle install
bundle exec rake test