Skip to content

Commit 29b2cbe

Browse files
author
Brian Sam-Bodden
committed
feat: add hybrid search combining text and vector similarity
Implements hybrid search capability that combines full-text search (BM25) with vector similarity search using weighted scoring. This provides powerful semantic search by leveraging both keyword matching and AI embeddings. Key changes: - Add hybridSearch() method to EntityStream API with alpha parameter - Integrate RedisVL's HybridQuery for FT.AGGREGATE execution - Generate _HYBRID_SCORE metamodel field for all entities - Add comprehensive integration tests (11 tests, all passing) - Create roms-hybrid demo application with REST API - Add complete documentation (hybrid-search.adoc) Technical details: - Uses formula: hybrid_score = (1-alpha) × text_score + alpha × vector_similarity - Requires Redis 7.4+ for @__score field support in aggregations - Supports filtering with category, price ranges, and other fields - Results automatically sorted by hybrid score (highest first) The demo includes 12 sample products with mock embeddings and shows text-only, vector-only, and hybrid search comparisons.
1 parent 9c81ce4 commit 29b2cbe

24 files changed

Lines changed: 2137 additions & 2 deletions

File tree

demos/roms-hybrid/.gitignore

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
HELP.md
2+
.gradle
3+
build/
4+
!gradle/wrapper/gradle-wrapper.jar
5+
!**/src/main/**/build/
6+
!**/src/test/**/build/
7+
8+
### STS ###
9+
.apt_generated
10+
.classpath
11+
.factorypath
12+
.project
13+
.settings
14+
.springBeans
15+
.sts4-cache
16+
bin/
17+
!**/src/main/**/bin/
18+
!**/src/test/**/bin/
19+
20+
### IntelliJ IDEA ###
21+
.idea
22+
*.iws
23+
*.iml
24+
*.ipr
25+
out/
26+
!**/src/main/**/out/
27+
!**/src/test/**/out/
28+
29+
### NetBeans ###
30+
/nbproject/private/
31+
/nbbuild/
32+
/dist/
33+
/nbdist/
34+
/.nb-gradle/
35+
36+
### VS Code ###
37+
.vscode/

demos/roms-hybrid/README.md

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
# Redis OM Spring Demo - Hybrid Search
2+
3+
This demo showcases Redis OM Spring's **Hybrid Search** capability, which combines full-text search (BM25) with vector similarity search to provide powerful semantic search functionality.
4+
5+
## What is Hybrid Search?
6+
7+
Hybrid search combines two complementary search approaches:
8+
9+
1. **Full-Text Search (BM25)**: Traditional keyword-based search that matches query terms against document text
10+
2. **Vector Similarity Search**: Semantic search using embeddings that captures meaning and context
11+
12+
The results are scored using a weighted combination:
13+
14+
```
15+
hybrid_score = (1 - alpha) × text_score + alpha × vector_similarity
16+
```
17+
18+
Where:
19+
- `alpha = 0.0` → Pure text search (BM25 only)
20+
- `alpha = 0.5` → Balanced hybrid search
21+
- `alpha = 0.7` → Default, favoring semantic similarity
22+
- `alpha = 1.0` → Pure vector search (semantic only)
23+
24+
## Features
25+
26+
This demo demonstrates:
27+
28+
- **Hybrid search** combining text and vector similarity
29+
- **Filtered hybrid search** (e.g., category + price range filters)
30+
- **Comparison** between text-only, vector-only, and hybrid search
31+
- **Alpha parameter tuning** to adjust text vs. vector weighting
32+
- **Real product data** with mock embeddings
33+
34+
## Prerequisites
35+
36+
- Java 21+
37+
- Redis Stack 7.4+ running locally (for hybrid search support)
38+
- Gradle
39+
40+
## Running Redis Stack
41+
42+
Start Redis Stack using Docker:
43+
44+
```bash
45+
docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest
46+
```
47+
48+
Or use Docker Compose from the project root:
49+
50+
```bash
51+
docker compose up
52+
```
53+
54+
## Running the Demo
55+
56+
From the project root:
57+
58+
```bash
59+
./mvnw install -Dmaven.test.skip && ./mvnw spring-boot:run -pl demos/roms-hybrid
60+
```
61+
62+
Or using Gradle:
63+
64+
```bash
65+
./gradlew :demos:roms-hybrid:bootRun
66+
```
67+
68+
The application will:
69+
1. Start on port 8080
70+
2. Connect to Redis on localhost:6379
71+
3. Create search indexes
72+
4. Load sample product data
73+
74+
## API Endpoints
75+
76+
### Basic Operations
77+
78+
**Get all products:**
79+
```bash
80+
curl http://localhost:8080/api/products
81+
```
82+
83+
**Get product by ID:**
84+
```bash
85+
curl http://localhost:8080/api/products/elec-001
86+
```
87+
88+
**Get products by category:**
89+
```bash
90+
curl http://localhost:8080/api/products/category/Electronics
91+
```
92+
93+
### Search Operations
94+
95+
**1. Hybrid Search (Recommended)**
96+
97+
Combines text and semantic search for best results:
98+
99+
```bash
100+
curl -X POST http://localhost:8080/api/products/hybrid-search \
101+
-H "Content-Type: application/json" \
102+
-d '{
103+
"text": "wireless headphones for music",
104+
"embedding": [0.8, 0.2, 0.7, ...], # 384 dimensions
105+
"alpha": 0.7,
106+
"limit": 5
107+
}'
108+
```
109+
110+
**With filters:**
111+
```bash
112+
curl -X POST "http://localhost:8080/api/products/hybrid-search?text=headphones&alpha=0.7&category=Electronics&minPrice=100&maxPrice=300&limit=10" \
113+
-H "Content-Type: application/json" \
114+
-d @embedding.json
115+
```
116+
117+
**2. Text-Only Search**
118+
119+
Traditional full-text search:
120+
121+
```bash
122+
curl "http://localhost:8080/api/products/search/text?query=wireless%20headphones&limit=5"
123+
```
124+
125+
**3. Vector-Only Search**
126+
127+
Pure semantic similarity:
128+
129+
```bash
130+
curl -X POST http://localhost:8080/api/products/search/semantic \
131+
-H "Content-Type: application/json" \
132+
-d @embedding.json
133+
```
134+
135+
## Sample Data
136+
137+
The demo includes 12 products across 4 categories:
138+
139+
- **Electronics**: Headphones, TV, Laptop, Mouse
140+
- **Home & Kitchen**: Coffee Maker, Cookware, Robot Vacuum
141+
- **Sports & Outdoors**: Yoga Mat, Running Shoes, Tent
142+
- **Books**: Redis Guide, Machine Learning Basics
143+
144+
Each product has:
145+
- Full-text searchable description
146+
- 384-dimensional embedding (mock data based on category/features)
147+
- Filterable attributes (category, price, brand, stock)
148+
149+
## Example Queries
150+
151+
### Find "wireless audio devices" (semantic understanding)
152+
153+
```bash
154+
# Hybrid search will find "Wireless Bluetooth Headphones" even though
155+
# the query says "audio devices" and product says "headphones"
156+
curl -X POST "http://localhost:8080/api/products/hybrid-search?text=audio%20devices&alpha=0.7&limit=5" \
157+
-H "Content-Type: application/json" \
158+
-d @headphone_embedding.json
159+
```
160+
161+
### Find Electronics under $500
162+
163+
```bash
164+
curl -X POST "http://localhost:8080/api/products/hybrid-search?text=electronics&alpha=0.5&category=Electronics&maxPrice=500&limit=10" \
165+
-H "Content-Type: application/json" \
166+
-d @electronics_embedding.json
167+
```
168+
169+
### Compare Search Methods
170+
171+
Try the same query with different alpha values:
172+
- `alpha=0.0` - Pure text matching (may miss semantic relationships)
173+
- `alpha=0.5` - Balanced (good for most use cases)
174+
- `alpha=1.0` - Pure semantic (may miss exact keyword matches)
175+
176+
## Code Examples
177+
178+
### Using EntityStream API
179+
180+
```java
181+
@Autowired
182+
private EntityStream entityStream;
183+
184+
// Hybrid search with filters
185+
List<Product> products = entityStream.of(Product.class)
186+
.filter(Product$.CATEGORY.eq("Electronics"))
187+
.filter(Product$.PRICE.between(100.0, 500.0))
188+
.hybridSearch(
189+
"wireless headphones",
190+
Product$.DESCRIPTION,
191+
queryEmbedding,
192+
Product$.EMBEDDING,
193+
0.7f // 70% vector, 30% text
194+
)
195+
.limit(10)
196+
.collect(Collectors.toList());
197+
```
198+
199+
### Adjusting Search Behavior
200+
201+
```java
202+
// More text-focused (good for specific keyword matching)
203+
.hybridSearch(text, textField, vector, vectorField, 0.3f)
204+
205+
// Balanced approach
206+
.hybridSearch(text, textField, vector, vectorField, 0.5f)
207+
208+
// More semantic-focused (good for conceptual similarity)
209+
.hybridSearch(text, textField, vector, vectorField, 0.7f)
210+
```
211+
212+
## How It Works
213+
214+
1. **Indexing**: Redis OM Spring creates a search index with:
215+
- Full-text index on `description` field
216+
- Vector index on `embedding` field (384D, COSINE distance)
217+
- Tag/numeric indexes on filterable fields
218+
219+
2. **Query Execution**: Uses RedisVL's `HybridQuery` which:
220+
- Performs FT.AGGREGATE with text and vector scoring
221+
- Combines scores using the alpha parameter
222+
- Applies filters using Redis query syntax
223+
- Returns results sorted by hybrid score
224+
225+
3. **Result Ranking**: Documents are scored as:
226+
```
227+
hybrid_score = (1-α) × BM25_score + α × (1 - cosine_distance)
228+
```
229+
230+
## Real-World Applications
231+
232+
Hybrid search is ideal for:
233+
234+
- **E-commerce**: "Find running shoes similar to these" + "for marathon training"
235+
- **Document Search**: "Contract documents about pricing" (semantic + keyword)
236+
- **Content Discovery**: "Articles like this one" + "about machine learning"
237+
- **Job Matching**: "Software engineer" + similar to candidate profile
238+
- **Customer Support**: "How to reset password" (handles variations semantically)
239+
240+
## Technology Stack
241+
242+
- **Redis Stack 7.4+**: Provides RediSearch with hybrid query support
243+
- **Redis OM Spring**: Object mapping and search capabilities
244+
- **RedisVL for Java**: Hybrid query implementation
245+
- **Spring Boot 3.x**: Application framework
246+
- **Lombok**: Reduces boilerplate code
247+
248+
## Learn More
249+
250+
- [Redis OM Spring Documentation](https://redis.io/docs/clients/om-clients/stack-spring/)
251+
- [RediSearch Hybrid Queries](https://redis.io/docs/interact/search-and-query/advanced-concepts/hybrid-queries/)
252+
- [Vector Similarity Search](https://redis.io/docs/interact/search-and-query/advanced-concepts/vectors/)
253+
254+
## License
255+
256+
This demo is part of Redis OM Spring and follows the same license.

demos/roms-hybrid/build.gradle

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
plugins {
2+
id 'java'
3+
id 'org.springframework.boot'
4+
id 'io.spring.dependency-management'
5+
}
6+
7+
java {
8+
toolchain {
9+
languageVersion = JavaLanguageVersion.of(21)
10+
}
11+
}
12+
13+
// Don't publish this module
14+
tasks.matching { it.name.startsWith('publish') }.configureEach {
15+
enabled = false
16+
}
17+
18+
repositories {
19+
mavenLocal()
20+
mavenCentral()
21+
maven {
22+
name = 'Spring Milestones'
23+
url = 'https://repo.spring.io/milestone'
24+
}
25+
maven {
26+
name = 'Spring Snapshots'
27+
url = 'https://repo.spring.io/snapshot'
28+
}
29+
}
30+
31+
dependencies {
32+
implementation project(':redis-om-spring')
33+
34+
// Important for RedisOM annotation processing!
35+
annotationProcessor project(':redis-om-spring')
36+
testAnnotationProcessor project(':redis-om-spring')
37+
38+
// Lombok
39+
compileOnly 'org.projectlombok:lombok'
40+
annotationProcessor 'org.projectlombok:lombok'
41+
testCompileOnly 'org.projectlombok:lombok'
42+
testAnnotationProcessor 'org.projectlombok:lombok'
43+
44+
// Spring Boot starters
45+
implementation 'org.springframework.boot:spring-boot-starter-web'
46+
implementation 'org.springframework.boot:spring-boot-devtools'
47+
48+
// Test dependencies
49+
testImplementation 'org.springframework.boot:spring-boot-starter-test'
50+
testImplementation "com.redis:testcontainers-redis:${testcontainersRedisVersion}"
51+
testImplementation "org.testcontainers:junit-jupiter:1.20.4"
52+
}
53+
54+
// Use -parameters flag for Spring
55+
tasks.withType(JavaCompile).configureEach {
56+
options.compilerArgs << '-parameters'
57+
}
58+
59+
test {
60+
useJUnitPlatform()
61+
}
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
package com.redis.om.hybrid;
2+
3+
import org.springframework.boot.SpringApplication;
4+
import org.springframework.boot.autoconfigure.SpringBootApplication;
5+
6+
import com.redis.om.spring.annotations.EnableRedisEnhancedRepositories;
7+
8+
/**
9+
* Redis OM Spring Hybrid Search Demo Application.
10+
*
11+
* Demonstrates hybrid search combining full-text search (BM25) with
12+
* vector similarity search for enhanced product discovery.
13+
*/
14+
@SpringBootApplication
15+
@EnableRedisEnhancedRepositories(
16+
basePackages = "com.redis.om.hybrid.repositories"
17+
)
18+
public class RomsHybridSearchApplication {
19+
20+
public static void main(String[] args) {
21+
SpringApplication.run(RomsHybridSearchApplication.class, args);
22+
}
23+
}

0 commit comments

Comments
 (0)