Skip to content

Commit a59f6e7

Browse files
committed
feat(search): add comprehensive Redis Query Engine support
Implement modular Search architecture with complete feature parity to redis-py: Core Components: - Schema and field definitions (TextField, NumericField, TagField, GeoField, VectorField, GeoShapeField) - Query builder with fluent API and advanced query syntax - Index management and operations (create, drop, alter, info) - Aggregation framework with reducers and grouping - Hybrid search combining text and vector queries - Result parsing and formatting Search Features: - Full-text search with stemming, phonetic matching, and stop words - Vector similarity search supporting FLAT, HNSW, and SVS-VAMANA algorithms - Geospatial search with radius and polygon queries - Numeric and tag filtering - Aggregations with grouping, sorting, applying, and reducing - Hybrid search with score combination (RRF, linear) - Auto-complete/suggestions with fuzzy matching - Spell checking and synonym management - Query profiling and explain Field Types: - TextField: full-text search with weights, phonetic matching, withsuffixtrie - NumericField: range queries with sortable option - TagField: exact-match filtering with case sensitivity and separators - GeoField: geospatial queries with radius search - VectorField: vector similarity with multiple algorithms and distance metrics - GeoShapeField: polygon-based geospatial queries Advanced Features: - IndexDefinition for fine-grained index control - Query parameters for dynamic queries - Multiple dialect support (1, 2, 3, 4) - Cursor-based pagination for large result sets - Score explanations and custom scorers - Highlighting and summarization Bug Fixes: - Fix PARAMS handling to preserve binary vector data (don't call .to_s on values) - Add DIALECT 2 requirement for KNN queries in Redis 8 - Add convenience API for SORTBY (:sort_by + :asc parameters) - Fix field option ordering for proper Redis command syntax Testing: - Add comprehensive test suite with 44 tests across 8 test files - Test coverage for all search features and edge cases - Hybrid search tests with vector + text queries - Aggregation tests with complex pipelines - Vector similarity tests with multiple algorithms Documentation: - Add 6 example files demonstrating all features: - search_quickstart.rb: basic search operations - search_ft_queries.rb: advanced query syntax - search_aggregations.rb: aggregation pipelines - search_geo.rb: geospatial queries - search_range.rb: numeric range queries - search_vector_similarity.rb: vector search - search_with_hashes.rb: search with Redis hashes Results: 44 tests, 215 assertions, 0 failures, 0 errors, 0 skips
1 parent 028059d commit a59f6e7

28 files changed

+5017
-0
lines changed

.claude/settings.local.json

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"permissions": {
3+
"allow": [
4+
"Bash(gh api:*)",
5+
"Bash(gh run list:*)",
6+
"Bash(git fetch:*)"
7+
]
8+
}
9+
}

examples/search_aggregations.rb

Lines changed: 313 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,313 @@
1+
# rubocop:disable Layout/LineLength
2+
# !/usr/bin/env ruby
3+
# frozen_string_literal: true
4+
5+
# Redis Search Aggregations Example
6+
#
7+
# This example demonstrates Redis Search aggregation and analytics capabilities including:
8+
# - FT.AGGREGATE with various reducers (COUNT, SUM, AVG, MIN, MAX)
9+
# - GROUPBY operations with multiple reducers
10+
# - APPLY transformations and expressions
11+
# - SORTBY for ordering results
12+
# - LIMIT for pagination
13+
# - FILTER for conditional filtering
14+
# - Complex aggregation pipelines
15+
#
16+
# Run this example with: ruby examples/search_aggregations.rb
17+
18+
require 'redis'
19+
require 'json'
20+
21+
# Connect to Redis
22+
redis = Redis.new(host: 'localhost', port: 6400)
23+
24+
# Clean up any existing index
25+
begin
26+
redis.ft_dropindex('idx:bicycle', delete_documents: true)
27+
rescue Redis::CommandError
28+
# Index doesn't exist, continue
29+
end
30+
31+
# Create index
32+
schema = Redis::Commands::Search::Schema.build do
33+
text_field '$.brand', as: 'brand'
34+
text_field '$.model', as: 'model'
35+
text_field '$.description', as: 'description'
36+
numeric_field '$.price', as: 'price'
37+
tag_field '$.condition', as: 'condition'
38+
end
39+
40+
definition = Redis::Commands::Search::IndexDefinition.new(
41+
prefix: ['bicycle:'],
42+
index_type: Redis::Commands::Search::IndexType::JSON
43+
)
44+
45+
redis.create_index('idx:bicycle', schema, definition: definition)
46+
47+
# Bicycle data
48+
bicycle_data = [
49+
{
50+
brand: 'Velorim',
51+
model: 'Jigger',
52+
price: 270,
53+
description: 'Small and powerful, the Jigger is the best ride for the smallest of tikes!',
54+
condition: 'new'
55+
},
56+
{
57+
brand: 'Bicyk',
58+
model: 'Hillcraft',
59+
price: 1200,
60+
description: 'Kids want to ride with as little weight as possible.',
61+
condition: 'used'
62+
},
63+
{
64+
brand: 'Nord',
65+
model: 'Chook air 5',
66+
price: 815,
67+
description: 'The Chook Air 5 gives kids aged six years and older a durable bike.',
68+
condition: 'used'
69+
},
70+
{
71+
brand: 'Eva',
72+
model: 'Eva 291',
73+
price: 3400,
74+
description: 'The sister company to Nord, Eva launched in 2005.',
75+
condition: 'used'
76+
},
77+
{
78+
brand: 'Noka Bikes',
79+
model: 'Kahuna',
80+
price: 3200,
81+
description: 'Whether you want to try your hand at XC racing.',
82+
condition: 'used'
83+
},
84+
{
85+
brand: 'Breakout',
86+
model: 'XBN 2.1 Alloy',
87+
price: 810,
88+
description: 'The XBN 2.1 Alloy is our entry-level road bike.',
89+
condition: 'new'
90+
},
91+
{
92+
brand: 'ScramBikes',
93+
model: 'WattBike',
94+
price: 2300,
95+
description: 'The WattBike is the best e-bike for people who still feel young at heart.',
96+
condition: 'new'
97+
},
98+
{
99+
brand: 'Peaknetic',
100+
model: 'Secto',
101+
price: 430,
102+
description: 'If you struggle with stiff fingers or a kinked neck.',
103+
condition: 'new'
104+
},
105+
{
106+
brand: 'nHill',
107+
model: 'Summit',
108+
price: 1200,
109+
description: 'This budget mountain bike from nHill performs well.',
110+
condition: 'new'
111+
},
112+
{
113+
brand: 'BikeShind',
114+
model: 'ThrillCycle',
115+
price: 815,
116+
description: 'An artsy, retro-inspired bicycle.',
117+
condition: 'refurbished'
118+
}
119+
]
120+
121+
# Add bicycle documents
122+
bicycle_data.each_with_index do |bike, i|
123+
redis.json_set("bicycle:#{i}", '$', bike)
124+
end
125+
126+
puts "Added #{bicycle_data.length} bicycle documents\n\n"
127+
128+
# STEP_START agg1
129+
# Example 1: APPLY transformation - Calculate discounted price for new bicycles
130+
puts "Example 1: APPLY transformation - Calculate discounted price for new bicycles"
131+
req1 = Redis::Commands::Search::AggregateRequest.new('@condition:{new}')
132+
.load('__key', 'price')
133+
.apply(discounted: '@price - (@price * 0.1)')
134+
135+
res1 = redis.ft_aggregate('idx:bicycle', req1)
136+
puts "Total results: #{res1[0]}"
137+
puts "Results:"
138+
(1...res1.length).each do |i|
139+
row = res1[i]
140+
puts " Key: #{row[row.index('__key') + 1]}, " \
141+
"Price: #{row[row.index('price') + 1]}, " \
142+
"Discounted: #{row[row.index('discounted') + 1]}"
143+
end
144+
puts
145+
# STEP_END
146+
147+
# STEP_START agg2
148+
# Example 2: GROUPBY with SUM reducer - Count affordable bikes by condition
149+
puts "Example 2: GROUPBY with SUM reducer - Count affordable bikes by condition"
150+
req2 = Redis::Commands::Search::AggregateRequest.new('*')
151+
.load('price')
152+
.apply(price_category: '@price<1000')
153+
.group_by('@condition', Redis::Commands::Search::Reducers.sum('@price_category').as('num_affordable'))
154+
155+
res2 = redis.ft_aggregate('idx:bicycle', req2)
156+
puts "Total results: #{res2[0]}"
157+
puts "Results:"
158+
(1...res2.length).each do |i|
159+
row = res2[i]
160+
puts " Condition: #{row[row.index('condition') + 1]}, " \
161+
"Num Affordable: #{row[row.index('num_affordable') + 1]}"
162+
end
163+
puts
164+
# STEP_END
165+
166+
# STEP_START agg3
167+
# Example 3: GROUPBY with COUNT reducer - Count total bicycles
168+
puts "Example 3: GROUPBY with COUNT reducer - Count total bicycles"
169+
req3 = Redis::Commands::Search::AggregateRequest.new('*')
170+
.apply(type: "'bicycle'")
171+
.group_by('@type', Redis::Commands::Search::Reducers.count.as('num_total'))
172+
173+
res3 = redis.ft_aggregate('idx:bicycle', req3)
174+
puts "Total results: #{res3[0]}"
175+
puts "Results:"
176+
(1...res3.length).each do |i|
177+
row = res3[i]
178+
puts " Type: #{row[row.index('type') + 1]}, " \
179+
"Total: #{row[row.index('num_total') + 1]}"
180+
end
181+
puts
182+
# STEP_END
183+
184+
# STEP_START agg4
185+
# Example 4: GROUPBY with TOLIST reducer - List bicycles by condition
186+
puts "Example 4: GROUPBY with TOLIST reducer - List bicycles by condition"
187+
req4 = Redis::Commands::Search::AggregateRequest.new('*')
188+
.load('__key')
189+
.group_by('@condition', Redis::Commands::Search::Reducers.tolist('__key').as('bicycles'))
190+
191+
res4 = redis.ft_aggregate('idx:bicycle', req4)
192+
puts "Total results: #{res4[0]}"
193+
puts "Results:"
194+
(1...res4.length).each do |i|
195+
row = res4[i]
196+
condition_idx = row.index('condition')
197+
bicycles_idx = row.index('bicycles')
198+
puts " Condition: #{row[condition_idx + 1]}"
199+
puts " Bicycles: #{row[bicycles_idx + 1]}"
200+
end
201+
puts
202+
# STEP_END
203+
204+
# STEP_START agg5
205+
# Example 5: GROUPBY with multiple reducers - Statistics by condition
206+
puts "Example 5: GROUPBY with multiple reducers - Statistics by condition"
207+
req5 = Redis::Commands::Search::AggregateRequest.new('*')
208+
.load('price')
209+
.group_by('@condition',
210+
Redis::Commands::Search::Reducers.count.as('count'),
211+
Redis::Commands::Search::Reducers.sum('@price').as('total_price'),
212+
Redis::Commands::Search::Reducers.avg('@price').as('avg_price'),
213+
Redis::Commands::Search::Reducers.min('@price').as('min_price'),
214+
Redis::Commands::Search::Reducers.max('@price').as('max_price'))
215+
216+
res5 = redis.ft_aggregate('idx:bicycle', req5)
217+
puts "Total results: #{res5[0]}"
218+
puts "Results:"
219+
(1...res5.length).each do |i|
220+
row = res5[i]
221+
puts " Condition: #{row[row.index('condition') + 1]}"
222+
puts " Count: #{row[row.index('count') + 1]}"
223+
puts " Total Price: #{row[row.index('total_price') + 1]}"
224+
puts " Avg Price: #{row[row.index('avg_price') + 1]}"
225+
puts " Min Price: #{row[row.index('min_price') + 1]}"
226+
puts " Max Price: #{row[row.index('max_price') + 1]}"
227+
end
228+
puts
229+
# STEP_END
230+
231+
# STEP_START agg6
232+
# Example 6: SORTBY - Sort results by price descending
233+
puts "Example 6: SORTBY - Sort results by price descending"
234+
req6 = Redis::Commands::Search::AggregateRequest.new('*')
235+
.load('__key', 'price', 'brand')
236+
.sort_by(Redis::Commands::Search::Desc.new('@price'))
237+
238+
res6 = redis.ft_aggregate('idx:bicycle', req6)
239+
puts "Total results: #{res6[0]}"
240+
puts "Results (top 5):"
241+
(1...[res6.length, 6].min).each do |i|
242+
row = res6[i]
243+
puts " Brand: #{row[row.index('brand') + 1]}, " \
244+
"Price: #{row[row.index('price') + 1]}"
245+
end
246+
puts
247+
# STEP_END
248+
249+
# STEP_START agg7
250+
# Example 7: LIMIT - Paginate results
251+
puts "Example 7: LIMIT - Paginate results"
252+
req7 = Redis::Commands::Search::AggregateRequest.new('*')
253+
.load('__key', 'price', 'brand')
254+
.sort_by(Redis::Commands::Search::Asc.new('@price'))
255+
.limit(2, 3) # Skip 2, return 3
256+
257+
res7 = redis.ft_aggregate('idx:bicycle', req7)
258+
puts "Total results: #{res7[0]}"
259+
puts "Results (offset 2, limit 3):"
260+
(1...res7.length).each do |i|
261+
row = res7[i]
262+
puts " Brand: #{row[row.index('brand') + 1]}, " \
263+
"Price: #{row[row.index('price') + 1]}"
264+
end
265+
puts
266+
# STEP_END
267+
268+
# STEP_START agg8
269+
# Example 8: FILTER - Filter aggregated results
270+
puts "Example 8: FILTER - Filter aggregated results"
271+
req8 = Redis::Commands::Search::AggregateRequest.new('*')
272+
.load('price')
273+
.group_by('@condition',
274+
Redis::Commands::Search::Reducers.avg('@price').as('avg_price'))
275+
.filter('@avg_price > 1000')
276+
277+
res8 = redis.ft_aggregate('idx:bicycle', req8)
278+
puts "Total results: #{res8[0]}"
279+
puts "Results (conditions with avg price > 1000):"
280+
(1...res8.length).each do |i|
281+
row = res8[i]
282+
puts " Condition: #{row[row.index('condition') + 1]}, " \
283+
"Avg Price: #{row[row.index('avg_price') + 1]}"
284+
end
285+
puts
286+
# STEP_END
287+
288+
# STEP_START agg9
289+
# Example 9: Complex aggregation pipeline
290+
puts "Example 9: Complex aggregation pipeline - Price analysis by condition"
291+
req9 = Redis::Commands::Search::AggregateRequest.new('*')
292+
.load('price', 'brand')
293+
.apply(price_range: '@price >= 1000 ? "high" : "low"')
294+
.group_by(['@condition', '@price_range'],
295+
Redis::Commands::Search::Reducers.count.as('count'),
296+
Redis::Commands::Search::Reducers.avg('@price').as('avg_price'))
297+
.sort_by(Redis::Commands::Search::Desc.new('@count'))
298+
299+
res9 = redis.ft_aggregate('idx:bicycle', req9)
300+
puts "Total results: #{res9[0]}"
301+
puts "Results:"
302+
(1...res9.length).each do |i|
303+
row = res9[i]
304+
puts " Condition: #{row[row.index('condition') + 1]}, " \
305+
"Price Range: #{row[row.index('price_range') + 1]}, " \
306+
"Count: #{row[row.index('count') + 1]}, " \
307+
"Avg Price: #{row[row.index('avg_price') + 1]}"
308+
end
309+
puts
310+
# STEP_END
311+
312+
puts "All aggregation examples completed successfully!"
313+
# rubocop:enable Layout/LineLength

0 commit comments

Comments
 (0)