ActiveRecord::Searchable

Database-native full-text search for Rails 8+ using SQLite FTS5, with zero external dependencies.

Features

• Zero Dependencies: Uses SQLite's built-in FTS5 extension
• Rails Native: Feels like ActiveRecord, works with existing scopes
• Automatic Sync: Search index updates via callbacks, always consistent
• Query Sanitization: Handles invalid FTS syntax gracefully
• Highlighting & Snippets: Built-in support for match highlighting
• BM25 Ranking: Relevance scoring with configurable field weights

Requirements

• Rails 8.0+ (for create_virtual_table support)
• Ruby 3.2+
• SQLite 3.8.0+ with FTS5 extension

Installation

Add to your Gemfile:

gem "activerecord-searchable"

Run:

bundle install

Quick Start

1. Add searchable to your model

class Article < ApplicationRecord
  include ActiveRecord::Searchable

  searchable do
    field :title, weight: 2.0  # Title matches ranked 2x higher
    field :body
  end
end

2. Generate the migration

rails generate searchable:migration Article

This creates:

class CreateArticleSearchIndex < ActiveRecord::Migration[8.0]
  def change
    create_virtual_table :article_search_index, :fts5, [
      "title",
      "body",
      "tokenize='porter'"
    ]

    Article.reindex_all
  end
end

3. Run the migration

rails db:migrate

4. Search!

# Basic search
Article.search("ruby on rails")

# With match highlighting
results = Article.search("ruby on rails", matches: true)
results.first.title_highlight  # => "Building with <mark>Ruby on Rails</mark>"
results.first.body_snippet     # => "...learn <mark>Ruby on Rails</mark>..."

# Chain with other scopes
Article.published.search("rails").where(author: current_user).limit(10)

Configuration

Field Options

searchable do
  # Basic field
  field :title
end

searchable do
  # Field with weight (for ranking)
  field :title, weight: 2.0
  # Field with custom extraction method
  field :body, via: :searchable_content
end

searchable do
  # Multiple fields
  field :title, weight: 2.0
  field :body
  field :author_name, via: :author_full_name
end

Custom Content Extraction

Use via: to extract content from a method instead of an attribute:

class Message < ApplicationRecord
  include ActiveRecord::Searchable

  has_rich_text :body

  searchable do
    field :body, via: :plain_text_body
  end

  def plain_text_body
    body.to_plain_text
  end
end

Conditional Indexing

Define a searchable? method to control which records get indexed:

class Article < ApplicationRecord
  include ActiveRecord::Searchable

  searchable do
    field :title
    field :body
  end

  def searchable?
    published? && body.present?
  end
end

Searching

Basic Search

Article.search("ruby programming")

Returns an ActiveRecord::Relation with matching records, ordered by relevance (BM25).

Search with Matches

# Smart defaults - first field highlighted, second field snippeted
results = Article.search("ruby programming", matches: true)
results.first.title_highlight  # Full content with <mark> tags
results.first.body_snippet     # ~20 word excerpt with <mark> tags

# Explicit control - specify which fields use highlight vs snippet
results = Article.search("ruby programming", matches: {
  highlight: [:title, :tags],
  snippet: [:body, :comments]
})
results.first.title_highlight  # Full title with <mark> tags
results.first.tags_highlight   # Full tags with <mark> tags
results.first.body_snippet     # Excerpt of body with <mark> tags
results.first.comments_snippet # Excerpt of comments with <mark> tags

# Only highlights, no snippets
results = Article.search("ruby", matches: { highlight: [:title] })
results.first.title_highlight  # Full title with <mark> tags
# No snippet columns generated

# No matches
results = Article.search("ruby")
# No highlight or snippet columns, just filtered/ranked records

Match columns use different suffixes based on the function:

• _highlight - Full content with <mark> tags (SQLite highlight() function)
• _snippet - Excerpt (~20 words) with <mark> tags (SQLite snippet() function)

Smart defaults (matches: true):

• First field → _highlight
• Second field → _snippet
• Other fields → no match columns

Query Sanitization

The gem automatically sanitizes queries to prevent FTS syntax errors:

Article.search('"unbalanced quote')  # Works - quotes removed
Article.search('test & invalid')     # Works - invalid chars removed
Article.search('')                   # Returns none (empty relation)

Chaining Scopes

Search integrates seamlessly with ActiveRecord:

Article.published
  .search("rails")
  .where(author: current_user)
  .order(created_at: :desc)
  .limit(20)

Index Management

Automatic Updates

The search index updates automatically via callbacks:

article = Article.create!(title: "Test", body: "Content")  # Indexed
article.update!(title: "Updated")                          # Re-indexed
article.destroy                                            # Removed from index

Manual Reindexing

Reindex a single record:

article.reindex

Reindex all records:

Article.reindex_all

The reindex_all method processes records in batches (1000 at a time by default) to prevent memory issues with large datasets. This is safe for tables with millions of records.

Custom Reindexing

For advanced needs like background jobs, progress tracking, or custom batch sizes, override reindex_all:

class Article < ApplicationRecord
  include ActiveRecord::Searchable

  searchable do
    field :title
    field :body
  end

  # Queue reindexing jobs in the background
  def self.reindex_all
    find_each do |article|
      ReindexJob.perform_later(article.id)
    end
  end
end

How It Works

FTS5 Virtual Tables

The gem creates a virtual table using SQLite's FTS5 extension:

CREATE VIRTUAL TABLE article_search_index USING fts5(
  title,
  body,
  tokenize='porter'
);

Lifecycle Callbacks

When you include ActiveRecord::Searchable, the gem adds callbacks:

• after_create_commit → Insert into FTS table
• after_update_commit → Update FTS table (or insert if missing)
• after_destroy_commit → Delete from FTS table

Search Queries

Search generates SQL like:

SELECT articles.*
FROM articles
JOIN article_search_index ON articles.id = article_search_index.rowid
WHERE article_search_index MATCH 'ruby programming'
ORDER BY bm25(article_search_index, 2.0, 1.0)

With matches:

SELECT articles.*,
  highlight(article_search_index, 0, '<mark>', '</mark>') as title_highlight,
  snippet(article_search_index, 1, '<mark>', '</mark>', '...', 20) as body_snippet
FROM articles
JOIN article_search_index ON articles.id = article_search_index.rowid
WHERE article_search_index MATCH 'ruby programming'
ORDER BY bm25(article_search_index, 2.0, 1.0)

Advanced Usage

Field Weights & Ranking

Field weights control relevance ranking. Higher weights = higher rank for matches:

searchable do
  field :title, weight: 3.0   # Title matches rank highest
  field :summary, weight: 2.0 # Summary matches rank medium
  field :body, weight: 1.0    # Body matches rank lowest
end

Uses SQLite's BM25 algorithm for scoring.

Porter Stemming

The gem uses Porter stemming by default (via tokenize='porter'). This means:

• "running" matches "run"
• "developer" matches "develop"
• "testing" matches "test"

Transaction Safety

Index updates happen in after_commit callbacks, ensuring the FTS table stays consistent with your data even if transactions roll back.

Roadmap

• PostgreSQL adapter (using tsvector and tsquery)
• MySQL adapter (using FULLTEXT indexes)
• Advanced query syntax support (FTS5 operators: AND, OR, NOT, prefix matching)
• Multi-language support

Credits

Inspired by 37signals ONCE products:

• Campfire - Team chat
• Writebook - Publishing platform

Contributing

Bug reports and pull requests welcome on GitHub.

License

MIT License. See MIT-LICENSE for details.