qmd Search Engine

Local search engine for markdown files designed specifically for llm-wiki-pattern implementations. Provides hybrid BM25/vector search with LLM re-ranking, all running on-device. Essential tool for scaling wiki navigation beyond simple index-based browsing.

Core Capabilities

Hybrid Search Architecture:

• BM25 for keyword-based relevance scoring
• Vector embeddings for semantic similarity
• LLM re-ranking for improved result quality
• Combines best of lexical and semantic search

Local Operation:

• Runs entirely on-device, no external services required
• Privacy-preserving - content never leaves local machine
• Fast response times for interactive exploration
• No API costs or rate limits

Integration Options

CLI Interface:

• LLM can shell out to qmd for search operations
• Scriptable for automated wiki maintenance tasks
• Batch processing capabilities for large-scale analysis

MCP Server:

• Native tool integration for LLM agents
• Structured query and response format
• Seamless integration with agent workflows

Use Cases in LLM Wikis

Query Workflow Enhancement:

• Replace simple index.md scanning with sophisticated search
• Find relevant pages across large wiki collections
• Support complex multi-term and semantic queries

Lint Workflow Support:

• Find orphan pages and missing cross-references
• Identify contradictions between related pages
• Discover content gaps and redundancies

Exploration and Discovery:

• Surface unexpected connections between concepts
• Find related content across different categories
• Support research and synthesis workflows

Scaling Considerations

When to Implement:

• Essential when wiki grows beyond ~100 pages
• Index.md becomes unwieldy for navigation
• Complex queries require semantic understanding

Performance Characteristics:

• Handles thousands of markdown files efficiently
• Real-time search for interactive exploration
• Batch analysis for maintenance tasks

Alternative Approaches

While qmd is recommended by andrej-karpathy, the llm-wiki-pattern is tool-agnostic. Simple alternatives include:

Naive Search Scripts:

• Basic grep-based search over markdown files
• LLM can help implement custom search logic
• Sufficient for smaller wikis or specific domains

Existing Tools:

• Standard text search utilities (ripgrep, ag)
• Markdown-specific search tools
• Platform-native search (macOS Spotlight, Windows Search)

Implementation Strategy

Incremental Adoption:

• Start with index.md for small wikis
• Add basic search when navigation becomes cumbersome
• Implement qmd or similar when semantic search becomes valuable

LLM Integration:

• Configure search tool in schema layer
• Train LLM on search syntax and capabilities
• Integrate search into standard workflows

See also

• llm-wiki-pattern - Overall framework requiring search at scale
• obsidian-integration - Primary interface that benefits from enhanced search
• query-workflow - Process enhanced by sophisticated search capabilities
• lint-workflow - Maintenance tasks supported by search tools