Curated Reference Datasets
Small, hand-picked collections of high-quality document examples with verified extractions used to bootstrap new document processing categories. Key insight from alan-health: a small, carefully curated reference dataset enables rapid deployment of new document types without requiring large validated pools.
Core Principle
Quality over Quantity: A small set of hand-picked reference documents with verified extractions gets you "surprisingly far" when bootstrapping new categories. Focus on representative, clean examples rather than large volumes of potentially inconsistent data.
Implementation Approach
Manual Curation: Human experts select representative examples covering typical document variations within a category.
Verified Ground Truth: Each reference document has manually validated, high-quality extraction serving as ground truth for evaluation and few-shot selection.
Category Bootstrapping: New document types can be launched with minimal initial data, then improved iteratively as more validated examples accumulate.
Production Benefits
- Enables rapid deployment of new document processing categories
- Provides reliable foundation for few-shot-learning without large data requirements
- Supports evaluation-framework-design with trusted ground truth references
- Scales naturally as system processes more documents over time