Persistent Learning
A paradigm in AI systems where knowledge accumulates and compounds over time rather than being re-derived from scratch for each interaction. Contrasts with stateless approaches where each query starts fresh. Core concept underlying andrej-karpathy's llm-wiki-pattern and compounding-artifacts.
Core Concept
Traditional systems like RAG rediscover knowledge from scratch on every question - finding relevant chunks, piecing together fragments, synthesizing answers from raw sources. Nothing accumulates. Ask a subtle question requiring synthesis across five documents, and the LLM repeats the same discovery process every time.
Persistent learning systems instead build and maintain accumulated knowledge structures that compound over interactions. The synthesis work is done once and then kept current, not repeated. Each new source strengthens the existing structure rather than existing in isolation.
Implementation in LLM Wiki Pattern
The llm-wiki-pattern implements persistent learning through:
Knowledge compilation - Raw sources transformed into structured, cross-referenced wiki pages that persist between sessions
Incremental integration - New information updates existing pages rather than creating isolated summaries
Maintained synthesis - Cross-references, contradictions, and connections continuously maintained as knowledge base evolves
Compounding exploration - Good query answers filed back as wiki pages, making explorations persistent rather than ephemeral
Benefits Over Stateless Systems
- Accumulated insights: Connections and contradictions already identified
- Refined understanding: Multiple sources integrated into coherent picture
- Efficient querying: Pre-compiled knowledge vs. real-time fragment assembly
- Progressive deepening: Each interaction builds on previous work
- Preserved discoveries: Valuable insights don't disappear into chat history
Historical Context
Concept relates to vannevar-bush's memex-vision - personal knowledge stores that accumulate and cross-reference over time. Also parallels human learning where new information integrates with existing knowledge structures rather than existing in isolation.
Implementation Challenges
Requires systematic maintenance that humans find tedious but LLMs handle naturally:
- Updating cross-references across multiple pages
- Noting contradictions between old and new information
- Maintaining consistency as knowledge base grows
- Organizing and categorizing accumulated knowledge