~/wiki
Récent+ Ajouter

Three Layer Architecture

Mis à jour le 2025-01-04Confiance : high
three-layer-architecturellm-wiki-patternseparation-of-concernsraw-sourceswiki-layerschema-layerimmutable-sourcesllm-ownershipconfiguration-managementandrej-karpathy

The foundational architectural pattern underlying andrej-karpathy's llm-wiki-pattern that separates concerns across three distinct layers: raw sources, wiki content, and schema configuration. This separation enables clear ownership models, version control, and collaborative development between humans and LLMs.

Layer Definitions

Raw Sources Layer:

  • Curated collection of source documents (articles, papers, images, data files)
  • Immutable - LLM reads from them but never modifies them
  • Serves as the authoritative source of truth
  • Examples: raw/articles/, raw/papers/, raw/transcripts/

Wiki Layer:

  • Directory of LLM-generated markdown files
  • Contains summaries, entity pages, concept pages, comparisons, syntheses
  • LLM owns this layer entirely - creates, updates, maintains cross-references
  • Clear division: humans read it, LLMs write it
  • Examples: wiki/concepts/, wiki/entities/, wiki/sources/

Schema Layer:

  • Configuration document (CLAUDE.md, AGENTS.md, SCHEMA.md, etc.)
  • Defines wiki structure, conventions, and operational workflows
  • Co-evolved between human and LLM based on actual usage patterns
  • Makes the LLM a disciplined wiki maintainer rather than generic chatbot
  • Contains ingest workflows, query patterns, maintenance procedures

Ownership Model

The architecture establishes clear ownership and modification rights:

Human Responsibilities:

  • Curate and add sources to raw layer
  • Modify schema based on evolving needs
  • Browse and explore wiki content
  • Direct analysis and ask questions

LLM Responsibilities:

  • Generate and maintain all wiki content
  • Follow schema conventions and workflows
  • Update multiple pages per source integration
  • Maintain cross-references and consistency

Benefits of Separation

Version Control: Each layer can be versioned independently, allowing rollback of wiki changes without affecting sources or schema evolution.

Clear Interfaces: Well-defined boundaries prevent scope creep and maintain system integrity. Sources remain pristine, wiki stays automatically maintained.

Collaborative Development: Humans and LLMs can work simultaneously without conflicts - humans curate sources and refine schema while LLMs maintain wiki content.

Debugging and Maintenance: Issues can be isolated to specific layers, making troubleshooting more systematic.

Implementation Patterns

Directory Structure:

knowledge-base/
├── raw/              # Sources layer (human-curated)
│   ├── articles/
│   ├── papers/
│   └── transcripts/
├── wiki/             # Content layer (LLM-maintained)
│   ├── concepts/
│   ├── entities/
│   └── sources/
└── SCHEMA.md         # Configuration layer (co-evolved)

Navigation Files:

  • index.md - Content catalog (wiki layer)
  • log.md - Operation history (wiki layer)
  • Both maintained by LLM according to schema specifications

Schema Evolution

The schema layer is unique in being co-evolved rather than owned by either human or LLM. As usage patterns emerge and domain needs evolve, both parties contribute to schema refinement:

  • Human adds new domains or changes priorities
  • LLM suggests workflow improvements based on operational experience
  • Both parties iterate on conventions and standards

This evolutionary approach ensures the system adapts to actual usage rather than theoretical ideals.

Scaling Considerations

The architecture scales naturally:

  • Raw sources can grow indefinitely without affecting other layers
  • Wiki layer scales through improved navigation and search tools
  • Schema layer remains stable once mature, requiring only periodic refinement

See also