Configuration Management

Confiance : high

configuration-managementenvironment-variablessystem-reliabilityproduction-deploymentconfiguration-driftcentralized-configurationruntime-validation

Systematic approach to managing application settings across different environments and deployment stages. Critical for production AI systems where configuration drift can cause silent failures or service outages.

Core Principles

Single Source of Truth

All configuration should originate from a centralized location:

# Good: Centralized configuration
class Config:
    QDRANT_COLLECTION = os.getenv("QDRANT_COLLECTION", "default_collection")
    MEILISEARCH_INDEX = os.getenv("MEILISEARCH_INDEX", "default_index")
    
    @classmethod
    def validate(cls):
        required = ["QDRANT_URL", "MEILISEARCH_URL"]
        missing = [var for var in required if not os.getenv(var)]
        if missing:
            raise ValueError(f"Missing required config: {missing}")

# Bad: Scattered configuration
def search_function():
    collection = os.getenv("QDRANT_COLLECTION", "hardcoded_fallback")  # Repeated everywhere

Environment Consistency

Documentation and code must remain synchronized:

.env.example: Template showing all required variables
Application Code: Only reads variables actually documented
Validation: Startup checks ensure all required variables are present

Common Anti-Patterns

Configuration Drift

When different parts of the system read different configuration sources:

# Ingestion reads from environment
collection_name = os.getenv("COLLECTION_NAME", "legal_docs")

# Search hard-codes the name
def search():
    return client.search("legi", query)  # Different collection!

Environment Variable Mismatch

When documentation doesn't match implementation:

# .env.example advertises
LLM_API_KEY=your_api_key
LLM_BASE_URL=your_base_url

# But code only reads
OPENAI_API_KEY=your_openai_key
ALBERT_API_KEY=your_albert_key

This causes runtime failures when following documented setup procedures.

Best Practices

Runtime Validation

Fail fast with clear error messages:

def validate_config():
    required_vars = ["QDRANT_URL", "MEILISEARCH_URL", "OPENAI_API_KEY"]
    missing = [var for var in required_vars if not os.getenv(var)]
    
    if missing:
        raise ConfigurationError(f"Missing required environment variables: {missing}")
    
    # Validate URLs are reachable
    for service_url in [Config.QDRANT_URL, Config.MEILISEARCH_URL]:
        if not is_reachable(service_url):
            raise ConfigurationError(f"Service unreachable: {service_url}")

Environment-Specific Overrides

Support different configurations across environments:

# config.py
class BaseConfig:
    QDRANT_COLLECTION = "legal_docs"
    
class DevelopmentConfig(BaseConfig):
    QDRANT_COLLECTION = "legal_docs_dev"
    
class ProductionConfig(BaseConfig):
    QDRANT_COLLECTION = "legal_docs_prod"

Configuration Testing

Test configuration consistency:

def test_env_example_completeness():
    """Ensure .env.example documents all required variables"""
    required_in_code = extract_env_vars_from_code()
    documented_in_example = parse_env_example()
    
    missing_from_docs = required_in_code - documented_in_example
    assert not missing_from_docs, f"Undocumented variables: {missing_from_docs}"

Tools and Patterns

Configuration Libraries

python-decouple: Strict separation of settings from code
pydantic-settings: Type-safe configuration with validation
hydra: Complex hierarchical configuration for ML projects

Deployment Patterns

ConfigMaps: Kubernetes-native configuration management
Secret Management: Separate handling of sensitive configuration
Feature Flags: Runtime configuration switches

Real-World Example

The assistant-rh project demonstrated configuration management failures:

Documentation Drift: .env.example advertised LLM_API_KEY but code read OPENAI_API_KEY
Hard-Coded Dependencies: Search functions ignored configurable collection names
No Validation: System started successfully but failed at runtime