Metadata-Version: 2.4
Name: dynamic-config-loader
Version: 1.0.2
Summary: A lightweight, strategy-driven engine to dynamically discover and recursively deep-merge layered YAML configurations.
Author-email: Avinash Kumar <halfhiddencode@gmail.com>
Project-URL: Homepage, https://github.com/halfhiddencode/python-dynamic-config-loader
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=6.0.1
Dynamic: license-file

# Dynamic Configuration Loader

A lightweight, production-grade, strategy-driven configuration orchestration engine designed to dynamically discover, alphabetically sort, and recursively deep-merge multi-layered YAML configurations across variable filesystem paths.

```text
Copyright (C) 2026 Avinash Kumar <halfhiddencode AT gmail DOT com>

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

```

## Table of Contents
1. [Core Features](#core-features)
2. [Repository Link](#repository-link)
3. [Installation](#installation)
4. [Quick Start & Basic Usage](#quick-start--basic-usage)

    * [1. Define Architecture](#1-define-architecture)
    * [2. Execute Loader Code](#2-execute-loader-code)
    * [3. Expected Output](#3-expected-output)
5. [Advanced Production Use Cases](#advanced-production-use-cases)

    * [Use Case A: Kubernetes Pod Deployment Sidecars](#use-case-a-kubernetes-pod-deployment-sidecars)
    * [Use Case B: Multi-Cloud Region Failover Schemes](#use-case-b-multi-cloud-region-failover-schemes)
    * [Use Case C: Microservice Feature Flag Isolation](#use-case-c-microservice-feature-flag-isolation)
6. [Production Security & Logging Architecture](#production-security--logging-architecture)

    * [Secure Mode (INFO)](#secure-mode-info)
    * [Diagnostic Mode (DEBUG)](#diagnostic-mode-debug)
7. [Automated Unit Testing](#automated-unit-testing)
8. [Contributing](#contributing)

---

## Core Features

* **Layered Ingestion Architecture:** Seeds a master configuration matrix from a static baseline file and patches runtime override layers over it dynamically.
* **Dynamic Multi-Directory Scanning:** Discovers and processes external asset folders via an isolated environment variable vector (`PYTHON_ADDITIONAL_CONFIG`).
* **Deterministic Processing Loop:** Enforces explicit alphabetical sorting on all discovered override files to make overlapping configuration mutations completely predictable.
* **Recursive Deep Merges:** Protects nested dictionary tracking structures from structural wiping or collision drops during runtime updates.
* **Dual-Layered Operational Security Logs:** Structural actions log cleanly at `INFO` with sensitive values redacted. Toggling to `DEBUG` unlocks full value visibility alongside a prominent terminal security warning banner.
* **Zero Bulk Dependencies:** Relies entirely on native Python system standards and `pyyaml`.
---

## Repository Link

The official source code, distribution assets, and issue tracking boards are hosted on GitLab:
**[python-dynamic-config-loader](https://gitlab.com/halfhiddencode/python-dynamic-config-loader)**

---

## Installation

You can install the package directly from PyPI using standard package installers:

```bash
pip install dynamic-config-loader
```

Alternatively, if you are developing locally or installing from a private distribution wheels repository:

```bash
pip install .
```

---

## Quick Start & Basic Usage

### 1. Define Your Configuration Architecture

Set up your files to match this example structural multi-layer layout:

**Baseline File:** `test_configs/base_config.yaml`

```yaml
app_name: "Production Gateway"
debug_mode: false
database:
  host: "127.0.0.1"
  port: 5432
  pool_size: 20
```

**Override Patch File:** `test_configs/overrides/01_dev_patch.yaml`

```yaml
debug_mode: true
database:
  host: "internal-dev-cluster.local"
```

### 2. Execute the Loader Code

Import the engine into your main execution entry point script (e.g., `run.py`):

```python
import os
import json
import logging
from dynamic_config_loader import DynamicConfigLoader

# 1. (Optional) Configure logging to view the configuration orchestration sequence
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(name)s: %(message)s")

# 2. Point the system context env variable to your overrides directory/directories
os.environ["PYTHON_ADDITIONAL_CONFIG"] = "./test_configs/overrides"

# 3. Instantiate the loading engine with your baseline tracking path
loader = DynamicConfigLoader(default_config_path="./test_configs/base_config.yaml")

# 4. Trigger the ingestion engine pipeline loop
final_config = loader.load()

# 5. Access your compiled matrix data properties safely
print(json.dumps(final_config, indent=4))
```

### 3. Expected Output Matrix

```json
{
    "app_name": "Production Gateway",
    "debug_mode": true,
    "database": {
        "host": "internal-dev-cluster.local",
        "port": 5432,
        "pool_size": 20
    }
}
```

---

## Advanced Production Use Cases

This section illustrates how to import and consume the library across different enterprise deployment patterns.

### Use Case A: Kubernetes Pod Deployment Sidecars

In a Kubernetes infrastructure, you often want a shared immutable base image, but need to patch runtime properties dynamically via external `ConfigMaps` mounted inside the container filesystem.

```python
import os
from dynamic_config_loader import DynamicConfigLoader

def run_kubernetes_runtime_boot():
    # Base configuration is packaged inside the immutable Docker image layer
    BASE_PATH = "/app/core/configs/base_app_config.yaml"
    
    # Mount point targets for external Kubernetes ConfigMap volumes mapped dynamically
    # Example: 01_resources_patch.yaml, 02_autoscaling_tweaks.yaml
    os.environ["PYTHON_ADDITIONAL_CONFIG"] = "/etc/pod-config-mounts/overrides"
    
    loader = DynamicConfigLoader(default_config_path=BASE_PATH)
    runtime_config = loader.load()
    
    return runtime_config
```

### Use Case B: Multi-Cloud Region Failover Schemes

When deploying an application across multiple cloud environments (e.g., AWS and Azure) or geographically isolated zones, you can chain multiple override lookup paths using comma separation.

```python
import os
from dynamic_config_loader import DynamicConfigLoader

def initialize_multi_cloud_mesh():
    # Baseline defaults common across all global infrastructure regions
    common_base = "./infrastructure/global_defaults.yaml"
    
    # Cascade configuration priorities sequentially by tokenizing directories:
    # 1. Layer cloud-specific parameters (Azure vs AWS)
    # 2. Layer region-specific endpoints (East US vs West US)
    os.environ["PYTHON_ADDITIONAL_CONFIG"] = "./infrastructure/cloud/azure, ./infrastructure/regions/eastus"
    
    loader = DynamicConfigLoader(default_config_path=common_base)
    
    # The engine walks through both directories alphabetically, executing a safe recursive deep-merge
    active_cloud_matrix = loader.load()
    return active_cloud_matrix
```

### Use Case C: Microservice Feature Flag Isolation

For testing out modern canary releases or runtime flags without updating base deployment states, you can point the engine to a scratch or ephemeral folder containing experimental overrides.

```python
import os
from dynamic_config_loader import DynamicConfigLoader

class MicroserviceOrchestrator:
    def __init__(self):
        self.base_config_path = "./services/gateway/manifest.yaml"
        
    def compile_active_manifest(self, isolated_canary_dir: str = None):
        if isolated_canary_dir:
            # Dynamically point the system environmental variable vector to an ephemeral directory
            os.environ["PYTHON_ADDITIONAL_CONFIG"] = isolated_canary_dir
        else:
            # Fall back to default production path states
            os.environ["PYTHON_ADDITIONAL_CONFIG"] = "./services/gateway/production_overrides"
            
        loader = DynamicConfigLoader(default_config_path=self.base_config_path)
        return loader.load()
```

---

## Production Security & Logging Architecture

The engine features isolated logging via the `dynamic_config_loader` log domain. It respects your production secrets by modifying output profiles based on the standard Python severity thresholds:

### Secure Production Mode (`logging.INFO`)

By default, the loader protects credentials, tokens, and passwords from leaking into system scrapers or monitoring logs. It reports structural pipeline keys but keeps their underlying values redacted:

```text
2026-06-06 12:35:10 [INFO] dynamic_config_loader.loader: Ingesting baseline configuration layer from target source: ./test_configs/base_config.yaml
2026-06-06 12:35:10 [INFO] dynamic_config_loader.loader: Processing override layer discovery file: 01_dev_patch.yaml
2026-06-06 12:35:10 [INFO] dynamic_config_loader.loader: Updating configuration value at path 'database.host' [Value Redacted]
```

### Diagnostic Troubleshooting Mode (`logging.DEBUG`)

If you need full visibility to debug a complex deep-merge mutation sequence, drop the log module level down to `DEBUG`. The loader will output an explicit security notice warning exactly once, followed by the raw values:

```text
2026-06-06 12:35:15 [WARNING] dynamic_config_loader.loader: 
=======================================================================
SECURITY NOTICE: DYNAMIC CONFIG LOADER OPERATING IN VERBOSE DEBUG MODE
Execution traces will output raw parameter contents to the log stream.
Ensure this terminal session does not capture or leak sensitive secrets.
=======================================================================

2026-06-06 12:35:15 [DEBUG] dynamic_config_loader.loader: Updating configuration value at path 'database.host': '127.0.0.1' -> 'internal-dev-cluster.local'
```

To selectively control or mute the configuration pipeline logging without affecting your main application logs, configure the module logger explicitly:

```python
logging.getLogger("dynamic_config_loader").setLevel(logging.WARNING)
```

## Contributing

We welcome additions, optimization rewrites, and security updates! Please thoroughly review our guidelines inside [CONTRIBUTING.md](https://www.google.com/search?q=CONTRIBUTING.md) to understand local sandbox workspace provisioning, testing workflows, and branching protocols before filing a Pull Request.
