Metadata-Version: 2.4
Name: emp-utl
Version: 0.1.0
Summary: Utility modules for reusable infrastructure components in the Enterprise Management Program (EMP)
Home-page: https://github.com/AbdoCherry/EMP_UTL-S
Author: AbdoCherry
License: MIT
Project-URL: Source, https://github.com/AbdoCherry/EMP_UTL-S
Project-URL: Tracker, https://github.com/AbdoCherry/EMP_UTL-S/issues
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: attrs==25.4.0
Requires-Dist: boltons==25.0.0
Requires-Dist: certifi==2026.1.4
Requires-Dist: charset-normalizer==3.4.4
Requires-Dist: config-client==1.5.0
Requires-Dist: confluent-kafka==2.13.0
Requires-Dist: docutils==0.22.4
Requires-Dist: face==26.0.0
Requires-Dist: glom==25.12.0
Requires-Dist: id==1.6.1
Requires-Dist: idna==3.11
Requires-Dist: jaraco.classes==3.4.0
Requires-Dist: jaraco.context==6.1.0
Requires-Dist: jaraco.functools==4.4.0
Requires-Dist: keyring==25.7.0
Requires-Dist: markdown-it-py==4.0.0
Requires-Dist: mdurl==0.1.2
Requires-Dist: more-itertools==10.8.0
Requires-Dist: mysql-connector-python==9.6.0
Requires-Dist: nh3==0.3.3
Requires-Dist: numpy==2.4.2
Requires-Dist: packaging==26.0
Requires-Dist: pandas==3.0.1
Requires-Dist: pyarrow==23.0.1
Requires-Dist: Pygments==2.19.2
Requires-Dist: python-dateutil==2.9.0.post0
Requires-Dist: readme_renderer==44.0
Requires-Dist: requests==2.32.5
Requires-Dist: requests-toolbelt==1.0.0
Requires-Dist: rfc3986==2.0.0
Requires-Dist: rich==14.3.3
Requires-Dist: setuptools==82.0.0
Requires-Dist: six==1.17.0
Requires-Dist: twine==6.2.0
Requires-Dist: urllib3==2.6.3
Requires-Dist: wheel==0.46.3
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: project-url
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# 🧰 EMP – Utility Library (emp-utl)

<!--![PyPI](https://img.shields.io/pypi/v/emp-utl)
![Python](https://img.shields.io/pypi/pyversions/emp-utl)
![License](https://img.shields.io/github/license/AbdoCherry/EMP_UTL-S)
![Tests](https://img.shields.io/github/actions/workflow/status/AbdoCherry/EMP_UTL-S/build.yml)
![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen)-->

The library emp-utl is the official reusable utility package of the Enterprise Management Program (EMP).

It provides standardized, production-ready infrastructure components for EMP microservices and serverless functions — bundled into a single installable Python package.

The library focuses on:
- 📝 Structured logging
- ⚙️ Centralized configuration loading
- 🗄️ MySQL schema & data orchestration

## 📦 Installation

```pip3 install emp-utl```

---

## 📝 Structured Logger

The logger provides a unified, environment-aware logging abstraction for EMP services.

It automatically adapts its behavior based on the **ENVIRONMENT** variable:
- dev → JSON file logging (local)
- int / stg / prd → Kafka-based centralized logging

**Key Characteristics:**
- 📦 Structured JSON format
- 🌍 Environment-aware behavior
- 📡 Kafka publishing in higher environments
- 🧾 Logback-compatible structure (ELK-ready)

**Usage:**

```python
from emp_utl import setup_logger

logger = setup_logger(service="Customer", module="Serverless")
logger.info("Service initialized successfully")
```

Logs are automatically formatted and routed according to the deployment environment.

---

## ⚙️ Configuration Loader

The configuration loader enables secure and dynamic configuration resolution across environments.

Resolution Strategy
1. **Primary Source – Spring Cloud Config Server (CNF-S)**
   1. Authenticated access via EMP_CONFIG_USERNAME and EMP_ENCRYPT_KEY
   2. Nested configuration key resolution supported

2. **Fallback – Local OS Environment**
   1. Automatically triggered if CNF-S is unavailable 
   2. Uses os.getenv for required keys
   
This ensures service startup even if the configuration server is unreachable.

**Usage:**

```python
from emp_utl import load_cnf

keys = ["DB_HOST", "DB_PORT", "DB_USER", "DB_PASSWORD"]

env_vars = load_cnf(
    service = "utility-service",
    keys = keys,
    logger = logger
)
```

Returned values are provided as a dictionary.

---

## 🗄️ MySQL Loader

The MySQL loader orchestrates schema setup and DataFrame-based data loading.

It performs:
- MySQL connection initialization
- SQL script execution (./sql directory)
- Foreign key handling
- Bulk insertion of pandas DataFrames
- Post-insert null normalization

Designed for:
- Local development
- Serverless execution
- Automated schema bootstrapping

**Usage:**

```python
from emp_utl import load_mysql

load_mysql(
    env_vars = env_vars,
    schema = env_vars["EMP_SCHEMA_UTILITY"],
    df_dict = df_dict,
    logger = logger
)
```

> 📌 df_dict must map table names to pandas DataFrames.
SQL schema files must exist in ./sql.

---

## ✅ Quality Standards
- Unit test coverage ≥ 90%
- Explicit public API
- Environment-safe behavior
- Production-ready structured logging
- Designed for CI/CD publishing via GitHub Actions

## 🎯 Purpose

emp-utl standardizes infrastructure behavior across the EMP microservice ecosystem.

It ensures:
- Consistency
- Observability
- Secure configuration handling
- Reusable database orchestration



# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.0] - 2026-03-06

#### Structured Logging
- Environment-aware structured logging via `setup_logger`
- JSON-formatted logs compatible with ELK Stack ingestion
- Local file logging for `dev` environment
- Kafka-based centralized logging for `int`, `stg`, and `prd` environments
- Custom Kafka logging handler with delivery reporting

#### Configuration Loader
- Dynamic configuration loading via `load_cnf`
- Primary configuration source: Spring Cloud Config Server (CNF-S)
- Secure authentication using `EMP_CONFIG_USERNAME` and `EMP_ENCRYPT_KEY`
- Automatic fallback to local OS environment variables if CNF-S is unavailable
- Nested configuration key resolution support

#### MySQL Utilities
- MySQL orchestration via `load_mysql`
- Automatic schema setup through SQL script execution
- Support for custom SQL `DELIMITER` parsing
- Foreign key management utilities (`disable_fk_check`, `enable_fk_check`)
- Bulk DataFrame insertion support
- Automatic placeholder normalization for NULL values
- Column-type-aware NULL restoration logic

#### Testing & Quality
- Complete unit test coverage (100%)
- Comprehensive error handling and logging
- CI-ready architecture for GitHub Actions publishing
