Metadata-Version: 2.4
Name: graftcode-python-logger
Version: 3.0.4
Summary: Advanced logging and tracing library for Python applications
Author-email: Graftcode <info@graftcode.com>
License: MIT
Project-URL: Homepage, https://dev.azure.com/graftcode/GRAFTCODE/_git/graftcode-logger?path=/src/python
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Requires-Dist: typing-extensions>=4.0.0; python_version < "3.8"
Requires-Dist: psutil>=5.9.0
Requires-Dist: applicationinsights
Requires-Dist: azure-monitor-opentelemetry
Requires-Dist: azure-monitor-opentelemetry-exporter
Requires-Dist: opentelemetry-instrumentation-logging
Requires-Dist: opentelemetry-instrumentation-requests
Requires-Dist: opentelemetry-instrumentation-fastapi
Requires-Dist: opencensus-ext-azure>=1.1.9
Requires-Dist: opencensus-ext-logging>=0.1.0
Provides-Extra: dev
Requires-Dist: fastapi; extra == "dev"
Requires-Dist: uvicorn; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: httpx; extra == "dev"
Requires-Dist: python-dotenv; extra == "dev"

# GraftCode Logger

A powerful, hierarchical logging system for Python applications with built-in Azure Application Insights integration and distributed tracing capabilities.

## Features

- **Hierarchical Logging**: Track parent-child operation relationships
- **Azure Integration**: Seamless export to Application Insights
- **FastAPI Integration**: Middleware and decorators for request tracking
- **Context Tracking**: Automatic correlation of logs across operations
- **Rich Metadata**: Add custom dimensions to any log
- **Flexible Configuration**: Extensive customization options

## Requirements

- Python 3.8+
- Dependencies (see `pyproject.toml`):
  - Azure Application Insights
  - OpenTelemetry instrumentation
  - OpenCensus for Azure exports
  - FastAPI (optional, for web integration)

## Installation

```bash
pip install .
# or with development/test dependencies:
pip install ".[dev]"
```

## Basic Usage

### Initialize Logger

```python
from graftcode_python_logger import GCLoggerFactory, GCLoggerOptions, LogLevel

# Create options with Application Insights connection string
options = GCLoggerOptions(
    app_insights_connection_string="YOUR_CONNECTION_STRING",
    minimum_log_level=LogLevel.INFO
)

# Initialize the factory with options
GCLoggerFactory.initialize(options)

# Get a logger instance
logger = GCLoggerFactory.get_logger(__name__)

# Basic logging
logger.info("Application started")
logger.warning("Warning message")
logger.error("Error occurred", exception=ValueError("Invalid input"))
```

### Tracking Operations

Operations create a hierarchical structure in your logs, making it easier to trace execution flow:

```python
from graftcode_python_logger import GCLoggerFactory

logger = GCLoggerFactory.get_logger(__name__)

# Track a main operation
with GCLoggerFactory.track_operation(logger, "MainProcess", {"component": "processor"}):
    logger.info("Starting main process")
    
    # Track a child operation
    with GCLoggerFactory.track_child_operation(logger, "DataValidation"):
        logger.info("Validating data")
        # Do validation work
    
    # Another child operation
    with GCLoggerFactory.track_child_operation(logger, "DataTransformation"):
        logger.info("Transforming data")
        # Do transformation work
        
    logger.info("Main process completed")
```

### Operation Tracking with Decorators

```python
from graftcode_python_logger import GCLoggerFactory

# Class method operation tracking
class DataProcessor:
    def __init__(self):
        self.logger = GCLoggerFactory.get_logger(__name__)
    
    @GCLoggerFactory.operation("ProcessData")
    def process(self, data):
        # Logger is automatically created and operation is tracked
        self.logger.info(f"Processing {len(data)} records")
        return transformed_data
```

## FastAPI Integration

GraftCode Logger provides comprehensive integration with FastAPI applications:

### Basic Setup

```python
from fastapi import FastAPI
from graftcode_python_logger import (
    GCLoggerFactory, 
    GCLoggerOptions,
    setup_request_logging
)

# Create app
app = FastAPI()

# Configure logger
options = GCLoggerOptions(
    app_insights_connection_string="YOUR_CONNECTION_STRING",
    service_name="my-api-service"
)
GCLoggerFactory.initialize(options)

# Setup request logging middleware
setup_request_logging(app)

# Get a logger for API endpoints
logger = GCLoggerFactory.get_logger("api")

# Define routes
@app.get("/")
async def root():
    logger.info("Root endpoint called")
    return {"message": "Hello World"}
```

### Tracking Endpoint Operations

```python
from fastapi import FastAPI, Depends
from graftcode_python_logger import (
    GCLoggerFactory,
    setup_request_logging,
    track_endpoint_operation
)

app = FastAPI()
setup_request_logging(app)
logger = GCLoggerFactory.get_logger("api")

@app.get("/items/{item_id}")
@track_endpoint_operation(
    logger,
    "get_item",
    {"component": "items"}
)
async def get_item(item_id: int):
    # This is automatically tracked as child operation
    # of the HTTP request parent operation
    logger.info(f"Retrieving item {item_id}")
    return {"item_id": item_id, "name": f"Item {item_id}"}
```

### Service with Nested Operations

```python
from fastapi import FastAPI, Depends
from graftcode_python_logger import GCLoggerFactory, setup_request_logging

app = FastAPI()
setup_request_logging(app)

class ItemService:
    def __init__(self):
        self.logger = GCLoggerFactory.get_logger("item_service")
        
    def get_item(self, item_id: int):
        # Create a child operation for database access
        with GCLoggerFactory.track_child_operation(
            self.logger, "DatabaseAccess", {"item_id": item_id}
        ):
            self.logger.info(f"Retrieving item {item_id} from database")
            # Database logic here
            return {"id": item_id, "name": f"Item {item_id}"}

# Dependency injection
def get_item_service():
    return ItemService()

@app.get("/items/{item_id}")
async def get_item(item_id: int, service: ItemService = Depends(get_item_service)):
    return service.get_item(item_id)
```

## Configuration Options

The `GCLoggerOptions` class provides extensive configuration:

```python
from graftcode_python_logger import GCLoggerOptions, LogLevel

options = GCLoggerOptions(
    # Basic configuration
    app_insights_connection_string="YOUR_CONNECTION_STRING",
    minimum_log_level=LogLevel.DEBUG,
    
    # Service identification
    service_name="my-service",
    service_version="1.0.0",
    
    # Export settings
    console_export_enabled=True,
    app_insights_export_enabled=True,
    app_insights_export_interval=5.0,  # seconds
    
    # Global attributes added to all logs
    custom_dimensions={
        "environment": "production",
        "region": "west-europe"
    }
)
```

### GCLoggerOptions Parameters

Here's a detailed description of all parameters available in `GCLoggerOptions`:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `app_insights_connection_string` | `str` | `None` | Azure Application Insights connection string for telemetry export. If not provided, App Insights export will be disabled. |
| `minimum_log_level` | `LogLevel` | `LogLevel.INFO` | Minimum severity level of logs to capture. Any logs below this level will be ignored. |
| `service_name` | `str` | `"unknown-service"` | Name of the service or application using the logger. Added to all logs for identification. |
| `service_version` | `str` | `"1.0.0"` | Version of the service or application. Added to all logs for identification. |
| `console_export_enabled` | `bool` | `True` | Whether to print logs to the console. |
| `console_export_format` | `str` | `"json"` | Format for console logs. Options: "json", "text". |
| `app_insights_export_enabled` | `bool` | `True` | Whether to export logs to Azure Application Insights. |
| `app_insights_export_interval` | `float` | `15.0` | Interval in seconds between batched exports to App Insights. Lower values reduce latency but increase network traffic. |
| `app_insights_connection_timeout` | `float` | `30.0` | Timeout in seconds for App Insights connections. |
| `custom_dimensions` | `Dict[str, str]` | `{}` | Global attributes added to all logs. Useful for environment, region, etc. |
| `source_context_enabled` | `bool` | `True` | Whether to automatically include source context (module, class, method) in logs. |
| `exception_stack_trace_enabled` | `bool` | `True` | Whether to include full stack traces when logging exceptions. |
| `operation_id_header_name` | `str` | `"Request-Id"` | HTTP header name for operation ID propagation in distributed systems. |
| `parent_id_header_name` | `str` | `"X-Parent-Operation-Id"` | HTTP header name for parent operation ID propagation. |
| `context_initializers` | `List[Callable]` | `[]` | List of callables that initialize operation context from various sources. |

## Log Levels

Available log levels in order of severity:

- `LogLevel.CRITICAL` - Critical failures requiring immediate attention
- `LogLevel.ERROR` - Error conditions preventing proper execution
- `LogLevel.WARNING` - Warning conditions indicating potential issues
- `LogLevel.INFO` - Informational messages about normal operation
- `LogLevel.DEBUG` - Detailed debugging information

## Azure Application Insights Integration

GraftCode Logger automatically exports logs, metrics, and traces to Azure Application Insights. This enables:

- End-to-end distributed tracing
- Correlation of logs across services
- Performance monitoring
- Exception tracking
- Custom metrics and dimensions

### Application Insights Queries (KQL)

When using Azure Application Insights, you can use these Kusto Query Language (KQL) queries to analyze your logs:

```kusto
// Find all logs for a specific operation
traces
| where operation_Id == "your-operation-id"
| order by timestamp asc

// View operation hierarchy and durations
traces
| where timestamp > ago(1h)
| where message has "Operation completed"
| extend operationName = tostring(customDimensions.OperationName)
| extend durationMs = todouble(customDimensions.DurationMs)
| project timestamp, operationName, durationMs, operation_Id, operation_ParentId
| order by timestamp asc

// Find error logs
traces
| where severityLevel >= 3  // ERROR or CRITICAL
| where timestamp > ago(24h)
| project timestamp, message, operation_Id, severityLevel, 
          tostring(customDimensions.SourceClass), 
          tostring(customDimensions.SourceMethod)

// Analyze operation performance
traces
| where timestamp > ago(24h)
| where message has "Operation completed"
| extend operationName = tostring(customDimensions.OperationName)
| extend durationMs = todouble(customDimensions.DurationMs)
| summarize 
    count=count(), 
    avgDuration=avg(durationMs), 
    p95Duration=percentile(durationMs, 95), 
    maxDuration=max(durationMs) 
  by operationName
| order by avgDuration desc

// View FastAPI endpoint performance
traces
| where timestamp > ago(24h)
| where message has "Request completed with status"
| extend path = tostring(customDimensions.path)
| extend method = tostring(customDimensions.method)
| extend statusCode = toint(extract("status ([0-9]+)", 1, message))
| extend durationSec = todouble(extract("in ([0-9]+\\.[0-9]+)s", 1, message))
| summarize 
    count=count(), 
    avgDurationMs=avg(durationSec * 1000), 
    p95DurationMs=percentile(durationSec * 1000, 95),
    countByStatus=make_set(statusCode)
  by path, method
| order by avgDurationMs desc
```

## Core API Methods

GraftCode Logger provides a set of core methods for creating and managing loggers:

### GCLoggerFactory Methods

| Method | Description | Parameters |
|--------|-------------|------------|
| `initialize(options)` | Initialize the global logger factory with options | `options`: GCLoggerOptions instance |
| `get_logger(name)` | Get a logger instance with the specified name | `name`: String name for the logger, typically `__name__` |
| `track_operation(logger, name, attributes)` | Create and track a parent operation | `logger`: Logger instance<br>`name`: Operation name<br>`attributes`: Optional dictionary of attributes |
| `track_child_operation(logger, name, attributes)` | Create and track a child operation within current parent | `logger`: Logger instance<br>`name`: Operation name<br>`attributes`: Optional dictionary of attributes |
| `operation(name, attributes)` | Decorator for tracking method execution as an operation | `name`: Operation name<br>`attributes`: Optional dictionary of attributes |
| `child_operation(name, attributes)` | Decorator for tracking method execution as a child operation | `name`: Operation name<br>`attributes`: Optional dictionary of attributes |

### Logger Methods

Each logger instance obtained from `GCLoggerFactory.get_logger()` provides these logging methods:

| Method | Description | Parameters |
|--------|-------------|------------|
| `debug(message, **kwargs)` | Log a debug message | `message`: Log message<br>`**kwargs`: Additional parameters including `exception` and custom dimensions |
| `info(message, **kwargs)` | Log an informational message | `message`: Log message<br>`**kwargs`: Additional parameters |
| `warning(message, **kwargs)` | Log a warning message | `message`: Log message<br>`**kwargs`: Additional parameters |
| `error(message, **kwargs)` | Log an error message | `message`: Log message<br>`**kwargs`: Additional parameters |
| `critical(message, **kwargs)` | Log a critical error message | `message`: Log message<br>`**kwargs`: Additional parameters |
| `exception(message, exc=None, **kwargs)` | Log an exception with stack trace | `message`: Log message<br>`exc`: Exception object (optional)<br>`**kwargs`: Additional parameters |

### OperationContext Methods

| Method | Description | Parameters |
|--------|-------------|------------|
| `current()` | Get the current operation context | None |
| `create_child(name, attributes)` | Create a child context from current context | `name`: Operation name<br>`attributes`: Optional dictionary of attributes |
| `get_operation_id()` | Get the operation ID of the current context | None |
| `get_parent_id()` | Get the parent ID of the current context | None |

## Advanced Usage

### Custom Operation Context

```python
from graftcode_python_logger import GCLoggerFactory, OperationContext

# Get the current operation context
context = OperationContext.current()

# Use it for correlation
correlation_id = context.operation_id

# Create a new child context
child_context = context.create_child("ChildOperation")

# Use the context manually
with child_context:
    logger.info("Working in child context")
```

### Manual Correlation

```python
# Get correlation identifiers
operation_id = OperationContext.current().operation_id
parent_id = OperationContext.current().parent_id

# Pass to external system
external_api_call(correlation_id=operation_id, parent_id=parent_id)

# Use these IDs in another process/service
with OperationContext(operation_id=operation_id, parent_id=parent_id):
    logger.info("Continuing operation in different context")
```

## Best Practices

1. **Use Hierarchical Operations**: Track parent-child relationships for better tracing
2. **Consistent Logger Naming**: Name loggers based on modules or components
3. **Include Context**: Add relevant context data to log messages
4. **Capture Exceptions**: Always log exceptions with the appropriate context
5. **Use Descriptive Operation Names**: Choose clear, descriptive names for operations

## Contributing

Please see the CONTRIBUTING.md file for guidelines on how to contribute to this project.

## License

This project is licensed under the terms specified in the LICENSE file. 
