Metadata-Version: 2.4
Name: dcnr-discovery
Version: 1.0.10
Summary: Application based on self-discovery pattern.
Project-URL: Homepage, https://github.com/gopa810/dcnr-discovery/blob/main/README.md
Project-URL: Changelog, https://github.com/gopa810/dcnr-discovery/blob/main/CHANGELOG.md
Author-email: Peter Kollath <peter.kollath@gopal.home.sk>
License: MIT
Keywords: functional-programming,interpreter,parser,python,sandbox
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# DCNR Discovery

A DCNR Discovery is package supporting discovery architecture of the application. The principle is, that project folder contains `app_discovery.yaml` files on various places in the folder structure and this package will discover all those files, performs actions defined in them and thus creates whole application from its parts and connects them.

It enables to connect application infrastructure with its broader environment in which application operates. For this purposes are used for example environment variables containing reference to configuration data, or configuration data itself.

---

## How It Works

At startup, the application factory walks the source tree starting from a given root directory, collecting every `app_discovery.yaml` file it encounters. Each file declares what its containing package contributes to the application: Flask blueprints, routes, health checks, context processors, and startup services. The factory processes all discovered files and assembles them into a single Flask application.

---

## `app_discovery.yaml` Reference

An `app_discovery.yaml` file has two optional top-level sections: `flask` and `app`.

```yaml
flask:
  # Flask-related declarations (blueprints, routes, health checks, context processors)

app:
  # Application lifecycle declarations (startup tasks, background threads)
```

Both sections are optional — a file may contain only `flask`, only `app`, or both.

---

### The `__self__` Placeholder

Any `function` value can use the special `__self__` placeholder, which resolves at runtime to the dotted Python package path of the directory containing the `app_discovery.yaml` file.

Resolution uses `sys.path` to find the longest matching root, then computes the relative dotted path.

| File location | `__self__` resolves to |
|---|---|
| `service/src/systems/adlib/app_discovery.yaml` | `systems.adlib` |
| `service/src/systems/alerts/app_discovery.yaml` | `systems.alerts` |
| `service/src/appdata/app_discovery.yaml` | `appdata` |

**Examples:**

```yaml
function: "__self__.handlers.alert_conf_create"   # → systems.alerts.handlers.alert_conf_create
```

This keeps discovery files portable — if a package is moved, only its filesystem location changes; the `__self__` references remain correct.

---

### Environment Variable Substitution

All string values support `${ENV_VAR:default}` syntax. The variable is resolved from the process environment at load time; if not set, the default (after the colon) is used. If no default is provided, the value resolves to an empty string.

```yaml
host: "${PDB_HOST:localhost}"        # → value of PDB_HOST, or "localhost"
password: "${PDB_PWD:}"             # → value of PDB_PWD, or empty string
```

Resolved values are automatically coerced to `int`, `float`, or `bool` when the entire string is a substitution placeholder.

---

## `flask` Section

### `flask.blueprints`

Declares Flask blueprints with associated template and static folders. Templates in the declared folder are scanned for `FLASK_PATH` directives for automatic route registration.

```yaml
flask:
  blueprints:
    - name: bp_main                          # Blueprint name (must be unique)
      folders:
        templates_folder: templates          # Relative to the YAML file's directory
        static_folder: static               # Relative to the YAML file's directory
        static_url_path: /monitor/static     # URL prefix for static files
```

| Property | Type | Required | Description |
|---|---|---|---|
| `name` | string | No | Blueprint name. Auto-generated if omitted. |
| `folders.templates_folder` | string | No | Path to templates directory, relative to the YAML file. Default: `templates` |
| `folders.static_folder` | string | No | Path to static files directory. Default: `static` |
| `folders.static_url_path` | string | No | URL prefix for serving static files. Default: `/monitor/static` |

#### Automatic Template Scanning (`FLASK_PATH`)

When a blueprint declares a `templates_folder`, all `*.html` files in that folder are scanned for a Jinja2 directive in the first 20 lines:

```html
{% set FLASK_PATH = "/monitor/my_page" %}
```

If found, a GET route for that path is automatically registered, rendering that template. This allows templates to self-declare their route with zero YAML or Python configuration.

---

### `flask.routes`

Declares HTTP route endpoints. Each route is registered on an auto-created blueprint scoped to the discovery file.

```yaml
flask:
  routes:
    - path: "/monitor/some_endpoint"
      methods: [GET]
      type: json_proxy
      function: "__self__.api.get_data"
```

| Property | Type | Required | Description |
|---|---|---|---|
| `path` | string | **Yes** | URL rule (Flask syntax, supports `<variable>` and `<int:variable>`) |
| `methods` | list | No | HTTP methods. Default: `[GET]` |
| `type` | string | **Yes** | One of: `page`, `json_proxy`, `handler` |
| `function` | string | Depends | Dotted path to a Python callable. Required for `json_proxy` and `handler`. Supports `__self__`. |
| `template` | string | Depends | Template filename. Required for `page`. |
| `template_args` | dict | No | Key-value arguments passed to `render_template`. Supports `lpc:` and `url:` prefixes (see below). |

#### Route Types

**`page`** — Renders a Jinja2 template. No Python handler needed.

```yaml
- path: "/monitor/dashboard"
  type: page
  template: "dashboard.html"
  template_args:
    report: "lpc:ui_get_report"     # call LPC command at render time
    job_id: "url:job_id"            # pull from URL parameter <job_id>
```

**`json_proxy`** — Calls a function and wraps the result in `jsonify()`. Ideal for data endpoints.

```yaml
- path: "/monitor/api/data"
  methods: [GET]
  type: json_proxy
  function: "__self__.api.get_data"
```

**`handler`** — Delegates to a Python function that receives URL parameters as keyword arguments and must return a Flask response (or tuple).

```yaml
- path: "/monitor/api/items/<int:item_id>"
  methods: [PUT, DELETE]
  type: handler
  function: "__self__.handlers.update_item"
```

#### Template Argument Prefixes

| Prefix | Meaning | Example |
|---|---|---|
| `lpc:` | Execute an LPC command at render time | `"lpc:ui_get_report"` |
| `url:` | Extract from URL path parameters | `"url:job_id"` |
| *(none)* | Literal string value | `"some text"` |

---

### `flask.health_checks`

Declares HTTP health-check endpoints that return a static text response.

```yaml
flask:
  health_checks:
    ready:
      path: "/ready"
      response: "I am ready!!!"
    live:
      path: "/live"
      response: "I am alive!!!"
```

| Property | Type | Required | Description |
|---|---|---|---|
| *(key)* | string | **Yes** | Endpoint name (used as Flask endpoint identifier) |
| `path` | string | **Yes** | URL path |
| `response` | string | No | Response body text. Default: `"OK"` |

---

### `flask.context_processors`

Declares variables injected into every Jinja2 template rendered by the application.

```yaml
flask:
  context_processors:
    - name: "adlib"
      type: module
      module: "systems.adlib"

    - name: "env_name"
      type: env
      value: "${APPLICATION_ENV:local}"
```

| Property | Type | Required | Description |
|---|---|---|---|
| `name` | string | **Yes** | Variable name available in templates (e.g. `{{ adlib }}`) |
| `type` | string | **Yes** | One of: `module`, `env` |
| `module` | string | Depends | Dotted import path. Required when `type: module`. |
| `value` | string | Depends | Literal or `${ENV_VAR:default}` value. Required when `type: env`. |

---

## `app` Section

### `app.startup`

Declares startup tasks and background services. Steps are executed asynchronously with dependency resolution — a step will wait for its `depends_on` target to complete before running.

```yaml
app:
  startup:
    - step: "set_postgresql_conf"
      type: "startup"
      description: "Configure PostgreSQL connection"
      function: "dcnr.postgres.set_configuration"
      kwargs:
        host: "${PDB_HOST:localhost}"
        port: "${PDB_PORT:5432}"

    - step: "schema_deploy"
      type: "startup"
      function: "__self__.deploy_schema"
      depends_on: "set_postgresql_conf"

    - step: "alert_daemon"
      type: "thread"
      function: "__self__.check_alerts_daemon"
      daemon: true
      condition: "${APPLICATION_RUN_SERVICES:False} == True"
      depends_on: "set_postgresql_conf"
```

| Property | Type | Required | Description |
|---|---|---|---|
| `step` | string | **Yes** | Unique identifier for this step (used in `depends_on` references) |
| `type` | string | **Yes** | `startup` (one-shot) or `thread` (long-running background thread) |
| `function` | string | **Yes** |  Dotted module/function path. Supports `__self__`. Function name within the module to call |
| `description` | string | No | Human-readable description (logged at startup) |
| `args` | list | No | Positional arguments passed to the function |
| `kwargs` | dict | No | Keyword arguments passed to the function. Values support `${ENV_VAR}` substitution. |
| `depends_on` | string | No | `step` name that must complete before this step runs |
| `condition` | string | No | Expression that must evaluate to `true` for the step to execute (see below) |
| `daemon` | bool | No | For `type: thread` only. Whether the thread is a daemon. Default: `true` |

#### Conditions

Conditions are strings evaluated at runtime after environment variable substitution. They support simple equality expressions:

```yaml
condition: "${APPLICATION_RUN_SERVICES:false} == true"
```

A condition prefixed with `!` is inverted:

```yaml
condition: "!${SKIP_SETUP:false}"
```

If no condition is specified, the step always executes.

#### Dependency Resolution

Steps with `depends_on` wait for the named step to complete. The runtime retries up to 12 times with 10-second intervals (total: 2 minutes). Steps whose dependencies are never satisfied are logged as unresolved.

```yaml
- step: "schema_deploy"
  depends_on: "set_postgresql_conf"    # waits for this step to finish first
```

Dependencies work across discovery files — a step in `systems/appdb/app_discovery.yaml` can depend on a step defined in `systems/emails/app_discovery.yaml`, as long as the `step` name matches.

---

## Complete Example

A minimal `app_discovery.yaml` for a subsystem package:

```yaml
flask:
  routes:
    - path: "/monitor/api/widgets"
      methods: [GET]
      type: json_proxy
      function: "__self__.api.get_widgets"

    - path: "/monitor/api/widgets"
      methods: [POST]
      type: handler
      function: "__self__.handlers.create_widget"

app:
  startup:
    - step: "widget_db_config"
      type: "startup"
      function: "__self__.config.set_connection"
      kwargs:
        host: "${WIDGET_DB_HOST:localhost}"
        port: "${WIDGET_DB_PORT:5432}"

    - step: "widget_cache_warmup"
      type: "startup"
      function: "__self__.warm_cache"
      depends_on: "widget_db_config"
```

This file declares:
- Two REST endpoints (auto-registered on a blueprint)
- A database configuration step at startup
- A cache warmup step that runs after the database is configured

No changes to `main.py`, no central route table, no central startup sequence.



# DCNR Introspection

In-memory message logging module for debugging purposes. Messages are stored per topic with a configurable size limit, keeping only the most recent entries.

## Usage

### Adding messages

```python
from dcnr.introspection import add_message

add_message("my-topic", "Something happened")
add_message("my-topic", "An error occurred", level="error", details="Stack trace...")
```

**Parameters:**
- `key` – Topic/category name for the message.
- `message` – The log message text.
- `level` – Severity level (default: `"info"`).
- `details` – Optional additional details.

### Configuring topic limits

Each topic retains up to 50 messages by default. When the limit is reached, the oldest message is removed.

```python
from dcnr.introspection.uimessages import set_topic_limit

set_topic_limit("my-topic", 100)
```

### Retrieving data

```python
from dcnr.introspection.uimessages import get_topics, get_messages_for_topic

topics = get_topics()          # List of all topics with message counts
msgs = get_messages_for_topic("my-topic")  # Messages for a specific topic
```

### Integration with other modules

The module exposes two Flask handler functions that can be registered as routes to provide message data to a UI or external consumer:

- `get_uimessages_topics()` – Returns all topics as JSON.
- `get_uimessages_messages()` – Returns messages for a given `?topic=` query parameter as JSON.

```python
from dcnr.introspection import get_uimessages_topics, get_uimessages_messages
```

## Thread Safety

All operations are protected by a threading lock, making it safe to call `add_message` from multiple threads.
