Metadata-Version: 2.4
Name: etiket-api
Version: 0.3.0b1
Summary: REST and Python API for the eTiKeT platform
Author: QHarbor team
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://qharbor.nl
Project-URL: Documentation, https://docs.qharbor.nl
Keywords: etiket,sync,api,data-synchronization,fastapi
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: etiket_sync_agent>=0.3.0b1
Requires-Dist: etiket_client>=0.3.0b1
Requires-Dist: fastapi<1.0,>=0.100
Requires-Dist: pydantic<3.0,>=2.0
Requires-Dist: uvicorn<1.0,>=0.20
Requires-Dist: etiket-service-manager>=0.3.0b1
Provides-Extra: test
Requires-Dist: pytest>=8.0; extra == "test"
Requires-Dist: pytest-asyncio>=0.23; extra == "test"
Requires-Dist: truststore; extra == "test"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: truststore; extra == "dev"
Dynamic: license-file

# eTiKeT API

REST and Python API for the eTiKeT platform. This package provides the local API service that allows external tools (like the [eTiKeT SDK](../etiket_sdk)) to interact with the sync agent and manage local authentication.

## Overview

The eTiKeT API runs as a local service. It provides:

- **REST API**: HTTP endpoints for authentication, sync management, and more
- **Python API**: Direct Python functions for internal use

> **Note:** Most users should use the [eTiKeT SDK](../etiket_sdk) instead of calling this API directly. The SDK provides a simpler, higher-level interface.

## Installation

```bash
pip install etiket-api
```

## Running the Service

Normally, the sync API runs as a system service in the background (managed by the eTiKeT installer). However, you can run it manually for development or debugging:

```bash
python3.11 -m etiket_api
```

By default, the service runs on `http://localhost:10410`.
The interactive API documentation is available at `http://localhost:10410/docs` (Swagger UI).

---

## REST API Overview

The REST API is available at `http://localhost:10410/api/v1/`. Interactive API documentation is available at `http://localhost:10410/docs` (Swagger UI).

### Local API Tokens

Manage API tokens stored on this machine. Tokens allow background sync to authenticate with the server without an active login session.

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/auth/local/api-tokens` | Create a new API token for the current user |
| `GET` | `/auth/local/api-tokens` | List all locally stored API tokens |
| `DELETE` | `/auth/local/api-tokens/{uid}` | Revoke a token on the server and delete it locally |
| `GET` | `/auth/local/api-tokens/{uid}/check` | Check if a token is still valid |

### Sync Agent

Manage the background sync agent service.

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/sync/agent/status` | Get current sync agent status |
| `GET` | `/sync/agent/errors` | Get sync agent errors (paginated, optional `viewed` filter) |
| `PATCH` | `/sync/agent/errors/{error_id}` | Update a sync-agent error record (currently only the `viewed` flag) |
| `POST` | `/sync/agent/errors/mark-all-viewed` | Mark all sync-agent errors as viewed. Returns the number flipped |
| `POST` | `/sync/agent/start` | Start the sync agent |
| `POST` | `/sync/agent/stop` | Stop the sync agent |
| `POST` | `/sync/agent/report-error` | Send an error report to qHarbor |
| `GET` | `/sync/agent/error-report` | Get an error report for manual sending |

### Sync Sources

Manage sync sources — configurations that define where data comes from.

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/sync/sources` | List all sync sources |
| `POST` | `/sync/sources` | Create a new sync source |
| `GET` | `/sync/sources/{id}` | Get a sync source by ID |
| `GET` | `/sync/sources/name/{name}` | Get a sync source by name |
| `PATCH` | `/sync/sources/{id}` | Update a sync source |
| `DELETE` | `/sync/sources/{id}` | Delete a sync source |
| `POST` | `/sync/sources/{id}/start` | Start a sync source |
| `POST` | `/sync/sources/{id}/stop` | Stop a sync source |
| `POST` | `/sync/sources/{id}/take-ownership` | Claim ownership of a sync source for the current user (overwrites any existing owner) |
| `POST` | `/sync/sources/{id}/release-ownership` | Release ownership of a sync source. Idempotent — any logged-in user may call (no-op when already unowned) |
| `POST` | `/sync/sources/{id}/reset-sync-items` | Reset sync items (retry failed or all) |
| `GET` | `/sync/sources/{id}/errors` | Get sync source errors (paginated, optional `viewed` filter) |
| `PATCH` | `/sync/sources/{id}/errors/{error_id}` | Update a sync-source error record (currently only the `viewed` flag) |
| `POST` | `/sync/sources/{id}/errors/mark-all-viewed` | Mark all errors of a sync source as viewed. Returns the number flipped |
| `GET` | `/sync/sources/backends` | List available backends |
| `POST` | `/sync/sources/{id}/report-error` | Send an error report to qHarbor |
| `GET` | `/sync/sources/{id}/error-report` | Get an error report for manual sending |
| `POST` | `/sync/sources/{id}/scope-mappings` | Create a new scope mapping |
| `PATCH` | `/sync/sources/{id}/scope-mappings/{identifier}` | Update a scope mapping |
| `DELETE` | `/sync/sources/{id}/scope-mappings/{identifier}` | Delete a scope mapping |
| `GET` | `/sync/sources/{id}/scope-mappings/unmapped` | Get unmapped scope identifiers |
| `POST` | `/sync/sources/{id}/scope-mappings/autoassign` | Auto-assign scope mappings |

### Sync Items

Manage sync items — individual datasets to be synchronized.

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/sync/items?source_id=...` | List sync items (with filtering and pagination) |
| `GET` | `/sync/items/{id}` | Get a sync item by ID |
| `PATCH` | `/sync/items/{id}/reset` | Reset a sync item for retry |
| `PATCH` | `/sync/items/{id}/prioritize` | Prioritize a sync item |
| `POST` | `/sync/items/{id}/report-error` | Send an error report to qHarbor |
| `GET` | `/sync/items/{id}/error-report` | Get an error report for manual sending |

### Sync Backends

Manage sync backend packages.

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/sync/backends` | List installed backends |
| `POST` | `/sync/backends/install/pypi` | Install a backend from PyPI |
| `POST` | `/sync/backends/install/local` | Install a backend from local path |
| `POST` | `/sync/backends/update/pypi` | Update a backend from PyPI |
| `POST` | `/sync/backends/update/local` | Update a backend from local path |
| `DELETE` | `/sync/backends/{name}` | Uninstall a backend |

### Sync Converters

Manage converter packages for file format conversion.

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/sync/converters` | List installed converters |
| `POST` | `/sync/converters/install/pypi` | Install a converter from PyPI |
| `POST` | `/sync/converters/install/local` | Install a converter from local path |
| `POST` | `/sync/converters/update/pypi` | Update a converter from PyPI |
| `POST` | `/sync/converters/update/local` | Update a converter from local path |
| `DELETE` | `/sync/converters/{name}` | Uninstall a converter |

### Admin

Inspect service state for support and debugging.

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/admin/logs/{service}?n=...` | Return the most recent N lines of a service's log file. `service` is `api` or `sync_agent`. Pass `n=-1` to return the full file (default `n=500`) |

---

## Error Handling

All API errors return a consistent JSON error model:

```json
{
  "error": "sync_source_not_found",
  "detail": "Sync source with id 123 not found"
}
```

- **`error`**: A machine-readable error code (snake_case)
- **`detail`**: A human-readable error message (optional)

### Error Codes Reference

#### General Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `server_error` | 500 | An internal server error occurred |
| `server_unreachable` | 502 | The remote server cannot be reached |

#### Auth Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `no_user_logged_in` | 401 | No user is currently logged in |
| `user_already_has_token` | 409 | User already has a local token (delete first) |
| `token_not_found` | 404 | Token uid not found in local database |

#### Sync Agent Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `sync_agent_not_installed` | 503 | Sync agent service is not installed |
| `sync_agent_already_running` | 409 | Sync agent is already running |
| `sync_agent_already_stopped` | 409 | Sync agent is already stopped |
| `sync_status_error_not_found` | 404 | Sync-status error record not found by id |

#### Sync Source Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `sync_source_not_found` | 404 | Sync source not found |
| `sync_source_scope_not_found` | 404 | Default scope not found |
| `sync_source_backend_not_found` | 404 | Backend not found |
| `sync_source_name_already_taken` | 409 | Source name already exists |
| `sync_source_already_running` | 409 | Sync source is already running |
| `sync_source_already_stopped` | 409 | Sync source is already stopped |
| `sync_source_config_error` | 422 | Invalid configuration |
| `sync_source_backend_has_no_owner` | 422 | Take-ownership called on a source whose backend does not support ownership (`has_owner=False`) |
| `sync_source_error_not_found` | 404 | Sync-source error record not found by id |
| `scope_mapping_not_found` | 404 | Scope mapping not found for identifier |
| `scope_mapping_already_exists` | 409 | Scope mapping already exists for identifier |
| `scope_mapping_invalid_scope` | 404 | Target scope UUID does not exist |

#### Sync Item Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `sync_item_not_found` | 404 | Sync item not found |
| `sync_item_scope_mapping_not_found` | 404 | Sync item has a scopeIdentifier but no mapping exists |

#### Backend Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `backend_not_found` | 404 | Backend not found |
| `backend_install_failed` | 422 | Backend installation failed |
| `backend_update_failed` | 422 | Backend update failed |
| `backend_uninstall_failed` | 422 | Backend uninstall failed |
| `backend_invalid_package_path` | 422 | Invalid local package path |

#### Converter Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `converter_not_found` | 404 | Converter not found |
| `converter_install_failed` | 422 | Converter installation failed |
| `converter_update_failed` | 422 | Converter update failed |
| `converter_uninstall_failed` | 422 | Converter uninstall failed |
| `converter_invalid_package_path` | 422 | Invalid local package path |

#### Admin Errors

| Error Code | Status | Description |
|------------|--------|-------------|
| `log_file_not_found` | 404 | Requested service's log file does not exist on disk |

---

## Python API

The Python API provides direct access to functionality. It's primarily intended for internal use by the REST API layer and the sync agent itself.

### Local API Tokens

```python
from etiket_api.python_api.auth.local.api_tokens import (
    create_api_token,
    list_api_tokens,
    delete_api_token,
    check_api_token,
)

# Create a token for the current user
token = await create_api_token(name="my-sync-token")
print(token.api_token)  # only shown on create

# List all local tokens
tokens = await list_api_tokens()
for t in tokens:
    print(f"{t.user_name}: {t.api_key_name} ({t.server_url})")

# Check if a token is still valid
result = await check_api_token(api_key_uid="...")
print(result.valid)

# Delete a token (revokes on server first)
await delete_api_token(api_key_uid="...")
```

### Sync Agent

```python
from etiket_api.python_api.sync.sync_agent import (
    get_sync_status,
    get_sync_errors,
    start_sync_agent,
    stop_sync_agent,
    set_sync_status_error_viewed,
    mark_all_sync_status_errors_viewed,
)

# Get current status
status = await get_sync_status()

# Fetch only unviewed errors (UI "unread" pane)
unread = await get_sync_errors(limit=10, skip=0, viewed=False)

# Mark one as viewed; or mark all of them
await set_sync_status_error_viewed(error_id=unread[0].id, viewed=True)
flipped = await mark_all_sync_status_errors_viewed()

# Start/stop the agent
await start_sync_agent()
await stop_sync_agent()
```

### Sync Sources

```python
from etiket_api.python_api.sync.sync_sources import (
    list_sync_sources,
    get_sync_source,
    get_sync_source_by_name,
    update_sync_source,
    delete_sync_source,
    start_sync_source,
    stop_sync_source,
    list_sync_source_definitions,
    reset_sync_source_sync_items,
    take_ownership,
    release_ownership,
    # Scope mapping functions
    create_scope_mapping,
    update_scope_mapping,
    delete_scope_mapping,
    get_unmapped_scope_identifiers,
    autoassign_scope_mappings,
    # Exceptions
    SyncSourceAlreadyRunningError,
    SyncSourceAlreadyStoppedError,
)

# List all sources
sources = list_sync_sources()

# Get a specific source
source = get_sync_source(source_id=1)
source = get_sync_source_by_name("my_source")

# Start/stop a sync source
try:
    source = start_sync_source(source_id=1)
except SyncSourceAlreadyRunningError:
    print("Source is already running")

try:
    source = stop_sync_source(source_id=1)
except SyncSourceAlreadyStoppedError:
    print("Source is already stopped")

# Reset failed sync items for a source (to retry them)
reset_count = reset_sync_source_sync_items(source_id=1)
print(f"Reset {reset_count} failed items")

# Reset ALL sync items (including successful ones)
reset_count = reset_sync_source_sync_items(source_id=1, include_successful=True)
print(f"Reset {reset_count} items (all)")

# List available backends
backends = list_sync_source_definitions()

# Claim ownership of a sync source for the current user (overwrites any existing owner)
source = await take_ownership(source_id=1)
print(source.owner)

# Release your own ownership (errors if you are not the current owner)
source = await release_ownership(source_id=1)
assert source.owner is None

# Error-log "viewed" management
from etiket_api.python_api.sync.sync_sources import (
    read_sync_source_errors,
    set_sync_source_error_viewed,
    mark_all_sync_source_errors_viewed,
)

# Fetch only unviewed errors (UI "unread" pane)
unread = await read_sync_source_errors(source_id=1, viewed=False)

# Mark one error as viewed, or un-view it
await set_sync_source_error_viewed(error_id=unread[0].id, viewed=True)

# Mark everything as viewed in one shot; returns the count flipped
flipped = await mark_all_sync_source_errors_viewed(source_id=1)
```

#### Scope Mappings

Scope mappings are used when a backend provides per-item scope routing. Each sync item may have a `scopeIdentifier` string that needs to be mapped to an actual scope UUID.

```python
import uuid
from etiket_api.python_api.sync.sync_sources import (
    create_scope_mapping,
    update_scope_mapping,
    delete_scope_mapping,
    get_unmapped_scope_identifiers,
    autoassign_scope_mappings,
)

# Find identifiers that don't have a mapping yet
unmapped = get_unmapped_scope_identifiers(sync_source_id=1)
print(f"Unmapped identifiers: {unmapped}")

# Create a mapping from identifier to scope UUID
mapping = create_scope_mapping(
    sync_source_id=1,
    scope_identifier="my_scope_identifier",
    scope_uuid=uuid.UUID("12345678-1234-1234-1234-123456789abc"),
)

# Update an existing mapping to point to a different scope
mapping = update_scope_mapping(
    sync_source_id=1,
    scope_identifier="my_scope_identifier",
    scope_uuid=uuid.UUID("87654321-4321-4321-4321-cba987654321"),
)

# Delete a mapping
delete_scope_mapping(sync_source_id=1, scope_identifier="my_scope_identifier")

# Auto-assign mappings based on scope name matching
# (creates mappings where identifier matches exactly one scope name)
created_count = autoassign_scope_mappings(sync_source_id=1)
print(f"Auto-assigned {created_count} mappings")
```

### Sync Items

```python
from etiket_api.python_api.sync.sync_items import (
    list_sync_items,
    get_sync_item,
    reset_sync_item,
    prioritize_sync_item,
)

# List items for a source
items = list_sync_items(source_id=1)

# Get a specific item
item = get_sync_item(sync_item_id=123)

# Reset or prioritize an item
reset_sync_item(sync_item_id=123)
prioritize_sync_item(sync_item_id=123)
```

### Backends

```python
from etiket_api.python_api.sync.backends import (
    list_backends,
    install_backend_from_pypi,
    install_backend_from_local_path,
    update_backend_from_pypi,
    uninstall_backend,
)

# List installed backends
backends = list_backends()

# Install from PyPI
install_backend_from_pypi("etiket-sync-agent-qcodes")

# Install from local path
install_backend_from_local_path("/path/to/my-backend")

# Update from PyPI
update_backend_from_pypi("etiket-sync-agent-qcodes")

# Uninstall
uninstall_backend("etiket_sync_agent_qcodes")
```

### Converters

```python
from etiket_api.python_api.sync.converters import (
    list_converters,
    install_converter_from_pypi,
    install_converter_from_local_path,
    update_converter_from_pypi,
    uninstall_converter,
)

# List installed converters
converters = list_converters()

# Install from PyPI
install_converter_from_pypi("my-converter-package")

# Install from local path
install_converter_from_local_path("/path/to/my-converter")

# Update from PyPI
update_converter_from_pypi("my-converter-package")

# Uninstall
uninstall_converter("my_converter")
```

### Admin / Logs

```python
from etiket_api.python_api.admin.logs import (
    LogService,
    get_recent_log_lines,
)

# Read the last 500 lines of the API's log file
lines = get_recent_log_lines(LogService.api, n=500)
for line in lines:
    print(line)

# Read the entire log file (up to the 10 MB rotation cap)
all_lines = get_recent_log_lines(LogService.sync_agent, n=-1)
```

---

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                      eTiKeT SDK                             │
│                   (User-facing Python SDK)                  │
└─────────────────────────┬───────────────────────────────────┘
                          │ HTTP
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                      eTiKeT API                             │
│  ┌──────────────────┐   ┌──────────────────────────────┐    │
│  │    REST API      │   │        Python API            │    │
│  │    (FastAPI)     │──▶│   auth/local, sync/agent,    │    │
│  │                  │   │   sync/sources, sync/items.. │    │
│  └──────────────────┘   └──────────────────────────────┘    │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                   eTiKeT Sync Agent                         │
│            (Background synchronization service)             │
└─────────────────────────────────────────────────────────────┘
```

---

## Requirements

- Python 3.10+
- etiket_sync_agent
- etiket_client
