# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.3.1] - 2026-04-13

### Fixed

- **Embedded migration execution**: Migrations now run end-to-end against embedded
  engines (`mem://`, `memory://`, `file://`, `surrealkv://`). Previously
  `execute_migration()` wrapped statements in `BEGIN TRANSACTION;` / `COMMIT
  TRANSACTION;`, which crashes with `IndexError: list index out of range` on
  embedded connections because the upstream `surrealdb` Python SDK's `query()`
  returns an empty result list for transaction-control statements in embedded
  mode. `execute_migration()` now detects embedded URL schemes on the client
  and skips the transaction wrapper. Migrations remain effectively atomic in
  embedded mode because the engine lives in the application process, so a
  crash during migration takes the whole process with it rather than leaving a
  partial remote schema.

## [1.3.0] - 2026-04-01

### Added

- **Aggregation functions** (#1): `math_mean()`, `math_sum()`, `math_max()`, `math_min()` for
  SurrealQL aggregation queries. Compose with `as_()` aliases and `group_by()`/`group_all()` clauses.
- **Query builder GROUP ALL** (#1): `group_all()` method on `Query` for full-table aggregation.
- **Record references** (#2): `record_ref(table, id)` generates `type::record('table', 'id')` expressions
  that render as raw SurrealQL in CRUD operations, not quoted as strings.
- **SurrealDB function values** (#3): `surql_fn(name, *args)` for passing server-side functions
  (time::now(), math::sum, etc.) as field values in CREATE/UPDATE operations.
- **Result extraction integration tests** (#4): Comprehensive tests for `extract_result()`,
  `extract_one()`, `extract_scalar()` against realistic SurrealDB response formats.
- **Embedded connection URLs**: `ConnectionConfig.validate_url` now accepts the full set of
  schemes supported by the underlying `surrealdb` SDK, including the embedded engines
  `mem://`, `memory://`, `file://`, and `surrealkv://`. `enable_live_queries=True` is now
  compatible with embedded engines (they run in-process), and only rejected for `http://` /
  `https://`. Enables edge/device deployments where each host owns its own SurrealDB
  instance without a separate server process.

## [1.2.1] - 2026-03-20

### Fixed

- **RecordID round-trip denormalization**: Added `_denormalize_params()` that recursively
  converts record ID strings (e.g. `'repo:abc123'`) back to `surrealdb.RecordID` objects
  before sending to the SDK. Applied to `create()`, `update()`, `merge()`, `execute()`, and
  `insert_relation()` input data/params. This fixes the round-trip where normalized response
  IDs (strings) were rejected by SurrealDB 3.x when passed back as field values in subsequent
  operations ("Expected `record` but found string")

### Testing

- Added 18 tests covering `_denormalize_params` unit behavior, round-trip identity,
  and input denormalization verification across all CRUD methods

## [1.2.0] - 2026-03-20

### Fixed

- **select() single-record unwrap**: `DatabaseClient.select()` now detects record ID targets
  (e.g. `user:alice`) and unwraps the single-element list returned by the SurrealDB 3.x SDK,
  returning a dict (or None) instead of a list for single-record selects
- **SDK RecordID normalization**: All `DatabaseClient` CRUD responses (`create`, `select`,
  `update`, `merge`, `delete`, `execute`, `insert_relation`) now recursively normalize
  SurrealDB SDK `RecordID` objects to plain strings, preventing type coercion errors when
  consumers pass returned IDs back as field values in subsequent operations

### Testing

- Added 38 tests covering `_is_record_id_target`, `_normalize_sdk_value`, single-record
  select unwrapping, and SDK type normalization across all CRUD methods

## [1.1.0] - 2026-03-19

### Added

- **HNSW vector indexes**: `IndexType.HNSW` for SurrealDB's HNSW (Hierarchical Navigable
  Small World) approximate nearest-neighbor index, the successor to MTREE in SurrealDB 2.x/3.x
- **HnswDistanceType enum**: 8 distance metrics -- CHEBYSHEV, COSINE, EUCLIDEAN, HAMMING,
  JACCARD, MANHATTAN, MINKOWSKI, PEARSON (superset of MTreeDistanceType)
- **hnsw_index() builder**: Convenience function for creating HNSW index definitions with
  dimension, distance metric, vector type, and optional EFC/M tuning parameters
- **HNSW SQL generation**: `generate_table_sql()` and `generate_edge_sql()` emit correct
  `DEFINE INDEX ... HNSW DIMENSION <n> DIST <dist> TYPE <type> [EFC <n>] [M <n>]` syntax
- **HNSW parsing**: Schema parser detects and extracts HNSW indexes with all parameters
  (dimension, distance, vector type, EFC, M) from SurrealDB INFO responses
- **HNSW validation**: Schema validator checks HNSW dimension, distance metric, vector type,
  EFC, and M parameters for code-vs-database consistency
- **HNSW migration diffs**: `diff_indexes()` generates correct forward/backward SQL for
  adding and dropping HNSW indexes
- **HNSW example**: `docs/examples/hnsw_vector_search.py` demonstrating HNSW usage with
  OpenAI embeddings, multiple distance types, and EFC/M tuning

### Testing

- **Test coverage**: 2215 tests passing (up from 2191 in 1.0.0), 9 skipped
- New test suite: `test_hnsw_diff.py` (24 tests covering SQL generation, add/drop diffs,
  all 8 distance types, EFC/M parameters, mixed index type diffs, error cases)

---

## [1.0.0] - 2026-03-13

### Added

- **Vector search threshold**: `vector_search()` now accepts a `threshold` parameter for
  MTREE similarity filtering, generating `<|K,DISTANCE,threshold|>` syntax
- **Similarity scoring**: `similarity_score()` method on Query adds
  `vector::similarity::{metric}(field, vector) AS alias` to SELECT fields
- **similarity_search_query()**: Convenience function combining `vector_search()` and
  `similarity_score()` for common vector search patterns (replaces manual SurrealQL
  construction in consumer projects)

### Fixed

- **Edge diff returns empty for modified edges**: `diff_edges()` now compares fields,
  indexes, events, and permissions when both old and new edges exist (previously returned
  an empty list with a TODO comment)
- **Event condition/action SQL injection**: Added `_validate_event_expression()` that
  rejects statement separators and SQL comments before interpolation into generated SQL
- **Permission rollback SQL always empty**: `_generate_modify_permissions_diff()` now
  generates rollback SQL from old permissions instead of always producing empty backward SQL
- **Bare exception blocks**: Narrowed 9 bare `except Exception:` blocks across migration/
  and connection/ modules to specific exception types

### Changed

- **Project renamed**: `reverie` -> `surql` (PyPI: `oneiriq-surql`, import: `surql`).
  Unified branding with the TypeScript SurrealDB toolkit under the Oneiriq org
- **Version 1.0.0**: First stable release. Development Status upgraded from Alpha to
  Production/Stable
- **CLI command**: `reverie` -> `surql` (e.g., `surql migrate up`, `surql schema show`)
- **Settings section**: `[tool.reverie]` -> `[tool.surql]` in pyproject.toml
- **Cache key prefix**: `reverie:` -> `surql:` by default
- **Split cli/schema.py** (1954 LOC): Extracted into `schema_inspect.py`, `schema_diff.py`,
  `schema_validate.py`, `schema_watch.py`, `schema_visualize.py` with thin command wrappers
- **Split cli/migrate.py** (1232 LOC): Extracted into `migrate_core.py`, `migrate_squash.py`,
  `migrate_advanced.py` with thin command wrappers
- **Split schema/validator.py** (1029 LOC): Extracted utility functions into
  `schema/validator_utils.py`. All files now comply with the 1000 LOC limit

### Testing

- **Test coverage**: 2191 tests passing (up from 2161 in 0.8.0)
- New test suites: `test_edge_diff.py` (edge diff, event validation, permission rollback)
- Extended: `test_query.py` with vector threshold and similarity scoring tests

---

## [0.8.0] - 2026-03-11

### Added

- **Typed Pydantic CRUD**: `create_typed()`, `get_typed()`, `query_typed()`, `update_typed()`,
  `upsert_typed()` functions that accept Pydantic model types and return validated model
  instances instead of raw dicts
- **DEFINE ACCESS support**: `AccessDefinition`, `AccessType`, `JwtConfig`, `RecordAccessConfig`
  schema types with `access_schema()`, `jwt_access()`, `record_access()` builders and
  `generate_access_sql()` for SurrealQL generation
- **IF NOT EXISTS support**: `if_not_exists` parameter on `generate_table_sql()`,
  `generate_edge_sql()`, and `generate_schema_sql()` for idempotent schema migrations
- **Reserved word validation**: `check_reserved_word()` and `SURREAL_RESERVED_WORDS` for
  detecting field names that collide with SurrealDB reserved words (emits warnings, not errors)

### Changed

- **Split query/builder.py**: Extracted `ReturnFormat` enum and 12 standalone free functions
  into `query/helpers.py`, bringing builder.py from 1137 to 947 LOC
- **Split query/graph.py**: Extracted `GraphQuery` class into `query/graph_query.py`, bringing
  graph.py from 1151 to 794 LOC. All files now comply with the 1000 LOC limit

### Testing

- **Test coverage**: 2161 tests passing (up from 2089 in 0.7.0)
- New test suites: `test_typed_crud.py`, `test_access.py`, `test_reserved_words.py`
- Extended: `test_schema_sql.py` with IF NOT EXISTS tests

---

## [0.7.0] - 2026-03-11

### Added

- **Upsert support**: `upsert()` query builder method and `upsert_record()` CRUD function
  for insert-or-update operations
- **Datetime coercion utilities**: `coerce_datetime()` and `coerce_record_datetimes()` for
  converting SurrealDB ISO datetime strings to Python datetime objects, including nanosecond
  truncation and timezone handling
- **SQL generation from schema definitions**: `generate_table_sql()`, `generate_edge_sql()`,
  and `generate_schema_sql()` for generating SurrealQL DEFINE statements directly from
  TableDefinition/EdgeDefinition objects
- **Additional exports**: `extract_result`, `extract_one`, `extract_scalar`, `has_results`,
  and `delete_records` added to package-level `__all__`
- **Field name validation**: Schema field builder functions now validate field names against
  SurrealDB identifier rules (alphanumeric + underscore, dot notation for nested fields)

### Fixed

- **CI format check was a no-op**: `ruff format src tests` (formats in-place, always passes)
  changed to `ruff format --check src tests`
- **GraphQuery.exists() mutated state**: `exists()` modified `self._limit` directly, violating
  immutability. Rewritten to use `count()` without mutation
- **SQL injection in migration diff defaults**: Field default values interpolated into SQL
  without sanitization. Added `_validate_default_value()` with safe literal pattern matching
- **Pytest marker mismatch**: Declared `asyncio` marker but tests use `anyio`. Fixed marker
  declaration to `anyio`
- **Migration executor lacked transactional wrapping**: Statements now execute within
  BEGIN/COMMIT/CANCEL TRANSACTION blocks for atomicity
- **Connection client reconnection**: Calling `connect()` when already connected now properly
  disconnects first before reconnecting
- **Cache TTL logic**: Fixed unreachable code path for custom TTL tracking
- **RecordID empty parts**: `parse()` now validates that table and id parts are non-empty
- **Nested transaction prevention**: Transaction manager checks for active transactions via
  `ContextVar` and raises `TransactionError` if nested
- **asyncio/trio incompatibility**: Replaced `asyncio.sleep`, `asyncio.gather`,
  `asyncio.Semaphore`, and `asyncio.create_task` with anyio equivalents in orchestration
  strategies and streaming module

### Changed

- **Edge schema RELATION mode validation**: Moved from construction-time to SQL generation
  time, allowing incremental composition via `with_from_table()`/`with_to_table()`
- **CHANGES file**: Updated to reflect versions 0.1.0 through 0.7.0

### Testing

- **Test coverage**: 2089 tests passing (up from ~1018)
- New test suites: `test_coerce.py`, `test_schema_sql.py`, `test_upsert.py`
- Field name validation tests added to `test_schema.py`

---

## [0.1.0] - 2026-01-02

### Added

- **SurrealDB Compatibility**: Complete compatibility with common SurrealDB patterns achieved
  - Result extraction utilities for handling SurrealDB response formats
    - `extract_result()` - Extract data from nested/flat result formats
    - `extract_one()` - Get first record or None
    - `extract_scalar()` - Extract aggregate values (COUNT, SUM, AVG, etc.)
    - `has_results()` - Check if result contains records
    - Location: `src/query/results.py:356-514`

  - RecordID angle bracket support for complex IDs
    - Support for `table:⟨complex-id⟩` format required by SurrealDB
    - Compatible with domain-based IDs like `outlet:⟨alaskabeacon.com⟩`
    - Compatible with compound IDs like `document:⟨domain:ulid⟩`
    - Location: `src/types/record_id.py:58-77`

  - SCHEMAFULL edge table support
    - `EdgeMode.SCHEMAFULL` for traditional edge tables with explicit in/out fields
    - `schemafull_edge()` helper function for traditional edge definitions
    - Compatible with entity_relation pattern
    - Location: `src/schema/edge.py:11-157`

  - Example implementations
    - `docs/examples/mtree_vector_search.py` - MTREE vector indexes (1024-dim, COSINE)
    - `docs/examples/schemafull_edge_example.py` - SCHEMAFULL edge table patterns

### Fixed

- **MTREE Index SQL Generation**: Changed from incorrect `FIELDS` keyword to correct `COLUMNS` keyword
  - Previous (incorrect): `DEFINE INDEX ... ON TABLE ... FIELDS embedding MTREE ...`
  - Current (correct): `DEFINE INDEX ... ON TABLE ... COLUMNS embedding MTREE ...`
  - Ensures compatibility with SurrealDB 1.0+ MTREE syntax
  - Location: `src/schema/table.py:393`
  - Tests: `tests/test_mtree_diff.py`

- **AsyncSurreal Client Implementation**: Verified correct usage of AsyncSurreal for async operations
  - Ensures all database operations use proper async/await patterns
  - Connection pooling and retry logic function correctly
  - Location: `src/connection/client.py:9, 89`

### Changed

- **RecordID Validation**: Enhanced to support both standard and angle bracket formats
  - Standard format: `table:id` (alphanumeric + underscores)
  - Angle bracket format: `table:⟨complex-id⟩` (any valid SurrealDB ID)
  - Backward compatible with existing code
  - Location: `src/types/record_id.py`

### Testing

- **Test Coverage**: 447 tests passing
  - Connection management (async operations, pooling, retry logic)
  - Schema definition (tables, fields, indexes, edges)
  - MTREE indexes (SQL generation, diff detection)
  - RecordID validation (standard and angle bracket formats)
  - Result extraction (nested/flat formats, aggregates)
  - CRUD operations (create, read, update, delete)
  - Query building (select, where, order, limit)
  - Migration system (up/down, history tracking)
  - Edge tables (TYPE RELATION and SCHEMAFULL modes)
  - CLI commands (migrate, schema, db)

---

[1.2.1]: https://github.com/Oneiriq/surql-py/releases/tag/v1.2.1
[1.2.0]: https://github.com/Oneiriq/surql-py/releases/tag/v1.2.0
[1.1.0]: https://github.com/Oneiriq/surql-py/releases/tag/v1.1.0
[1.0.0]: https://github.com/Oneiriq/surql-py/releases/tag/v1.0.0
[0.8.0]: https://github.com/Oneiriq/surql-py/releases/tag/v0.8.0
[0.7.0]: https://github.com/Oneiriq/surql-py/releases/tag/v0.7.0
[0.1.0]: https://github.com/Oneiriq/surql-py/releases/tag/v0.1.0
