dran.storage package#
Submodules#
dran.storage.db_introspection module#
- dran.storage.db_introspection.record_exists(conn, table, key_field, key_value)[source]#
Fast existence check using an indexed lookup (UNIQUE field).
Returns True if a record exists, else False.
- Parameters:
conn (Connection)
table (str)
key_field (str)
key_value (Any)
- Return type:
- dran.storage.db_introspection.ensure_processed_files_table(conn)[source]#
Ensure a small registry table exists for processed files.
This enables fast de-duplication across path changes and symlinks.
- Parameters:
conn (Connection)
- Return type:
None
- dran.storage.db_introspection.processed_file_exists_by_path(conn, filepath)[source]#
- Parameters:
conn (Connection)
filepath (str)
- Return type:
- dran.storage.db_introspection.processed_file_hashes_by_size(conn, file_size)[source]#
- Parameters:
conn (Connection)
file_size (int)
- Return type:
dran.storage.sqlite_connection module#
- dran.storage.sqlite_connection.get_connection(db_path, log=None)[source]#
Open a SQLite connection with pragmatic defaults for local workloads.
Settings applied: - WAL mode for better concurrent reads/writes - synchronous NORMAL for balanced durability and speed - busy_timeout to reduce “database is locked” failures
- Parameters:
- Return type:
dran.storage.sqlite_repository module#
- dran.storage.sqlite_repository.insert_dict(conn, table, item)[source]#
Insert a dict into table and return inserted row id.
- dran.storage.sqlite_repository.fetch_row(conn, table, row_id)[source]#
Fetch a row and reconstruct arrays from BLOBs where possible.
- dran.storage.sqlite_repository.get_existing_keys(conn, table, key)[source]#
Load all existing values of a key into a set for fast membership checks.
- Parameters:
conn (Connection)
table (str)
key (str)
- Return type:
dran.storage.sqlite_schema module#
- dran.storage.sqlite_schema.infer_sqlite_type(value)[source]#
Infer an SQLite column type from a sample value.
Uses: - BLOB for non-scalar NumPy arrays - REAL for int/float scalars - TEXT for everything else
dran.storage.sqlite_types module#
- dran.storage.sqlite_types.array_to_blob(arr)[source]#
Encode a NumPy array as bytes for SQLite storage.
Uses np.save into an in-memory buffer, preserving dtype and shape.
Module contents#
- dran.storage.get_connection(db_path, log=None)[source]#
Open a SQLite connection with pragmatic defaults for local workloads.
Settings applied: - WAL mode for better concurrent reads/writes - synchronous NORMAL for balanced durability and speed - busy_timeout to reduce “database is locked” failures
- Parameters:
- Return type:
- dran.storage.ensure_table_from_dict(conn, table, sample, unique_field='FILENAME')[source]#
Create a table if it does not exist.
Column names are taken from sample keys. Each column type is inferred from sample values.
unique_field is used as a UNIQUE constraint if it exists in sample.
- dran.storage.insert_dict(conn, table, item)[source]#
Insert a dict into table and return inserted row id.
- dran.storage.fetch_row(conn, table, row_id)[source]#
Fetch a row and reconstruct arrays from BLOBs where possible.
- dran.storage.get_existing_keys(conn, table, key)[source]#
Load all existing values of a key into a set for fast membership checks.
- Parameters:
conn (Connection)
table (str)
key (str)
- Return type:
- dran.storage.save_record(conn, table, item, *, create_table_fn=None)[source]#
Insert one record. Returns row id.
create_table_fn is optional and lets callers ensure schema before insert.
- dran.storage.array_to_blob(arr)[source]#
Encode a NumPy array as bytes for SQLite storage.
Uses np.save into an in-memory buffer, preserving dtype and shape.