Operational Map implementation walkthrough

This page explains how the operational map works and the reasoning a reviewer should assess. Release findings and verification results are recorded separately in the repository-root OPERATIONAL_MAP_REVIEW.md.

Purpose and boundaries

The map is a view of Mission Plan deployment coordinates. It overlays those positions on locally uploaded raster MBTiles archives. It does not track a device through GPS, poll for position updates, or deploy configuration. Reloading the page refreshes its mission snapshot.

“Offline” means Nautobot and its local storage remain reachable while Internet access is unavailable. It does not mean a browser can operate after disconnecting from Nautobot. Leaflet assets and basemap tiles are served locally. There is no external tile provider fallback in the map script.

Request flow

flowchart TD
    Upload[Atlas upload through UI or REST] --> Validate[Model validates MBTiles metadata]
    Validate --> Store[Private archive storage and MapTileSet metadata]
    Browser[Authenticated browser] --> Page[OperationalMapView]
    Page --> Permissions[Restrict missions, operations, and basemaps]
    Permissions --> JSON[Escaped JSON configuration in template]
    JSON --> Leaflet[Local Leaflet renderer]
    Leaflet --> Tiles[Permission-checked XYZ tile endpoint]
    Tiles --> Cache[Versioned tile-byte cache]
    Cache -->|Miss| SQLite[Read archive using TMS row coordinates]
    Browser --> Download[Object-permission-checked archive download]
    Download --> Store

Archives have no generic media URL. UI downloads use the atlas view permission; REST downloads use the viewset's restricted object lookup and support API authentication. Both return attachments with a fixed binary content type.

Files and responsibilities

File	Responsibility
models/map.py	Defines MapTileSet, derived metadata, and upload validation/save behavior
map_storage.py	Private filesystem storage, authorized file URLs, safe attachments, and commit-aware cleanup
utils.py	Reads archive metadata and retrieves one tile using SQLite
forms.py	Atlas UI forms and permission-restricted operation selection
api/serializers.py, api/views.py, api/urls.py	Standard Nautobot REST management of atlas records/uploads
views.py	Atlas UI, filtered mission data, basemap configuration, tile authorization and caching
urls.py, navigation.py	Registers routes and permission-aware navigation links
templates/nautobot_trax/operational_map.html	Displays filters, markers, labels, basemap selection, and zoom behavior
static/nautobot_trax/leaflet/	Locally served Leaflet 1.9.4 assets
migrations/0009_*, 0010_*, 0011_*	Creates the atlas model, records available zooms, and migrates legacy public archives to private storage
tests/test_operational_map.py	Metadata, uploads, permissions, filters, and cache regressions
tests/test_map_security.py	Download authorization, hostile input limits, private storage, migration, cleanup, and demo-role regressions
management/commands/cleanup_map_archives.py	Age-guarded preview/removal of unreferenced private archives
tools/check_operational_map_browser.py	Exercises the actual map script in local headless Chrome
tools/build_mbtiles_overviews.py	Optional offline helper that creates overview imagery in a separate archive

Model and upload handling

MapTileSet stores a unique name, description, uploaded file, and activation flag. The raster format, attribution, zoom limits/list, and geographic bounds are extracted from the archive instead of accepting the browser's assertions.

clean() reads the file while it is still an upload. The helper copies the file to a temporary filesystem path because SQLite needs a path, and restores the upload stream's original position. A label/activation edit skips rescanning an unchanged committed archive. save() also validates a newly assigned upload so programmatic/API creation populates derived fields. Updating only the file extends update_fields to persist the derived fields and timestamp as well.

The parser accepts the MBTiles table/view interfaces, raster PNG/JPEG/WebP format names, and common MIME aliases. It queries actual zoom levels instead of trusting potentially stale minzoom/maxzoom metadata. Missing or invalid bounds are derived from coverage at the most detailed zoom. Vector PBF archives are rejected because Leaflet's raster layer cannot render vector data.

New uploads require an .mbtiles extension and receive UUID-based storage names. PrivateMapStorage rejects roots inside media/static storage, writes files with mode 0600, and creates directories with mode 0700. Its generic url() raises an error; the custom field supplies the authorized atlas download route instead. An administrator must also keep the directory out of proxy/static routes and mount persistent storage in every service that accesses it.

The local workspace Dockerfiles build from the workspace parent. Their adjacent Dockerfile.dockerignore files allow only the source projects they copy and exclude environment files, uploaded media, atlas files, and the MOBAC workspace. A .dockerignore inside the TRAX directory alone does not protect a parent build context; see Docker's build-context rules.

GitHub Actions uses development/Dockerfile.ci with the TRAX checkout as its build context. It installs companion apps from the constrained Poetry lockfile, so CI does not require neighboring source checkouts. Its adjacent ignore file excludes local credentials, archives, and caches. CI tests archive processing on Python 3.12 and 3.13; Python 3.10 retains core coverage and verifies that archive processing fails closed when SQLite resource limits are unavailable.

Uploaded schemas can contain executable SQLite views. _mbtiles_connection() therefore opens a read-only URI, disables trusted-schema features and memory mapping, enables query-only mode, and installs SQLite resource limits and a progress-handler deadline. Python 3.11 or later is required for setlimit(); older runtimes fail closed when processing an archive. Validation has a 30-second default query budget, tile reads 2 seconds. The progress handler interrupts SQLite VM work; this is not a hard wall-clock deadline on filesystem I/O or a process-wide memory limit. Ordinary indexed tables and views remain supported within the limits.

Validation bounds the temporary copy to 5 GiB by default, limits metadata entries and values, checks every coordinate against its zoom grid even when bounds are provided, and limits tile BLOBs to 16 MiB. SQL parameters bind requested coordinates. These checks establish structure, supported metadata, and bounded query work; they do not decode every tile to prove it is valid raster imagery. Web-server upload limits, filesystem quotas, and representative atlas timings remain deployment responsibilities.

Replacement/deletion schedules removal of unreferenced files with transaction.on_commit(). Rollback preserves the previous file; shared files remain while referenced. A new upload written before rollback can remain as a private orphan. cleanup_map_archives previews such files; --apply removes only unreferenced regular files older than the configured age guard.

Migration 0011 moves the entire legacy trax_map_tiles namespace, including old orphaned archives, out of media storage. It verifies SHA-256 copies before removing originals and preserves names so interrupted migrations can be retried. Missing referenced files or conflicting copies stop the upgrade. Run the migration with services stopped and both original media and private storage mounted. Its reverse intentionally does not restore public files. See the administrator instructions for configuration and recovery prerequisites.

Mission selection and permissions

OperationalMapView uses Nautobot's ObjectPermissionRequiredMixin. Its dispatch checks mission-view permission and replaces the view queryset with an object-restricted queryset. get_context_data() then excludes expired missions and records without both coordinates, and joins tenant/location to avoid a separate query for each marker.

The operation selector independently restricts its queryset to operations the user may view. Multiple selections use OR semantics and .distinct() to avoid duplicating a mission shared by two operations. Invalid or inaccessible selected operations produce form errors and an empty mission result. Clearing the filter includes all otherwise permitted plotted missions, including unassigned ones.

Active basemaps use their own restricted queryset. Permission to view a mission does not grant permission to view a basemap, and operation selection does not expand mission permissions. Navigation permissions hide links, while server checks remain the enforcement boundary.

Tenant/location names are included as part of an authorized mission's displayed summary. A deployment needing independent redaction of related-object names should explicitly define and test that policy.

Browser rendering and injection prevention

Django's json_script serializes the two configuration objects into inert JSON script elements. The renderer reads their text content rather than evaluating JavaScript generated from user data. Popup fields, permanent mission labels, layer names, and attribution pass through escapeHtml() before Leaflet can interpret their HTML strings.

Circle markers use a shared canvas renderer to reduce individual SVG element cost. Permanent labels still create DOM elements; the implementation is not a clustered or paginated view for arbitrarily large mission inventories. Add performance tests with representative mission counts before promising scale.

The marker colors correspond to active/planned mission status. Expiration here means the stored expired status; a past end date alone does not remove a marker. Coordinates are already normalized/stored on MissionPlan; the browser does not implement a second MGRS conversion algorithm.

Offline zoom and viewport behavior

The first permitted active atlas is selected initially. The viewport prioritizes missions inside its coverage, then atlas coverage, then a default view. Missions outside coverage remain in the data without pulling the initial view away from available imagery. Switching a basemap updates both zoom limits and viewport.

The tile layer uses actual zoom_levels. Its _clampZoom override chooses the closest available coarser native level when an intermediate level is absent. This avoids requesting missing zooms or synthesizing a broad overview from a large number of finer tiles. One zoom level finer can require four times as many tiles for the same area; four levels finer can require 256 times as many.

Two additional display zoom levels stretch existing imagery to separate nearby markers. They add no cartographic detail. Zooming out below the native minimum requires real overview tiles. The optional overview builder copies an archive, reduces each group of four children into a parent, respects TMS north/south orientation, and never overwrites the source or an existing destination.

The _clampZoom hook is a private Leaflet interface. Pinning/vendoring 1.9.4 and running the browser scenarios are part of maintaining that decision. A Leaflet upgrade requires rechecking this behavior. Browser scenarios substitute a local image for tile requests, so they test rendering/request counts rather than the HTTP authentication or binary image-serving path.

Tile endpoint and cache

MapTileView applies the atlas permission mixin before loading an object, cache entry, or conditional response. It rejects invalid XYZ values before computing large powers, then rejects unavailable zoom levels. The SQLite query uses bound parameters for coordinates. MBTiles uses bottom-origin TMS rows, so a request with top-origin XYZ row y is translated to 2**z - 1 - y.

The server caches tile bytes, or a missing-tile sentinel, for one day. Cache keys include atlas ID, last-updated timestamp, and tile coordinates. Tile URLs also include a version parameter, so a successful file replacement gets fresh URLs. A missing tile returns 404. Successful responses include an ETag and private, max-age=0, must-revalidate; conditional requests can return 304 only after object permission checks. Thus new requests after permission revocation are denied even for a cached tile. Already downloaded copies cannot be recalled. Archive downloads use private, no-store, attachment disposition, a generated .mbtiles download name, application/octet-stream, and nosniff.

Missing/corrupt storage and expected SQLite errors produce a controlled 404 and a server warning identifying the atlas. Unexpected programming errors are not silently swallowed. Large atlases still need a cache capacity policy; permissions and TTL alone do not bound Redis memory consumption.

Directly changing an archive on disk bypasses timestamp-based invalidation. Use supported model/form/API updates. File lookup currently requires local filesystem storage because the view accesses FileField.path; remote object storage requires a separate retrieval/local-cache design.

Review questions and expected evidence

Question	Evidence to request
Are mission and operation constraints independent?	Restricted-user and inaccessible-filter tests
Can an atlas be read through a different URL?	Direct raw-media/download authorization regression tests
Can an uploaded filename become active content?	Reject/safe-download tests for HTML-named SQLite files
What stops an expensive schema or huge BLOB?	SQLite budget/limit tests and representative large-atlas timings
Do file changes update metadata and caches?	UI/API replacement tests and new-version ETag checks
What happens after storage loss or deletion?	Expected error response, cleanup/retention tests, restore procedure
Can zoom changes multiply requests unexpectedly?	Browser scenarios for sparse, zero-only, and detail-only atlases
Is the installed package self-contained?	Fresh-wheel install, migrations, collectstatic, and license/content inspection
Is the release safe outside the demo?	Explicit opt-in for demo account creation and documented role provisioning

Claims to keep precise

The design supports local raster basemaps, stored mission positions, and server-side object-restricted map, tile, and archive-download views. Query limits reduce hostile-schema exposure; they do not establish complete sandboxing of SQLite or a hard process memory budget. A passing map suite does not establish live position tracking, recall of downloaded imagery, large-inventory performance, or compatibility with every version/backend in broad dependency ranges.

The fixed-password operator/operator login is restricted to explicit demo provisioning. enable_demo_user defaults to False; the supplied development configuration enables it. setup_operator_user refreshes the demo operator's existing mission/operation access and MapTileSet view/add/change/delete permissions without resetting an existing password unless --reset is supplied.