Operational Map implementation walkthrough

This page explains how the operational map works and the reasoning a reviewer should assess. Release findings and verification results are recorded separately in the repository-root OPERATIONAL_MAP_REVIEW.md.

Purpose and boundaries

The map is a view of Mission Plan deployment coordinates. It overlays those positions on locally uploaded raster MBTiles archives. It does not track a device through GPS, poll for position updates, or deploy configuration. Reloading the page refreshes its mission snapshot.

“Offline” means Nautobot and its local storage remain reachable while Internet access is unavailable. It does not mean a browser can operate after disconnecting from Nautobot. OpenLayers (10.11.0, BSD 2-Clause, vendored locally -- see static/nautobot_trax/openlayers/README.txt) renders the map; basemap tiles are served locally too. There is no external tile provider fallback in the map script.

Request flow

flowchart TD
    Upload[Atlas upload through UI or REST] --> Validate[Model validates MBTiles metadata]
    Validate --> Store[Private archive storage and MapTileSet metadata]
    Browser[Authenticated browser] --> Page[OperationalMapView]
    Page --> Permissions[Restrict missions, operations, and basemaps]
    Permissions --> JSON[Escaped JSON configuration in template]
    JSON --> OpenLayers[Local OpenLayers renderer]
    OpenLayers --> Tiles[Permission-checked XYZ tile endpoint]
    Tiles --> Available[Check archive availability]
    Available --> Cache[Versioned tile-byte cache]
    Cache -->|Miss| SQLite[Read archive using TMS row coordinates]
    Browser --> Download[Object-permission-checked archive download]
    Download --> Store

Archives have no generic media URL. UI downloads use the atlas view permission; REST downloads use the viewset's restricted object lookup and support API authentication. Both return attachments with a fixed binary content type.

Files and responsibilities

File	Responsibility
models/map.py	Defines MapTileSet, derived metadata, and upload validation/save behavior
map_storage.py	Private filesystem storage, authorized file URLs, safe attachments, and commit-aware cleanup
utils.py	Reads archive metadata and retrieves one tile using SQLite
forms.py	Atlas UI forms and permission-restricted operation selection
api/serializers.py, api/views.py, api/urls.py	Standard Nautobot REST management of atlas records/uploads
views.py	Atlas UI, filtered mission data, basemap configuration, tile authorization and caching
urls.py, navigation.py	Registers routes and permission-aware navigation links
templates/nautobot_trax/operational_map.html	Displays filters, markers, labels, basemap selection, and zoom behavior
static/nautobot_trax/openlayers/	Locally served OpenLayers 10.11.0 assets
migrations/0009_*, 0010_*, 0011_*	Creates the atlas model, records available zooms, and migrates legacy public archives to private storage
migrations/0012_*	Adds optional operation basemap associations and a preferred map
tests/test_operational_map.py	Metadata, uploads, permissions, filters, and cache regressions
tests/test_map_security.py	Download authorization, hostile input limits, private storage, migration, cleanup, and demo-role regressions
tests/test_map_availability.py	Storage loss/restoration, conditional responses, operation basemap selection, and deployment diagnostics
management/commands/check_map_storage.py	Checks private-directory writes and referenced archive readability on the current instance
management/commands/cleanup_map_archives.py	Age-guarded preview/removal of unreferenced private archives
tools/check_operational_map_browser.py	Exercises the actual map script in local headless Chrome
tools/build_mbtiles_overviews.py	Optional offline helper that creates overview imagery in a separate archive

Model and upload handling

MapTileSet stores a unique name, description, uploaded file, and activation flag. The raster format, attribution, zoom limits/list, and geographic bounds are extracted from the archive instead of accepting the browser's assertions.

clean() reads the file while it is still an upload. The helper copies the file to a temporary filesystem path because SQLite needs a path, and restores the upload stream's original position. A label/activation edit skips rescanning an unchanged committed archive. save() also validates a newly assigned upload so programmatic/API creation populates derived fields. Updating only the file extends update_fields to persist the derived fields and timestamp as well.

The parser accepts the MBTiles table/view interfaces, raster PNG/JPEG/WebP format names, and common MIME aliases. It queries actual zoom levels instead of trusting potentially stale minzoom/maxzoom metadata. Missing or invalid bounds are derived from coverage at the most detailed zoom. Vector PBF archives are rejected because the map's ol.source.XYZ raster tile layer cannot render vector data.

New uploads require an .mbtiles extension and receive UUID-based storage names. PrivateMapStorage rejects roots inside media/static storage, writes files with mode 0600, and creates directories with mode 0700. Its generic url() raises an error; the custom field supplies the authorized atlas download route instead. An administrator must also keep the directory out of proxy/static routes and mount persistent storage in every service that accesses it.

The local workspace Dockerfiles build from the workspace parent. Their adjacent Dockerfile.dockerignore files allow only the source projects they copy and exclude environment files, uploaded media, atlas files, and the MOBAC workspace. A .dockerignore inside the TRAX directory alone does not protect a parent build context; see Docker's build-context rules.

GitHub Actions uses development/Dockerfile.ci with the TRAX checkout as its build context. It installs companion apps from the constrained Poetry lockfile, so CI does not require neighboring source checkouts. Its adjacent ignore file excludes local credentials, archives, and caches. CI tests archive processing on Python 3.12 and 3.13; Python 3.10 retains core coverage and verifies that archive processing fails closed when SQLite resource limits are unavailable.

Uploaded schemas can contain executable SQLite views. _mbtiles_connection() therefore opens a read-only URI, disables trusted-schema features and memory mapping, enables query-only mode, and installs SQLite resource limits and a progress-handler deadline. Python 3.11 or later is required for setlimit(); older runtimes fail closed when processing an archive. Validation has a 30-second default query budget, tile reads 2 seconds. The progress handler interrupts SQLite VM work; this is not a hard wall-clock deadline on filesystem I/O or a process-wide memory limit. Ordinary indexed tables and views remain supported within the limits.

Validation bounds the temporary copy to 5 GiB by default, limits metadata entries and values, checks every coordinate against its zoom grid even when bounds are provided, and limits tile BLOBs to 16 MiB. SQL parameters bind requested coordinates. These checks establish structure, supported metadata, and bounded query work; they do not decode every tile to prove it is valid raster imagery. Web-server upload limits, filesystem quotas, and representative atlas timings remain deployment responsibilities.

Replacement/deletion schedules removal of unreferenced files with transaction.on_commit(). Rollback preserves the previous file; shared files remain while referenced. A new upload written before rollback can remain as a private orphan. cleanup_map_archives previews such files; --apply removes only unreferenced regular files older than the configured age guard.

Migration 0011 moves the entire legacy trax_map_tiles namespace, including old orphaned archives, out of media storage. It verifies SHA-256 copies before removing originals and preserves names so interrupted migrations can be retried. Missing referenced files or conflicting copies stop the upgrade. Run the migration with services stopped and both original media and private storage mounted. Its reverse intentionally does not restore public files. See the administrator instructions for configuration and recovery prerequisites.

Mission selection and permissions

OperationalMapView uses Nautobot's ObjectPermissionRequiredMixin. Its dispatch checks mission-view permission and replaces the view queryset with an object-restricted queryset. get_context_data() then excludes expired missions and records without both coordinates, and joins tenant/location to avoid a separate query for each marker.

The operation selector independently restricts its queryset to operations the user may view. Multiple selections use OR semantics and .distinct() to avoid duplicating a mission shared by two operations. Invalid or inaccessible selected operations produce form errors and an empty mission result. Clearing the filter includes all otherwise permitted plotted missions, including unassigned ones.

Active basemaps use their own restricted queryset. Permission to view a mission does not grant permission to view a basemap, and operation selection does not expand mission permissions. Navigation permissions hide links, while server checks remain the enforcement boundary.

Operation records can associate several Map Tile Sets and one preferred map. The preferred map is associated implicitly. _operational_basemaps() intersects these references with permitted active maps and checks each archive's SQLite header without scanning its tiles. Preferred ready maps sort first, other associated ready maps next, then remaining permitted ready maps. Name and UUID break ties independently of operation input order. The selector retains manual alternatives; a notice explains fallback when no associated map is available. Unavailable maps are reported only within the user's permitted atlas queryset. No stored availability flag can remain stale after restoring an archive.

Tenant/location names are included as part of an authorized mission's displayed summary. A deployment needing independent redaction of related-object names should explicitly define and test that policy.

Browser rendering and injection prevention

Django's json_script serializes the two configuration objects into inert JSON script elements. The renderer reads their text content rather than evaluating JavaScript generated from user data. Two distinct, both-safe mechanisms handle untrusted strings downstream, matched to how each one is rendered: permanent mission labels and the basemap switcher's option text are assigned via textContent, which never interprets its input as markup regardless of content. Popup fields and tile attribution are built as HTML strings (innerHTML) and therefore pass through escapeHtml() first -- the one path in this file where omitting it would be exploitable.

Circle markers render through an ol.layer.Vector (OpenLayers' own canvas renderer), avoiding individual SVG/DOM element cost per marker. Permanent labels still create one DOM element (an ol.Overlay) each; the implementation is not a clustered or paginated view for arbitrarily large mission inventories. Add performance tests with representative mission counts before promising scale.

The marker colors correspond to active/planned mission status. Expiration here means the stored expired status; a past end date alone does not remove a marker. Coordinates are already normalized/stored on MissionPlan; the browser does not implement a second MGRS conversion algorithm.

Offline zoom and viewport behavior

The first permitted active atlas is selected initially. The viewport prioritizes missions inside its coverage, then atlas coverage, then a default view. Missions outside coverage remain in the data without pulling the initial view away from available imagery. Switching a basemap updates both zoom limits and viewport.

The tile layer builds an OpenLayers TileGrid from the actual zoom_levels. Each source-grid index represents one exported XYZ resolution. A custom tileUrlFunction translates that index to the native XYZ zoom in the URL; the grid already computes the correct x/y coordinates and geographic extent. For older records without zoom_levels, the grid includes every level between min_zoom and max_zoom.

The source's zDirection: 1 chooses the coarser available resolution between exported levels. OpenLayers stretches that tile over its full native extent, including when the view zoom exceeds the archive maximum. Remapping a parent tile URL into a finer grid would repeat the entire image inside each child tile and misplace geography. Using native resolutions also avoids multiplying fine-tile requests for a regional overview. One zoom level finer can require four times as many tiles for the same area; four levels finer can require 256 times as many. See the upstream TileGrid API.

Two additional display zoom levels stretch existing imagery to separate nearby markers. They add no cartographic detail. Zooming out below the native minimum requires real overview tiles. The optional overview builder copies an archive, reduces each group of four children into a parent, respects TMS north/south orientation, and never overwrites the source or an existing destination.

tools/check_operational_map_browser.py checks native zoom URLs and samples the final rendered canvas at known geographic coordinates using a four-color tile. These regressions cover native levels, gaps between levels, zoom beyond the native maximum, legacy metadata, and a zero-only atlas. The imagery is local and synthetic; HTTP authentication and binary image serving have separate tests.

The view also respects OpenLayers' default projection/viewport constraints. On a large viewport the effective zoom-out floor can exceed the configured minZoom to keep the viewport within one world. This is independent of archive resolution selection; see the upstream View constraints.

Tile endpoint and cache

Wheel input uses OpenLayers' own ol.interaction.MouseWheelZoom, configured with constrainResolution: false (continuous, not snapped to whole levels) and useAnchor: true (zooms toward the pointer, not the map center). It accumulates a fractional target, eases toward it, and anchors to the pointer natively -- no custom animation code involved. The View's own constrainResolution: false keeps gestures fractional; the Zoom control (the on-screen +/- buttons) steps by a whole level on its own, independent of the wheel interaction's configuration.

Interrupt handling (a drag, a basemap switch, or page unload interrupting an in-flight wheel gesture) and tile-loading behavior during a gesture are OpenLayers' own internal responsibility, not code this project maintains, so tools/check_operational_map_browser.py does not re-verify them. It checks what this project's own code actually owns: that the gesture produces continuous, pointer-anchored zoom, that zoom buttons remain whole-level, that native tiles remain geographically aligned across display zooms, that mission labels/popups never interpret untrusted names as HTML, and that coverage/basemap-switching notices update correctly. The virtual-clock tests schedule frame callbacks deterministically; they verify interaction and request behavior, not a workstation's physical frame rate.

The north_carolina and austria dictionaries in the browser checks are synthetic mission fixtures. [north_carolina] passes a one-mission list to a case; it does not configure production coverage or an AOR. US/European bounds exercise both inside-coverage and outside-coverage behavior. Gesture tests take their starting coordinates from the fixture instead of a separate hard-coded location.

MapTileView applies the atlas permission mixin before loading an object, cache entry, or conditional response. It rejects invalid XYZ values before computing large powers, then rejects unavailable zoom levels. The SQLite query uses bound parameters for coordinates. MBTiles uses bottom-origin TMS rows, so a request with top-origin XYZ row y is translated to 2**z - 1 - y.

The server caches tile bytes for one day and missing-tile sentinels for ten minutes. Cache keys include atlas ID, last-updated timestamp, and tile coordinates. Tile URLs also include a version parameter, so a successful file replacement gets fresh URLs. A missing tile returns 404. Successful responses include an ETag and private, max-age=0, must-revalidate; conditional requests can return 304 only after object permission, live archive readability, and tile-existence checks. Thus new requests after permission revocation are denied even for a cached tile. Already downloaded copies cannot be recalled. Archive downloads use private, no-store, attachment disposition, a generated .mbtiles download name, application/octet-stream, and nosniff.

Missing/unreadable storage, invalid archive headers, and expected SQLite errors produce a controlled 503 with private, no-store and Retry-After: 30. SQLite query failures also log an atlas-ID warning. An ordinary missing tile remains a 404. Availability is checked before serving bytes cached in Redis or validating a browser's saved representation, so caches cannot conceal a lost mount. Unexpected programming errors are not silently swallowed. Large atlases still need a cache capacity policy; permissions and TTL alone do not bound Redis memory consumption.

Directly changing an archive on disk bypasses timestamp-based invalidation. Use supported model/form/API updates. File lookup currently requires local filesystem storage because the view accesses FileField.path; remote object storage requires a separate retrieval/local-cache design.

Review questions and expected evidence

Question	Evidence to request
Are mission and operation constraints independent?	Restricted-user and inaccessible-filter tests
Can an atlas be read through a different URL?	Direct raw-media/download authorization regression tests
Can an uploaded filename become active content?	Reject/safe-download tests for HTML-named SQLite files
What stops an expensive schema or huge BLOB?	SQLite budget/limit tests and representative large-atlas timings
Do file changes update metadata and caches?	UI/API replacement tests and new-version ETag checks
What happens after storage loss or deletion?	Expected error response, cleanup/retention tests, restore procedure
Can zoom changes multiply requests unexpectedly?	Browser scenarios for sparse, zero-only, and detail-only atlases
Is the installed package self-contained?	Fresh-wheel install, migrations, collectstatic, and license/content inspection
Is the release safe outside the demo?	Explicit opt-in for demo account creation and documented role provisioning

Claims to keep precise

The design supports local raster basemaps, stored mission positions, and server-side object-restricted map, tile, and archive-download views. Query limits reduce hostile-schema exposure; they do not establish complete sandboxing of SQLite or a hard process memory budget. A passing map suite does not establish live position tracking, recall of downloaded imagery, large-inventory performance, or compatibility with every version/backend in broad dependency ranges.

The fixed-password operator/operator login is restricted to explicit demo provisioning. enable_demo_user defaults to False; the supplied development configuration enables it. setup_operator_user refreshes the demo operator's existing mission/operation access and MapTileSet view/add/change/delete permissions without resetting an existing password unless --reset is supplied.