TODO:
- [ ] Primitives
    - [ ] 2D Points
    - [x] 2D Lines
    - [ ] Spheres
    - [ ] Cilinders (with/without caps)
    - [-] Meshes
    - [x] 2D Images
    - [x] 3D Images (e.g. textured quads)
- [ ] Renderer architecture / path tracer support
- [-] Server
    - [ ] Test design and finish implementing
    - [ ] Client python package
    - [ ] C/C++ and rust lib
- [x] Frame helpers (ideally at object level, shared across all primitives)
    - Fixed framerate
    - Variable rate
    - Missing frames / holes in data (different from hold last pose)
    - Repeat vs hold vs disappear at end of sequence
    -> think about how this fits together with streaming/prefetching/animated properties
    -> also think about how to visualize this in a timeline viewer
- [x] Transform properties, kinematic trees and node descriptor for constants
    - [x] Transform 2D
    - [x] Cleanup constants dup
- [x] Uniform pool helper
- [x] Images and upload
- [x] .create currently has to be called manually on primitives
- [x] Streaming properties
    - [x] lines with list instead of np array
    - [x] heterogeneous size with max_size
    - [x] streaming prop
    - [x] trigger all uploads before blocking for rendering -> made properties managed by renderer
    - [x] prefetching
            -> likely make a simple mesh class and recreate sequence
            -> this means also porting some GUI, likey making some helpers, ideally also adding the profiler
    - [x] unify GpuBufferProperty and GpuImageProperty
    - [ ] More viewer playback controls (e.g. timeline view, property inspectors)
        - Look at imguizmo sequencer, but likely re-implement with custom use-case in mind
        - Likely want to do most of the ui work in C++ and just offer callbacks for input like click/over/tooltip etc..
        - Look at rerun timeline for feature ideas
    - [ ] Think about getter/setters to update properties and explicit redraw to update properties
- [x] Camera movement
    - [x] Define interface / config / keybindings
    - [x] Initialize camera controls with up / distance from config
    - [x] Implement getters for front, up, right vectors from camera
    - [x] Drag start / end detection
    - [x] Camera mode dependent rotate/pan
    - [x] Scroll zoom (how to handle moving the target (or reducing the distance)?)
    Extra:
    - [ ] 2D controls -> pan only and change zoom mode to modify ortho size?
- [ ] Viewport:
    - [x] resize
    - [ ] multiple viewports
    - [ ] related to UI if decide to do viewports in ImGui windows, not clear how to do default placement (check docs)
- [ ] UI: -> likely in common ui place that can be customized / modified (helpers for things like default layout as well, likely configurable)
    - [x] Scene tree and property view -> custom widget callback per property?
    - [x] Playback UI
    - [x] Fps display for debug
    - [x] Configurable keybindings
    - [ ] NEXT: Expose imgui.get_io for things like want_capture_mouse (or add helper for that)
    - [ ] Port built-in profiler
- [ ] Shaders
    - [ ] cache
    - [ ] export to spirv during package
    - [ ] hot reloading
- [x] Config:
    - [x] Improve camera initialization (especially ortho for 2D case)
    - [x] vulkan validation options
    - [x] add a way to set viewer position in addition to size? (expose glfw for this)
    - [ ] load from disk / yaml + default locations (e.g. home? cwd?)
    - [ ] Think about clean way to handle handedness / zy up conventions.
        -> wrap things that care in a separate module, make different default symbols base on config ?
        -> maybe math and camera utils should expose both, and viewer use config to pick
- [x] Hook pyxpg logging into python logging
- [ ] Cleanup before release:
    - [ ] Public vs private API (hide / protect internal stuff)
    - [ ] Unit tests on all python versions (likely with lavapipe on CI)
- [ ] Extra Features (likely at viewer level with pyxpg helpers / wrappers (optional xpg features enabled on python release)):
    - [ ] Meshoptimizer + meshlets
    - [ ] Gaussian splats
    - [ ] Ray marching / octrees
    - [ ] Marching cubes
    - [ ] Pointclouds
    - [ ] Framegraph

Tools:
- [x] mypy:
    # configured in tool.mypy
    pip install mypy
    mypy ambra
- [x] ruff:
    # configured in tool.ruff.*
    pip install ruff
    # Sort imports
    ruff check --select I --fix
    # Format
    ruff format
    # Lint (not including examples)
    ruff check --exclude examples
    ruff check --exclude examples --fix
- [ ] run on CI once we have automated build / tests


Notes:

Rendering features:
    - Raster (normal viewer):
        - 2D and 3D viewports
        - multi-layer OIT (start from a PoC with depth peeling)
    - Path tracer:
        - Objects that could potentially be path traced:
            - Meshes
            - Other primitives -> should just be meshed or use intersection shaders? depends on complexity of primitive maybe?
            - Need light info too
        - Support for more complex camera models and effects
            - Lens distortion (could be useful even for visualizing different camera models)
            - DoF
            - Vignetting
        - Accumulation settings
        - Light sampling with alias tables
    - Plans:
        - Lights
            - Point, Directional and area first (maybe also leverage slang type, for path tracer)?
            - Environment -> only for path tracer or also do prefiltered IBL?
        - Materials
            - start with common material model for all objects, or maybe just a few materials with dynamic dispatch (leverage slang types)
        - Emissive:
            - Not sure if we need this or if better to just do mesh lights at first, we are not building a general purpose path tracer

- Ideas:
    - Generic viewer type with just basics + specialized viewer types for specific use cases (careful about composition vs specialization)
        - Scene viewer -> classic scene graph
            - Single frame viewer
            - Sequence viewer

Server:
-> 3 layers
    -> raw message (format, type, length, data)
    -> per format parser
    -> parsed message
    Protocols: tcp, http, websockets
    Formats: binary, json, msgpack, pickle (maybe behind off-by-default flag for better security?)
    Builtin-messages:
    - Frame / playback control
    - camera control
    - create, update, delete objects
-> what about REST? it might be convenient to speak with the viewer direclty in rest, different API?
   wrap this API into a rest API? e.g. JSON for body is same as this, and type encoded in endpoint? seems doable
    -> what we have now will be a TcpServer, can also have an  HttpServer and maybe others too?, basically different ways to produce a RawMessage
-> Handle shutdown of TcpServer
    -> exceptions in parsing raw messages should be handled gracefully and log (wrap async callback and print info)
    -> exceptions in main thread should still have the http server exit -> this does not seem to happen correctly atm (maybe connections are keeping this alive in read_exact?) need to switch to async?
-> try small http server and port of websockets server as PoC

Renderer:
-> think about what is the best way to support different types of rendering, and how to not duplicate a huge amount of code
-> 2D vs 3D, raster vs raytrace vs path trace (e.g. accumulation), quality mode (e.g. depth peeling, MSAA, etc..)
-> how does this play out with implicit prefetching / scene stepping? ideally orthogonal?

Thoughts:
-> Lets have the interface always be CPU objects, the distinction between data and streaming properties makes sense to
   me and allows users to customize how the data is loaded but still giving the easy interface with implicit conversion for arrays
-> Renderables should be able to constru
-> Later we can maybe provide some kind of escape hatch for giving gpu buffers directly for these properties, another option
   would be to have renderables that can
-> Big questions that remain:
    -> can properties be shared across objects? since animation is on the property, i dont see any issue with this
    -> If properties are shared, how to handle their GPU counterparts? Are those owned by objects?
       Are they part of the property itself but optional? How does the user configure prefetching vs preload?
