Metadata-Version: 2.4
Name: diagmunge
Version: 2026.11.8.post1
Summary: Shared Qualcomm DIAG transport core (+ format-munge tools)
Author: Luke Jenkins
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/lukejenkins/diagmunge
Project-URL: Source, https://github.com/lukejenkins/diagmunge
Project-URL: Changelog, https://github.com/lukejenkins/diagmunge/releases
Keywords: diag,modem,cellular,baseband,lte,5g,pcap,gsmtap,hdlc
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Telecommunications Industry
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Communications
Classifier: Topic :: System :: Hardware
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Provides-Extra: serial
Requires-Dist: pyserial; extra == "serial"
Provides-Extra: libusb
Requires-Dist: pyusb; extra == "libusb"
Provides-Extra: decode
Requires-Dist: diaggrok>=2026.11.8; extra == "decode"
Provides-Extra: munge
Requires-Dist: diaggrok>=2026.11.8; extra == "munge"
Dynamic: license-file

# diagmunge

> Part of the **[cellular `diag*` toolkit](https://github.com/lukejenkins/cellular#the-diag-toolkit)**: start there for how the capture/decode pieces fit together.

Shared **Qualcomm DIAG transport core + offline format-munge tools**: arm
the DIAG log mask, pump raw HDLC frames over serial, raw-fd, TCP, or UDP,
and munge DIAG capture formats (HDLC ↔ DLF ↔ JSONL) offline. It is the
transport foundation shared by the capture/daemon siblings in this
constellation, and stands on its own as a DIAG transport core you can
build on.

## Public API

```python
import diagmunge

client = diagmunge.DiagClient.from_serial("/dev/ttyUSB0")   # or .from_tcp_client(host, port)
client.subscribe_logs([0xB0C0, 0xB0E2])                      # arm the mask (skip if a daemon owns it)
try:
    while True:
        opcode, payload = client.recv(timeout=30.0)            # one de-framed HDLC frame
        if opcode == diagmunge.DIAG_LOG_F:
            ...                                                # decode with diaggrok, or store
finally:
    client.close()
```

`recv()` returns one `(opcode, payload)` frame at a time, already
unescaped and CRC-checked, and raises `TimeoutError` when nothing arrives
in time. To split and filter the stream yourself, `read_raw()` returns the
raw, still-escaped HDLC bytes instead.

Re-exported at the top level: `DiagClient`, `Dge1SeqTracker`,
`parse_dge1_header`, and the `DIAG_*` opcode constants. Internal transport
classes live in `diagmunge.transport`.

## The munge tools (`diagmunge.munge`)

Offline DIAG format converters, each a self-contained CLI:

| Tool | Purpose |
|------|---------|
| `python -m diagmunge.munge.hdlc_to_dlf` | raw HDLC stream → DLF record format |
| `python -m diagmunge.munge.capture_dlf_from_diag` | subscribe to log codes on a live device, capture frames → DLF |
| `python -m diagmunge.munge.dlf_to_pcap` | DIAG capture → Wireshark pcap: GSMTAP/LTE on UDP/4729, plus a sibling `.nr.pcap` (Exported-PDU/NR signalling) |
| `diagmunge.munge.replay` (library) | offline DLF/HDLC replay core: `replay_to_jsonl(dispatch=…, code_filter=…)` with an injected dispatch |
| `diagmunge.gsmtap` (library) | GSMTAP v2 encoder for decoded LTE RRC/NAS records (`encode`), plus GSMTAP v3 parse/convert |
| `diagmunge.exported_pdu` (library) | Wireshark Exported-PDU framing for NR RRC/NAS (pcap link type 252) |
| `diagmunge.pcap` (library) | stdlib-only pcap/pcapng writer and a synthetic Eth/IPv4/UDP frame builder |
| `diagmunge.gsmtap_sink` (library) | `GsmtapUdpSink`: send GSMTAP frames live over UDP to a listening Wireshark |

Console entry points `diagmunge-hdlc-to-dlf`, `diagmunge-capture-dlf`, and
`diagmunge-dlf-to-pcap` are declared for installed use. The munge tools
decode via `diaggrok` (the `munge` extra); a bare `import diagmunge` never
pulls it in.

### `dlf_to_pcap`: DIAG capture to a Wireshark pcap

Reads a DIAG/HDLC/DLF capture and encapsulates the signalling records into
a Wireshark-readable pcap. LTE RRC/NAS records become GSMTAP frames on
UDP/4729 in `<capture>.diaggrok.pcap`; NR RRC/NAS records become
Exported-PDU blocks in a sibling `<capture>.diaggrok.nr.pcap` (a separate
file because NR uses a different pcap link type). Frame timestamps come
from the capture's GNSS ts64→UTC calibration when the installed `diaggrok`
provides `diaggrok.ts64_cal` and the capture carries GNSS fixes. Otherwise
they use a synthetic monotonic clock, and the tool says so on stderr.

```bash
diagmunge-dlf-to-pcap capture.dlf
diagmunge-dlf-to-pcap capture.hdlc.zst -o out.pcap
diagmunge-dlf-to-pcap capture.dlf --log-code 0xB0C0
```

The encapsulation is clean-room, built on the public Osmocom GSMTAP and
Wireshark Exported-PDU header layouts; `tshark` is used only as an output
oracle in the tests, never as a source.

## Installation

From [PyPI](https://pypi.org/project/diagmunge/):

```bash
pip install diagmunge
# with extras:
pip install "diagmunge[munge]"
```

Each PyPI version is the same code as the GitHub tag of the same number
(`diagmunge==2026.11.8` is tag `v2026.11.8`). If you only want a tool
(`diaggulp`, `diaggpsd`), install the tool; it pulls diagmunge itself.

## Dependencies

The core transport (TCP / UDP / raw-fd) is **stdlib-only**. Optional
extras cover the lazy integrations:

| Extra | Pulls | Used by |
|-------|-------|---------|
| `diagmunge[serial]` | `pyserial` | `DiagClient.from_serial()` |
| `diagmunge[libusb]` | `pyusb` | `DiagClient.from_libusb()` |
| `diagmunge[decode]` | `diaggrok` | `DiagClient.wait_for_log_codes()` |
| `diagmunge[munge]` | `diaggrok` | the `diagmunge.munge` CLI tools |

`import diagmunge` never requires any of them; all are imported lazily
at their call sites. The `decode` and `munge` extras pull `diaggrok` from
PyPI, at least the release this diagmunge was tested against.

`diagmunge.diag_source` (the pcap-native DIAG record source behind
`diaggulp --pcap` replay) needs `diaggrok` too, and reads capture files with
`diaggulp`'s container reader, so install `diaggulp` to use it on a capture.

## Constellation position

```
                 diaggrok   (pure DECODE library; never depends on diagmunge)
                    ▲  (lazy: wait_for_log_codes -> diaggrok.frame.parse_outer_frame)
                    │
                 diagmunge  (this: shared DiagClient transport + munge tools)
                 ▲   ▲   ▲
                 │   │   └── diaggpsd
                 │   └────── diaggulp  (consumes diagmunge transport)
                 └────────── other tools/consumers
```

Dependency edges point inward. `diagmunge -> diaggrok` (lazy) is the only
cross-package edge; the reverse never exists.

## Roadmap

The semantic/foreign-format converters belong here too; that is the
*munge* in diagmunge. GSMTAP pcap output has now landed (`dlf_to_pcap`,
above); the remaining converters (WiGLE, Kismet, RINEX) are still to
follow.

## License

Apache License 2.0. See [`LICENSE`](https://github.com/lukejenkins/diagmunge/blob/main/LICENSE) and [`NOTICE`](https://github.com/lukejenkins/diagmunge/blob/main/NOTICE).

The measured QSH-trace masksets
([`src/diagmunge/data/`](https://github.com/lukejenkins/diagmunge/blob/main/src/diagmunge/data/QSH_MASKSET_MEASURED.md), also installed as `diagmunge/data/QSH_MASKSET_MEASURED.md`) come with
**no warranty**. Read their document, which also says how they were measured,
before arming one.
