Metadata-Version: 2.4
Name: ducobelt
Version: 0.1.1
Summary: Lowlevel tool to interact with Rhizomatica's mercury modem.
Author-email: "Dr. Andreas Krüger, DJ3EI" <dj3ei@famsik.de>
Project-URL: Homepage, https://codeberg.org/omwb/ducobelt
Project-URL: Repository, https://codeberg.org/omwb/ducobelt
Project-URL: Issues, https://codeberg.org/omwb/ducobelt/issues
Keywords: ham,amateur radio
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Communications :: Ham Radio
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# DuCoBelt: Dual Conveyor Belt

## What does it do, in a nutshell

This program, called "ducobelt", makes it more convenient for amateur radio
operators to operate the [mercury](https://github.com/Rhizomatica/mercury)
modem, in particular, its
[TNC](https://github.com/Rhizomatica/mercury/blob/mercuryv2/docs/TNC.md)
network API.

- You can type TNC commands that are to be sent to mercury's command port
  (TCP port 8300).
    - The line endings you type are normalized to `\r` (aka `CR`) as wanted by
      mercury.
    - The characters you type are normalized to upper case as required by mercury.
- You can see what mercury answers via that command port.
    - These answers are typically given in lines terminated with `\r`.
      If those are displayed as is, each line fully or partially replaces
      the previous one. So ducobelt replaces `\r` line ends coming from mercury
      with `\r\n` (aka `CR LF`) line endings that will display nicely.

So this is a bit like [netcat](https://packages.debian.org/trixie/netcat-openbsd),
[ncat](https://nmap.org/ncat/), or [socat](https://packages.debian.org/trixie/socat),
but with some convenience conversations.

This README also contains hints that may be of general use for mercury users,
whether they choose to use `ducobelt` or not.

## Why the name?

At the end of the day, `ducobelt` just transfers data.
What `mercury` supplies via the ARQ TCP/IP socket is transferred to `np`
and, simultaneously, what you provide via your keyboard
is transferred to that socket, in a parallel **du**al **con**veyer **belt**.

## Alternative

There is an alternative solution by the makers of `mercury`, namely
[mercury-client](https://github.com/Rhizomatica/mercury-client).
That software can be used for first experiments quite as well as `ducobelt`.

The main differences (from my point of view as the `ducobelt` author):

- You need to compile `mercury-client`, while `ducobelt` works out of the box
  with Python.
- `ducobelt` presently will not work on Windows and may or may not work on Mac,
  while `mercury-client` serves all three platforms.
- `mercury-client` gives you a GUI, while `ducobelt` can be run "headless",
  that is, only needs text terminals.
- `mercury-client` tries to offer a unified GUI, while `docubelt` follows the
  time-honored approach "pull what you need out of a well-stocked toolbox".
- Other than `mercury-client`, `ducobelt` leaves the data port (usually 8301) alone.
  This may change in later versions, but I plan to always provide an option to
  not touch that port. On the other hand, it is unclear what to do in
  `mercury-client` to free that port for use by other software.
- The `ducobelt` approach means: You *need* the toolbox. E.g., for text chat,
  you'll need to use some network-enabled program to connect to the data port,
  e.g., `nc` (from one of the `netcat` packages).
- On the other hand, it is more straightforward to rig up fun experiments with
  `ducobelt`. E.g., we played around with `sz` and `rz` (from the `lrzsz` package)
  to send files via the venerable ZMODEM protocol (see below). This is probably
  not the most efficient choice on top of Mercury, but totally can be made to work.

## How to set this up

- Install Linux (`ducobelt` will presently not run unter Windows,
  it _might_ run on Mac).  Personally, I used Debian Trixie.
- Install Python (version 3.12 or later).  An easy way to do that
  under Debian Trixie is: `sudo apt-get install python3.13-venv` - this
  will probably work quite as well under other Debian-based Linux
  distributions.
- Create a virtual environment
    - either [barefeet](https://docs.python.org/3/library/venv.html#creating-virtual-environments)
    - or install [uv](https://github.com/astral-sh/uv) (which also allows to install Python 
      3.12 or better, if you are stuck with an older version) and
      [use](https://docs.astral.sh/uv/pip/environments/) that.
- Activate that virtual env and install `ducobelt` with `pip` (either
  plain `pip install ducobelt` or `uv pip install ducobelt`, depending
  on what you did in the previous step).
- Install [mercury](https://github.com/Rhizomatica/mercury)
  (either plain headless `mercury` or GUI `mercury-ui`, both do;
  unless you need headless, use `mercury-ui`). The easiest way
  to install under Debian or Debian-based Linux distributions is to use
  Rhizomatica's Debian reository as described in
  https://github.com/Rhizomatica/mercury/#install-via-debian-package-on-linux
- **Do not use pip to install mercury!**
  That's the totally wrong "mercury", a completely different software
  quite unrelated to what we want to do here.
- Install some `netcat` version. If in doubt:
  `sudo apt-get install netcat-openbsd` 

## How to run: The four-terminal setup

I suggest using four different terminals.

### How to use four terminals in one window with `tmux`

Personally, I like to use `tmux` to manage many terminals in the same
terminal window.  If you want to do it that way, too: Install `tmux` via
`apt-get install tmux` and start it.  It will display a green bar at the
bottom. You can hit first `Ctrl+B` and then `"`, doing so will split the
part of the window you're presently in into two.  You can move from
one window part to the next with `Ctrl+B` and then `o`.
Shutting down any program that may be runinng with one or two `Ctrl+C`
and then the underlying shell with `Ctrl+D` will close the part
of the window you're in.

### The four-window-plan

What is the plan? If you follow the instructions below, these are the terminals
you'll end up with:

- **Terminal 1** runs the `ducobelt` binary. You type your
  [TNC commands](https://github.com/Rhizomatica/mercury/blob/mercuryv2/docs/TNC.md)
  commands into this terminal. They'll be slightly changed, to upper-case and
  `\r` line endings, and sent to port 8300. 
- **Terminal 2** echoes the replies that come back from the `mercury` or
  `mercury-ui` binary from port 8300.
- **Terminal 3** The `mercury` or `mercury-ui` binary itself runs here.
  You'll see the  log lines scroll by.
- **Terminal 4** In this terminal, you can type chat messages to your QSO partner
  and see what they type back, and/or trigger sending or receipt of (text or
  binary) data.

The most complicated thing here is: How does the `ducobelt` programm get its output
into terminal 2?

Presently, `ducobelt` is hard-wired to write this output into some file `np`
(or named pipe, or terminal, or whatever it finds under that name)
in its current directory. So you need some way to display whatever is written
there in terminal 2.

There is more than one way to do this.

My original way was:

- I create a "named pipe" via `mknod np p` in the directory 
  in which I intend to run the `docubelt` binary later.
- In terminal 2, I start `cat np` to see everything that is written to `np`.

Nowadays,

- I run `ps ax | grep 'ps ax'` in terminal 2 to find out which terminal,
  in the sense of `/dev/pts/N` or similar, is connected with terminal 2.
- In the directory in which I intend to run the `ducobelt` binary
  (and probably already in terminal 1), I run `ln -s /dev/pts/N np`
  (while replacing `N` with the number I found).
- In that same directory (probably in terminal 1), I test this with
  `echo > np hello`, which should cause `hello` to appear in terminal 2.

After that has been accomplished, start either `mercury` or `mercury-ui` 
in terminal 3.  It is your choice which of the two you use.  If in doubt,
`mercury-ui` is recommended: It allows you to set up audio devices and
the cat connection to your radio, and it also allows setting audio level.
It also offers easy access to adjust transmit audio volume.
On the other hand, using plain `mercury` allows headless operation without
a monitor attached to whatever computer runs it.

**Afterwards**, start `ducobelt` in terminal 1.  To do so, you'll have
to activate the Python virtual environment into which you installed `ducobelt`
in that terminal first. 
https://docs.python.org/3/library/venv.html#how-venvs-work tells you how.

If all runs well, the text `BUFFER 0` should appear in terminal 2, and,
every so often, `IAMALIVE`.

You can now type things like `mycall <mycall>` (replacing `<mycall>`
with whatever my real amateur radio call is, omitting the `<>`)
into terminal 1 and should see an `OK` appear in terminal 2.
Besides that, my typical setup nowadays consists of the commands
`bw2750` and `listen on`.  Sending a CQ I do with `cqframe <mycall> 2750`,
after seeing one, I'd do `connect <mycall> <heircall>`. When it is all over,
`disconnect`.

That's the nutshell. For the things you can type, refer to mercury's
[TNC](https://github.com/Rhizomatica/mercury/blob/mercuryv2/docs/TNC.md)
documentation. The manual page of `mercury` also contains this information.

Of course, both the `cqframe` and the `connect` bit will cause `mercury-ui`
(or plain `mercury`) to key your trx and send pertinent data.

The setup of the fourth terminal is only needed when a connection with
another station has actually been established.

To chat with that station, I use `nc` (the binary from the `netcat` package)
to connect to the data port 8301 of `mercury` or `mercury-ui`:

```commandline
nc -v -v localhost 8301
```
This causes each messages from the other side to be spewn out whenever
it happens to come in. This may well be in the middle of my typing
some message, optically disrupting what I type. This is a disadvantage
of using plain raw `nc`.

Any data piped into the tcp/ip socket 8301 on my machine will come out of
the tcp/ip socket 8301 on the other machine, be it text or binary.
There is no checking or conversion. On the plus side, this has the
advantage that you can type any character you want, including any Unicode
characters, which in turn also includes emoticons. Whatever character the
terminal window on the other side has in its font, will display nicely.

On the minus side: **This is also a bit dangerous.**  Once connected
via `mercury`, someone who knows what they are doing might be able to send
crafted escape sequences to your ("intelligent") terminal to launch an
attack on your computer, e.g., with the view of taking it over.

Sending binary data is also possible and straightforward, from the
mercury point of view. It merely becomes an organisational problem:
You don't want such data to spill into your terminal, but typically
want to save it into a file.

So, if I want to transfer binary data, my QSO partner needs to be ready
to receive such data and save it into a file. 

What will work, to a degree: On my machine, I can pipe some file
`binary.data` into the socket to transfer it to the other side with

```commandline
nc -v -v localhost 8301 < binary.data
```

and on the other end, someone can receive that data into a file
via

```commandline
nc -v -v localhost 8301 > binary.data
```

This will transfer the data all right. But then the whole process comes to
a halt. The problem: Mercury itself does not provide a way for the sending
side to communicate to  the other side "this batch of data has now come
to an end", to be followed either by "let's return to chatting" or
"here comes another batch of data (presumably to be saved in another file)".

There is nothing wrong with this. Mercury need not deal with this.
This is the job of some more intelligent software to be used
instead of raw `nc`.

In tests, we have successfully used ZMODEM here. This is probably overkill,
but it worked. Details, if you are interested:

- We used `sz` and `rz` from the `lrzsz` package on both Linux and Mac.
- On Linux, `--tcp-client 127.0.0.1 8301` connect either of the two with
  the data port.
- On Mac, `sz` / `rz` didn't provide that switch. So we pulled `socat`
  out of our toolbox, which worked nicely (with `EXEC:` addresses, see its
  manual page).
- `--disable-timeouts` was essential.
- `-b` worked (needed on the `sz` side), quite as we expected.
- We initially used `-L 42` (on the `sz` side), but should probably have used
  much larger windows.
- `-v -v` keeps you more entertained and informed what's going on.

## Mercury hints

I have found the `alsa` sound access hard to use, but the `pulse`
interface works nicely on my Debian Trixie machine (that actually
uses pipewire audio).

Mercury does not use FSK like most ham radio digital modes do.
(The FSK crowd includes FT-8, JS8, Olivia, RTTY, and more).
Mercury uses OFDM, which is much more sensitive to overloading and
likely to result in splatter. You should certainly not see even a hint
of AGN action with the audio level you use. But even so, the
audio level may still be too high. Once a connection has been established,
you can try to reduce the tx audio (by 2 dB or so) via `mercury-ui`,
and observe whether the SNR reported by the other station actually improves.

For first experiments, I put a `mercury.ini` into the directory in which I
intend to start the `marcury-ui` (or plain `mercury`) binary.
https://github.com/Rhizomatica/mercury/blob/mercuryv2/mercury.ini.example
gives you something to start from.

My first experiments did not actually use my transceiver, but just listen
to the noise that comes out of my laptop's speakers resp. feed such
noise into my laptop's microphone. For this, this is my "acoustic couple"
`mercury.ini`:

```
[main]
ui_enabled = true
ui_port = 10000
ui_protocol = ws
waterfall_enabled = true
radio_model = 6
radio_device = ""
input_device = "alsa_input.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__Mic1__source"
output_device = "alsa_output.pci-0000_00_1f.3-platform-skl_hda_dsp_generic.HiFi__Speaker__sink"
capture_channel = stereo
sound_system = pulse
arq_tcp_base_port = 8300
broadcast_tcp_port = 8100
verbose = true
freedv_verbosity = 1
hamlib_log_level = 3
radio_serial_speed = 0

[arq]
no_progress_timeout_s = 180
disconnect_drain_timeout_s = 30
data_retry_slots = 10
mode_hold_after_downgrade_s = 6
ladder_up_successes = 2
retry_downgrade_threshold = 2
channel_guard_ms = 700
iss_post_ack_guard_ms = 900
keepalive_interval_s = 20
keepalive_miss_limit = 5
peer_payload_hold_s = 15
startup_max_s = 10

[channel]
busy_detect = false
busy_threshold_db = 10
busy_hysteresis_db = 3
busy_on_debounce_ms = 300
busy_hang_ms = 1500

[audio]
tx_gain_db = -8.50

[tnc]
keepalive_s = 60
buffer_report_ms = 1000
```

You will probably want to set your own values for `input_device`,
`output_device`, and `tx_gain_db`.

## An actual first experiment that does not require a QSO partner

My actual first experiment was: With no trx connected,

- send `MYCALL` and then `CONNECT` commands, trying to connect
  to a fake call,
- record the noise with my smartphone,
- switch to the fake call using `MYCALL`,
- let the smartphone play back the sound to the laptop's microphone.

Sample cuts of the four terminals:

Terminal 1:

```
$ ducobelt 
> mycall dj3ei
> connect dj3ei dl1ejw
> mycall dl1ejw
> listen on
> 
```

Terminal 2, the head is missing, this starts when
the smartphone sent the recording:

```
SN 1.6                                                                                             │andreas@uhu:~
BITRATE (3) 321 BPS                                                                                │$ nc -v -v localhost 8301
PENDING                                                                                            │nc: connect to localhost (::1) port 8301 (tcp) failed: Connection refused
PTT ON                                                                                             │Connection to localhost (127.0.0.1) 8301 port [tcp/*] succeeded!
PTT OFF                                                                                            │
SN 5.8                                                                                             │
BITRATE (15) 68 BPS                                                                                │
PTT ON                                                                                             │
PTT OFF                                                                                            │
SN 3.4                                                                                             │
BITRATE (15) 68 BPS                                                                                │
PTT ON                                                                                             │
PTT OFF                                                                                            │
PTT ON                                                                                             │
PTT OFF                                                                                            │
PTT ON                                                                                             │
IAMALIVE                                                                                           │
PTT OFF                                                                                            │
PTT ON                                                                                             │
PTT OFF                                                                                            │
CANCELPENDING                                                                                      │
IAMALIVE
```

And here is the last part of the log of `mercury-ui`, slightly earlier,
starting with the moment when `listen on` was typed into terminal 1
(and converted to upper case `LISTEN ON` by ducobelt).

```
18:11:28.896 [+119.274s] [INF] [tcp-ctl] Command received: LISTEN ON
18:11:39.865 [+130.243s] [INF] [arq] Incoming connection from DJ3EI on DL1EJW (pending)
18:11:40.608 [+130.986s] [INF] [radio] TX enabled (PTT ON)
18:11:44.318 [+134.696s] [INF] [radio] TX disabled (PTT OFF)
18:11:53.352 [+143.730s] [INF] [radio] TX enabled (PTT ON)
18:11:57.063 [+147.441s] [INF] [radio] TX disabled (PTT OFF)
18:12:06.096 [+156.474s] [INF] [radio] TX enabled (PTT ON)
18:12:09.806 [+160.185s] [INF] [radio] TX disabled (PTT OFF)
18:12:18.837 [+169.216s] [INF] [radio] TX enabled (PTT ON)
18:12:22.548 [+172.926s] [INF] [radio] TX disabled (PTT OFF)
18:12:31.548 [+181.926s] [INF] [arq] Incoming connection cancelled
```

The mercury program sends out four times its affirmative answer
to the incoming audio (from the smart phone recording) that
offered an incoming connection originating from DJ3EI.
