Metadata-Version: 2.4
Name: marla-agents
Version: 0.1.0
Summary: MARLA: Multi-Agent Reinforcement Learning Architecture for NASimEmu
Project-URL: Homepage, https://github.com/FranEnguix/marla
Project-URL: Repository, https://github.com/FranEnguix/marla
Project-URL: Issues, https://github.com/FranEnguix/marla/issues
Project-URL: Documentation, https://marla.readthedocs.io
Author-email: Fran Enguix <enguix.fco@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: large-language-models,multi-agent,nasimemu,offensive-security,ppo,reinforcement-learning
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Requires-Python: <3.11,>=3.10
Requires-Dist: matplotlib
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: spade>=4.0
Requires-Dist: torch
Requires-Dist: torch-geometric
Requires-Dist: typer>=0.12
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: myst-parser>=2.0; extra == 'docs'
Requires-Dist: sphinx-rtd-theme>=2.0; extra == 'docs'
Requires-Dist: sphinx>=7.0; extra == 'docs'
Provides-Extra: local-lm
Requires-Dist: accelerate>=0.30; extra == 'local-lm'
Requires-Dist: transformers>=4.40; extra == 'local-lm'
Description-Content-Type: text/markdown

# Multi-Agent Reinforcement Learning Architecture for offensive AI (MARLA)

MARLA is a research-oriented offensive AI platform for simulated environments. It is designed to operate in NASimEmu scenarios and to study how a centralized reinforcement learning attacker can be augmented by advisory agents with specialized reasoning roles.

Concretely: a reinforcement-learning cyber agent (recurrent PPO over NASimEmu) that can
improve its decisions by selectively consulting an external, frozen,
language-model-based Plan Maker through a schema-validating Gatekeeper.

See `docs/` for the full documentation (installation, configuration
reference, architecture, CLI, and metrics/plots) -- build it locally with
`pip install -e ".[docs]"` and `sphinx-build -b html docs docs/_build/html`.

## Install

```bash
pip install -e ".[dev]"
```

## CLI

```bash
marla --help
marla validate experiment.yaml
marla run experiment.yaml
marla summarize runs/<experiment-name>/<run-id>
marla version
python -m marla --help
```

## Known limitation: embedded XMPP server flakiness in assisted mode

`execution: local` runs use SPADE's built-in embedded XMPP server
(`pyjabber`) so `marla run` works with zero setup -- `xmpp.server` can just
be `localhost`. For the **baseline** variant (no Gatekeeper/Plan Maker) this
is fully reliable since there is no presence-subscription traffic at all.

For the **assisted** variant, `pyjabber` has an observed race condition in
its roster/presence-subscription handling: with 3+ agents connecting and
subscribing to each other's presence, roughly 1-in-4 runs raise an
unhandled `sqlite`/`asyncio` error during startup or (less harmfully)
during shutdown after the run's actual result was already produced. This is
a `pyjabber` robustness issue, not a MARLA correctness issue -- but for long
real research runs where a crash mid-training would be costly, point
`xmpp.server` at a real, separately-deployed XMPP server (e.g. Prosody or
ejabberd) instead of relying on the embedded one. Distributed mode already
requires a real reachable XMPP server, so this only matters for assisted
*local* runs.

The same `pyjabber` race shows up in **distributed** mode too, and more
reliably, when run as a standalone (non-embedded) server: the full
multi-agent handshake needs two concurrent presence subscriptions
(Gatekeeper -> Orchestrator, Plan Maker -> Gatekeeper) where the
single-agent baseline case needs none, and standalone `pyjabber` failed
this consistently in testing (see `tests/test_distributed.py`'s
`test_distributed_multiagent_completes`, marked `xfail` for this reason).
The full multi-agent distributed handshake -- READY_CHECK/READY/
START_EXPERIMENT/STOP_EXPERIMENT and real ADVISORY_REQUEST/ADVISORY_RESPONSE
round trips across separate processes -- was verified working end-to-end
against a properly configured Prosody server; use a real XMPP server for
distributed assisted runs, not standalone `pyjabber`.

## Fixed: keepalive/reconnect cascade during real Plan Maker inference

SPADE enables a XEP-0199 keepalive ping (every 55s) that **reconnects the
client if a ping times out**. The local Plan Maker backend runs
`generate()` synchronously on the shared event loop by design (see
`local_backend.py`'s module docstring: `asyncio.to_thread` reproducibly
hangs for this call in this environment) -- a real model doing real
inference against a real (and, as an episode progresses, growing) prompt
can block that loop long enough to miss a ping. The resulting reconnect
re-registers the agent (SPADE's default `auto_register=True`), and if the
loop is blocked *again* at that moment, registration itself times out,
crashing the run with an unrelated-looking `RegistrationException` --
this was the actual cause of a run appearing to hang or crash partway
through, not a Plan Maker or model bug. Fixed by disabling ping-triggered
reconnection for all agents (`agents/lifecycle_behaviours.py`'s
`disable_reconnect_on_missed_ping`): a genuinely dropped peer is still
caught by the presence-based disconnect detection used in distributed
mode, which doesn't depend on ping timing.
