Metadata-Version: 2.4
Name: markdown-pytest
Version: 0.6.4
Summary: Pytest plugin for runs tests directly from Markdown files
Project-URL: Source, https://github.com/mosquito/markdown-pytest
Project-URL: Tracker, https://github.com/mosquito/markdown-pytest/issues
Project-URL: Documentation, https://mosquito.github.io/markdown-pytest/
Author-email: Dmitry Orlov <me@mosquito.su>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: documentation,markdown,pytest
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Plugins
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Provides-Extra: async
Requires-Dist: pytest-asyncio<2,>=1; extra == 'async'
Description-Content-Type: text/markdown

markdown-pytest
===============

[![Github Actions](https://github.com/mosquito/markdown-pytest/workflows/tests/badge.svg)](https://github.com/mosquito/markdown-pytest/actions?query=workflow%3Atests)
[![PyPI Version](https://img.shields.io/pypi/v/markdown-pytest.svg)](https://pypi.python.org/pypi/markdown-pytest/)
[![PyPI Wheel](https://img.shields.io/pypi/wheel/markdown-pytest.svg)](https://pypi.python.org/pypi/markdown-pytest/)
[![Python Versions](https://img.shields.io/pypi/pyversions/markdown-pytest.svg)](https://pypi.python.org/pypi/markdown-pytest/)
[![License](https://img.shields.io/pypi/l/markdown-pytest.svg)](https://pypi.python.org/pypi/markdown-pytest/)

A `pytest` plugin that collects and executes Python code blocks from Markdown
files, so your documentation examples are always tested.

> **By the way:** this README is itself tested by `markdown-pytest`.
> Every Python code block below is a real test that runs on every CI build.
> The HTML comments that mark tests (like `<!-- name: ... -->`) are invisible
> in the rendered view — view the
> [raw Markdown source](https://raw.githubusercontent.com/mosquito/markdown-pytest/master/README.md)
> to see them.

Quick start
-----------

Install with pip:

    pip install markdown-pytest

Or with asyncio support:

    pip install markdown-pytest[async]

Place an HTML comment with a `name` key directly above a `python` code
fence. The plugin collects it as a test. In your `.md` file write this:

``````
<!-- name: test_quick_start -->
```python
assert 2 + 2 == 4
```
``````

When rendered, the HTML comment becomes invisible — readers see only a
clean code block:

<!-- name: test_quick_start -->
```python
assert 2 + 2 == 4
```

Run it:

    $ pytest -v README.md

That is the only requirement. Everything below is optional and lets you
handle progressively more complex scenarios.

Code split
----------

You can split a test across several code blocks by giving them the same
`name`. The blocks are combined into a single test, preserving source line
numbers for accurate tracebacks. In the raw Markdown it looks like this:

``````
<!-- name: test_example -->
```python
from itertools import chain
```

Some explanatory prose in between...

<!-- name: test_example -->
```python
assert list(chain(range(2), range(2))) == [0, 1, 0, 1]
```
``````

Here is a live example. This block performs import:

<!-- name: test_example -->
```python
from itertools import chain
```

`chain` usage example:

<!-- name: test_example -->
```python
assert list(chain(range(2), range(2))) == [0, 1, 0, 1]
```

The split blocks do not need to be consecutive. Blocks with the same name
are combined even when separated by other tests. In the example below,
`test_non_consecutive_a` is defined in two blocks with a completely
unrelated test in between — the plugin combines only the matching blocks:

<!-- name: test_non_consecutive_a -->
```python
value = 42
```

<!-- name: test_non_consecutive_helper -->
```python
assert True
```

<!-- name: test_non_consecutive_a -->
```python
assert value == 42
```

Subtests
--------

Add `case: case_name` to run a block as a subtest. Shared setup code goes
in a block without a `case`, and each subsequent `case` block runs as an
independent subtest. In the raw Markdown:

``````
<!-- name: test_counter -->
```python
from collections import Counter
```

<!-- name: test_counter; case: initialize_counter -->
```python
counter = Counter()
```
``````

Live example:

<!-- name: test_counter -->
```python
from collections import Counter
```

<!--
    name: test_counter;
    case: initialize_counter
-->
```python
counter = Counter()
```

<!--
    name: test_counter;
    case: assign_counter_value
-->
```python
counter["foo"] += 1

assert counter["foo"] == 1
```

Subtests support is built into pytest 9.0+ and requires no extra packages.

Fixtures
--------

You can request pytest fixtures by adding `fixtures: name1, name2` to the
comment. Any standard pytest fixture (`tmp_path`, `monkeypatch`, `capsys`,
`request`, etc.) or custom fixtures defined in `conftest.py` can be used.
The requested fixtures are available as variables in the code block.

In the raw Markdown:

``````
<!-- name: test_with_tmp_path; fixtures: tmp_path -->
```python
p = tmp_path / "hello.txt"
p.write_text("hello")
assert p.read_text() == "hello"
```
``````

### Single fixture

``````
<!-- name: test_with_tmp_path; fixtures: tmp_path -->
```python
p = tmp_path / "hello.txt"
p.write_text("hello")
assert p.read_text() == "hello"
```
``````

<!-- name: test_with_tmp_path; fixtures: tmp_path -->
```python
p = tmp_path / "hello.txt"
p.write_text("hello")
assert p.read_text() == "hello"
```

### Multiple fixtures

``````
<!-- name: test_multi_fixtures; fixtures: tmp_path, monkeypatch -->
```python
import os
monkeypatch.setenv("DATA_DIR", str(tmp_path))
assert os.environ["DATA_DIR"] == str(tmp_path)
```
``````

<!-- name: test_multi_fixtures; fixtures: tmp_path, monkeypatch -->
```python
import os
monkeypatch.setenv("DATA_DIR", str(tmp_path))
assert os.environ["DATA_DIR"] == str(tmp_path)
```

Fixture lists can also span multiple lines (see
[Comment syntax](#comment-syntax) for all supported formats):

``````
<!--
    name: test_multiline_fixtures;
    fixtures: tmp_path,
              monkeypatch
-->
```python
import os
monkeypatch.setenv("ML_DIR", str(tmp_path))
assert os.environ["ML_DIR"] == str(tmp_path)
```
``````

<!--
    name: test_multiline_fixtures;
    fixtures: tmp_path,
              monkeypatch
-->
```python
import os
monkeypatch.setenv("ML_DIR", str(tmp_path))
assert os.environ["ML_DIR"] == str(tmp_path)
```

``````
<!--
    name: test_separate_fixtures;
    fixtures: tmp_path;
    fixtures: capsys
-->
```python
p = tmp_path / "out.txt"
p.write_text("ok")
print(p.read_text())
captured = capsys.readouterr()
assert captured.out.strip() == "ok"
```
``````

<!--
    name: test_separate_fixtures;
    fixtures: tmp_path;
    fixtures: capsys
-->
```python
p = tmp_path / "out.txt"
p.write_text("ok")
print(p.read_text())
captured = capsys.readouterr()
assert captured.out.strip() == "ok"
```

### Fixtures with split blocks

Only the first block needs the `fixtures:` declaration. All blocks with the
same name share the namespace:

``````
<!-- name: test_split_fixtures; fixtures: tmp_path -->
```python
p = tmp_path / "data.txt"
p.write_text("hello")
```

<!-- name: test_split_fixtures -->
```python
assert p.read_text() == "hello"
```
``````

<!-- name: test_split_fixtures; fixtures: tmp_path -->
```python
p = tmp_path / "data.txt"
p.write_text("hello")
```

<!-- name: test_split_fixtures -->
```python
assert p.read_text() == "hello"
```

You can also declare different fixtures in different blocks — they are
merged together:

``````
<!-- name: test_merged_fixtures; fixtures: tmp_path -->
```python
p = tmp_path / "output.txt"
p.write_text("merged")
```

<!-- name: test_merged_fixtures; fixtures: capsys -->
```python
print(p.read_text())
captured = capsys.readouterr()
assert captured.out.strip() == "merged"
```
``````

<!-- name: test_merged_fixtures; fixtures: tmp_path -->
```python
p = tmp_path / "output.txt"
p.write_text("merged")
```

<!-- name: test_merged_fixtures; fixtures: capsys -->
```python
print(p.read_text())
captured = capsys.readouterr()
assert captured.out.strip() == "merged"
```

### Fixtures with subtests

Fixtures and subtests (`case:`) can be freely combined:

``````
<!-- name: test_fixture_cases; fixtures: tmp_path -->
```python
data_file = tmp_path / "data.txt"
```

<!-- name: test_fixture_cases; case: write -->
```python
data_file.write_text("hello world")
assert data_file.exists()
```

<!-- name: test_fixture_cases; case: read back -->
```python
assert data_file.read_text() == "hello world"
```
``````

<!-- name: test_fixture_cases; fixtures: tmp_path -->
```python
data_file = tmp_path / "data.txt"
```

<!-- name: test_fixture_cases; case: write -->
```python
data_file.write_text("hello world")
assert data_file.exists()
```

<!-- name: test_fixture_cases; case: read back -->
```python
assert data_file.read_text() == "hello world"
```

Hidden code blocks
------------------

A code block placed *inside* the comment is invisible to readers but runs
before the visible block. This is useful for imports, boilerplate, or test
data preparation. In the raw Markdown it looks like this:

``````
<!--
name: test_hidden_init
```python
init_value = 123
```
-->
```python
assert init_value == 123
```
``````

When rendered, readers see only `assert init_value == 123` — the
assignment is hidden in the HTML comment. Here is the live example:

<!--
name: test_hidden_init
```python
init_value = 123
```
-->
```python
assert init_value == 123
```

### Hidden setup, visible demo, hidden assertions

You can combine hidden blocks and split blocks to create documentation that
reads like a tutorial: the setup and assertions are invisible, and the reader
sees only the interesting part. This is the **recommended pattern** for
polished documentation examples.

The following example demonstrates a CSV parser. In the raw Markdown the
setup and assertions are inside HTML comments:

``````
<!--
name: test_hidden_demo;
fixtures: tmp_path
```python
csv_file = tmp_path / "users.csv"
csv_file.write_text("name,role\nAlice,admin\nBob,viewer\n")
```
-->
```python
rows = []
for line in csv_file.read_text().strip().split("\n")[1:]:
    name, role = line.split(",")
    rows.append({"name": name, "role": role})
```
<!--
name: test_hidden_demo
```python
assert rows[0] == {"name": "Alice", "role": "admin"}
```
-->
``````

When rendered, readers see only the `for` loop — file creation and
assertions are both hidden:

<!--
name: test_hidden_demo;
fixtures: tmp_path
```python
csv_file = tmp_path / "users.csv"
csv_file.write_text("name,role\nAlice,admin\nBob,viewer\n")
```
-->
```python
rows = []
for line in csv_file.read_text().strip().split("\n")[1:]:
    name, role = line.split(",")
    rows.append({"name": name, "role": role})
```

<!--
name: test_hidden_demo
```python
assert len(rows) == 2
assert rows[0] == {"name": "Alice", "role": "admin"}
assert rows[1] == {"name": "Bob", "role": "viewer"}
```
-->

Subprocess mode
---------------

Add `subprocess: true` to run a test in its own Python subprocess instead
of the main pytest process. This is useful when the code under test
modifies global state, calls `os.exit()`, or needs full process isolation.

``````
<!-- name: test_isolated; subprocess: true -->
```python
import sys
sys.modules["__demo_marker__"] = True
assert "__demo_marker__" in sys.modules
```
``````

<!-- name: test_isolated; subprocess: true -->
```python
import sys
sys.modules["__demo_marker__"] = True
assert "__demo_marker__" in sys.modules
```

Subprocess tests support split blocks — multiple blocks with the same name
are combined before being executed in a single subprocess:

``````
<!-- name: test_sub_split; subprocess: true -->
```python
data = {"key": "value"}
```

<!-- name: test_sub_split; subprocess: true -->
```python
assert data["key"] == "value"
```
``````

<!-- name: test_sub_split; subprocess: true -->
```python
data = {"key": "value"}
```

<!-- name: test_sub_split; subprocess: true -->
```python
assert data["key"] == "value"
```

Hidden blocks also work with subprocess mode:

``````
<!--
name: test_sub_hidden;
subprocess: true
```python
_setup_value = 99
```
-->
```python
assert _setup_value == 99
```
``````

<!--
name: test_sub_hidden;
subprocess: true
```python
_setup_value = 99
```
-->
```python
assert _setup_value == 99
```

> **Note:** subprocess tests cannot use pytest fixtures or subtests — those
> features require the in-process test runner. If a test needs fixtures,
> omit `subprocess: true`.

Asyncio
-----------

Prefix the test name with `async ` to enable top-level `await` inside the
code block and use async fixtures.

Requires an async pytest plugin to manage the event loop — `pytest-asyncio`,
`aiomisc-pytest`, or any other plugin that can run `async def` test functions.
Without such a plugin, async tests fail immediately with a message listing
available options. Install `pytest-asyncio` via
`pip install markdown-pytest[async]`.

``````
<!-- name: async test_async -->
```python
import asyncio
await asyncio.sleep(0)
assert True
```
``````

Live example:

<!-- name: async test_async_quick_start -->
```python
import asyncio
await asyncio.sleep(0)
assert True
```

When a test is split across multiple blocks, only one declaration
needs the `async ` prefix — the whole test is then treated as async:

``````
<!-- name: async test_async_split -->
```python
import asyncio

async def double(x):
    await asyncio.sleep(0)
    return x * 2
```

<!-- name: test_async_split -->
```python
assert await double(21) == 42
```
``````

<!-- name: async test_async_split -->
```python
import asyncio

async def double(x):
    return x * 2
```

<!-- name: test_async_split -->
```python
assert await double(21) == 42
```

Async tests work with fixtures, subtests (`case:`), marks and
`subprocess: true`. In subprocess mode the collected source is wrapped in
`async def __amain(): ...; asyncio.run(__amain())` before being executed
in the child process.


Doctest / REPL
--------------

Add `repl: true` to run a block as a
[doctest](https://docs.python.org/3/library/doctest.html). Lines starting
with `>>>` (or `...` for continuation) are the code; lines that follow
without a prompt are the expected output. This is Python's standard
interactive shell format, rendered as normal syntax-highlighted Python in
every Markdown viewer.

``````
<!-- name: test_repl_greet; repl: true -->
```python
>>> def greet(name):
...     return f"Hello, {name}!"
>>> greet("world")
'Hello, world!'
>>> print(greet("pytest"))
Hello, pytest!
```
``````

Live example:

<!-- name: test_repl_greet; repl: true -->
```python
>>> def greet(name):
...     return f"Hello, {name}!"
>>> greet("world")
'Hello, world!'
>>> print(greet("pytest"))
Hello, pytest!
```

All standard doctest directives work — `# doctest: +ELLIPSIS`,
`# doctest: +NORMALIZE_WHITESPACE`, `# doctest: +SKIP`, and so on:

``````
<!-- name: test_repl_ellipsis; repl: true -->
```python
>>> [1, 2, 3, 4, 5]  # doctest: +ELLIPSIS
[1, 2, ...]
```
``````

<!-- name: test_repl_ellipsis; repl: true -->
```python
>>> [1, 2, 3, 4, 5]  # doctest: +ELLIPSIS
[1, 2, ...]
```

Split blocks work the same way as with regular tests — multiple blocks
with the same name share a session, so variables defined in an earlier
block are available in later ones:

``````
<!-- name: test_repl_split_demo; repl: true -->
```python
>>> items = [1, 2, 3]
```

Some prose in between...

<!-- name: test_repl_split_demo; repl: true -->
```python
>>> items.append(4)
>>> items
[1, 2, 3, 4]
```
``````

<!-- name: test_repl_split_demo; repl: true -->
```python
>>> items = [1, 2, 3]
```

<!-- name: test_repl_split_demo; repl: true -->
```python
>>> items.append(4)
>>> items
[1, 2, 3, 4]
```

### Async doctest

Prefix the name with `async ` to enable top-level `await` inside a REPL
block — the same prefix used for regular async tests. State is preserved
across all examples within the same event loop:

``````
<!-- name: async test_repl_async; repl: true -->
```python
>>> import asyncio
>>> async def fetch():
...     await asyncio.sleep(0)
...     return 42
>>> await fetch()
42
```
``````

<!-- name: async test_repl_async; repl: true -->
```python
>>> import asyncio
>>> async def fetch():
...     await asyncio.sleep(0)
...     return 42
>>> await fetch()
42
```

> **Note:** `subprocess: true` and `repl: true` cannot be combined.

> **Note:** sync REPL blocks (no `async ` prefix) work without any extra
> dependencies. Async REPL blocks require an async pytest plugin to manage
> the event loop — `pytest-asyncio`, `aiomisc-pytest`, or any other plugin
> that can run `async def` test functions. Without such a plugin, async REPL
> tests fail immediately with a clear message listing available options.
> Install `pytest-asyncio` via `pip install markdown-pytest[async]`.

Marks
-----

Add `mark: <expression>` to apply any
[pytest mark](https://docs.pytest.org/en/stable/how-to/mark.html) to a
test. The expression is evaluated as `pytest.mark.<expression>`.

### Expected failure

``````
<!-- name: test_divide_by_zero; mark: xfail(raises=ZeroDivisionError) -->
```python
1 / 0
```
``````

<!-- name: test_divide_by_zero; mark: xfail(raises=ZeroDivisionError) -->
```python
1 / 0
```

### xfail with reason

``````
<!-- name: test_xfail_reason; mark: xfail(reason="not implemented yet") -->
```python
assert False, "not implemented"
```
``````

<!-- name: test_xfail_reason; mark: xfail(reason="not implemented yet") -->
```python
assert False, "not implemented"
```

### Skip a test

``````
<!-- name: test_skipped; mark: skip(reason="requires network") -->
```python
import urllib.request
urllib.request.urlopen("http://localhost:99999")
```
``````

<!-- name: test_skipped; mark: skip(reason="requires network") -->
```python
import urllib.request
urllib.request.urlopen("http://localhost:99999")
```

### Marks with split blocks

Only the first block needs the `mark:` declaration:

``````
<!-- name: test_mark_split_demo; mark: xfail -->
```python
x = 1
```

<!-- name: test_mark_split_demo -->
```python
assert x == 2, "expected to fail"
```
``````

<!-- name: test_mark_split_demo; mark: xfail -->
```python
x = 1
```

<!-- name: test_mark_split_demo -->
```python
assert x == 2, "expected to fail"
```

Comment syntax
--------------

The comment format uses colon-separated key-value pairs, separated by
semicolons. The trailing semicolon is optional:

    <!-- key1: value1; key2: value2 -->

Comments can span multiple lines:

    <!--
        name: test_name;
        fixtures: tmp_path, monkeypatch
    -->

Both two-dash and three-dash variants are supported. All of the following
are parsed identically:

    <!--  name: test_name -->
    <!--- name: test_name --->
    <!--  name: test_name --->
    <!--- name: test_name -->

Available comment parameters:

* `name` (required) — the test name. Must start with `test` by default
  (see [Configuration](#configuration) to change the prefix). Prefix the
  name with `async ` (e.g. `name: async test_foo`) to enable top-level
  `await` in the block — see [Async tests](#async-tests).
* `case` — marks the block as a subtest (see [Subtests](#subtests)).
* `fixtures` — comma-separated list of pytest fixtures to inject
  (see [Fixtures](#fixtures)).
* `subprocess` — set to `true` to run the test in a separate Python
  process (see [Subprocess mode](#subprocess-mode)).
* `repl` — set to `true` to run the block as a doctest using Python's
  interactive shell format (see [Doctest / REPL](#doctest--repl)).
* `mark` — a pytest mark expression to apply to the test
  (see [Marks](#marks)). Examples: `xfail`, `skip(reason="...")`,
  `xfail(raises=ZeroDivisionError)`.

Fixture lists can be written in several ways:

    <!-- name: test_foo; fixtures: tmp_path, monkeypatch, capsys -->

    <!--
        name: test_foo;
        fixtures: tmp_path,
                  monkeypatch,
                  capsys
    -->

    <!--
        name: test_foo;
        fixtures: tmp_path;
        fixtures: monkeypatch;
        fixtures: capsys
    -->

Values for duplicate keys are merged automatically.

Code blocks without a `name` comment are silently ignored — regular
documentation examples keep working as before.

Mixing with other languages
---------------------------

Markdown files often contain non-Python code blocks. The plugin safely
skips any fenced block that is not tagged as `python` — including
` ```bash `, ` ```json `, ` ```yaml `, bare ` ``` ` blocks, and
four-backtick (```` ```` ````) fences:

``````
```bash
echo "this is ignored by markdown-pytest"
```

```json
{"this": "is also ignored"}
```

```
Bare fences without a language tag are ignored too.
```
``````

Only blocks explicitly tagged ` ```python ` and preceded by a
`<!-- name: ... -->` comment are collected as tests.

Indented code blocks
--------------------

Code blocks inside HTML elements like `<details>` may be indented.
The plugin strips the leading indentation automatically:

<details>
<summary>Indented example</summary>

    <!-- name: test_indented -->
    ```python
    assert True
    ```

</details>

Configuration
-------------

### Test prefix

By default only blocks whose `name` starts with `test` are collected.
Use `--md-prefix` to change the prefix:

    $ pytest --md-prefix=check README.md

The prefix can also be set permanently in `pyproject.toml`:

    [tool.pytest.ini_options]
    addopts = "--md-prefix=check"

Or in `pytest.ini` / `setup.cfg`:

    [pytest]
    addopts = --md-prefix=check

Supported environments
----------------------

* **Python** >= 3.10 (CPython and PyPy)
* Both `.md` and `.markdown` file extensions are collected automatically
* The plugin auto-registers via the `pytest11` entry point — no
  configuration is needed beyond `pip install`
* Tracebacks from failing tests preserve the original Markdown line numbers,
  so you can jump straight to the source

Usage example
-------------

This README.md file can be tested like this:

    $ pytest -v README.md

Sample output:

    README.md::test_quick_start PASSED
    README.md::test_example PASSED
    README.md::test_counter PASSED
    README.md::test_with_tmp_path PASSED
    README.md::test_hidden_demo PASSED
    README.md::test_isolated PASSED
    README.md::test_sub_split PASSED
    README.md::test_sub_hidden PASSED
