Metadata-Version: 2.4
Name: fdu
Version: 0.6.0
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Classifier: Topic :: System :: Filesystems
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
License-File: LICENSE
Summary: Fast du replacement, with .gitignore-aware sizes and code and document counts, for the command line, Python, and Rust
Keywords: disk-usage,filesystem,rollup,rust,cli
Author: Joshua Levy
License-Expression: MIT
Requires-Python: >=3.12
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://github.com/jlevy/fdu/blob/main/docs/usage.md
Project-URL: Repository, https://github.com/jlevy/fdu

# fdu (Python)

**Fast du replacement, with `.gitignore`-aware sizes and code and document counts, for
the command line, Python, and Rust**

Python bindings for [fdu](https://github.com/jlevy/fdu), which reports, for every
directory in a tree at once, its size, file count, recency, and file kinds, and on
request lines of code by language and words by document type.
It walks a tree on several threads through each platform’s native directory interface;
[performance measurements](https://github.com/jlevy/fdu/blob/main/docs/performance-measurements.md)
compare it with other tools by build, platform, and tree.

## Set Up with Any Coding Agent

Give the agent this instruction:

> Run `uvx --no-build fdu@latest --install-skill` from the project root to install fdu’s
> self-contained skill for current and future agent sessions.

The skill needs no prior session context or installed command; `fdu --skill` prints it.

## Install the Command Line

```shell
uvx --no-build fdu@latest .
uv tool install --no-build fdu
```

The first command runs fdu once; the second keeps it on `PATH`.
`uv tool upgrade --no-build fdu` updates that install;
`uvx --no-build fdu@<version> --help` runs one exact release.
The [repository README](https://github.com/jlevy/fdu#install-the-command-line) has the
details. Prebuilt `abi3` wheels cover GIL-enabled CPython 3.12 and newer on Linux glibc
(x86-64 and arm64), macOS (x86-64 and arm64), and Windows x86-64, so installing needs no
Rust toolchain. Free-threaded CPython, such as `3.14t`, cannot install them and is not
supported; an installer there may fall back to building the source distribution.
`--no-build` makes uv fail instead of compiling when no compatible wheel exists.
If uv selects a free-threaded interpreter, pass `--python 3.14`. An `exclude-newer`
policy in uv can filter a newly published fdu release.
Review and allow the first-party `fdu` package in that policy, or wait for its cool-off
to expire.

## Use as a Python Library

```shell
uv add fdu
pip install fdu
```

Choose among three entry-point groups over the same Rust engine:

| API | Choose it for |
| --- | --- |
| Report API: `fdu.report()` | One answer per invocation or job, from a summary to a detailed report |
| Index API: `fdu.open()` / `fdu.scan()` | Prepare once, reuse queries, and control refresh; supports content analysis and cache policy on open |
| Progressive API: `fdu.opened.OpenedIndex.open()` | Read during discovery, consume bounded pages, or follow changes in offline jobs or interactive applications |

For a standalone final metadata summary, use an explicit summary request:

```python
from pathlib import Path

import fdu

report = fdu.report(Path("."), fdu.Query(views=(fdu.View.SUMMARY,)))
```

Under the default cache policy, a sole unfiltered metadata summary over the full
population without analyzers can use a compact native reduction without a per-file
index. Filters, analysis, or snapshot requirements can need retained index state.
Query an appropriate existing handle instead of rescanning for its summary.

`Index` and `OpenedIndex` both retain inventory in Rust.
`Index` returns after requested preparation and leaves refresh to the caller; status
records incomplete coverage.
An `Index` returned by `fdu.open(..., stale_ok=True)` cannot refresh or start a watch;
reopen with `stale_ok=False` for filesystem verification and subsequent refresh.
`OpenedIndex` allows reads during discovery and can own background observation when
requested. Offline jobs can leave `observe=False` (the default), wait for required
coverage, inspect partial/limit outcomes, and consume and release pages.
An early partial summary is not a final whole-tree answer; discovery can invalidate
continuations.
Query-bound `Index.watch()` is a separate convenience: it copies the index
into an independent fixed-query session whose iterator applies updates and drives
persistence.

Keep per-file discovery, analysis, reconciliation, observation, queries, and shutdown in
Rust. Python should send coarse requests and receive selected answers.
Choose detail separately: an `Index` total can be small, and an opened report can be
large and eager.
Complete exports support custom Python processing at allocation and copy
cost; native queries avoid exporting input rows but still require engine work.
Paging adds calls and cursor handling while reducing each response and potentially peak
retention if pages are released; total conversion and version invalidation still apply.
The
[surface guide](https://github.com/jlevy/fdu/blob/main/docs/project/architecture/fdu-surface-architecture.md#choosing-an-api-for-a-new-application)
explains lifecycle choices and costs; performance depends on the measured workload.

Lines of code by language and words by document type, from one scan:

```python
from pathlib import Path

import fdu

report = fdu.report(
    Path("/path/to/tree"),
    fdu.Query(views=(fdu.View.CODE, fdu.View.DOCUMENTS)),
)
print(report.render())
print(report.notes)
```

The two content views run the `code` and `words` analyzers they show; `print` gives the
same tables as `fdu . --view=code,documents`, and `report.sections` holds them as typed
values.

`fdu.report` also takes a sequence of disjoint paths, reported as one, as `fdu docs src`
does: every size, row, share, and bound is over the union.
`report.root` is then `None` and `report.roots` names each root
(`ReportRoot(label, path)`); file rows, status errors, and refused `.gitignore` files
carry `root`, the position of their root in `roots`, so
`report.roots[row.root].path / row.path` names the entry; and a tree section has a
`total` and one `RootTree` per root.
A `str` or `Path` is always one root.
`fdu.open` and `fdu.scan` take one root.

The public package is `fdu`; `fdu._native` is private build machinery.
The supported API includes typed query and scan options, immutable report sections,
roll-ups, per-path provenance, cache management, refresh results, and change feeds.
`.gitignore` is read by default: tree, summary, extension, and file rows carry their
ignored share, `Selection(ignored=...)` selects one side, and `Status.ignore_rules`
names any file the limits refused.
`ScanOptions` carries the switch (`read_controls`) and both limits (`control_budget`,
`control_line_limit`). Cache status is typed as `CacheState`, with `StaleReason` and
`LeftoverKind`, and `clear_all_caches` returns a `ClearSummary`. Filesystem failures
that prevent an operation from starting remain exceptions.
Errors that make a scan partial remain structured data on `Status`, so callers can use
the covered result without losing the reason it is incomplete.
The
[roll-up adapter example](https://github.com/jlevy/fdu/blob/main/crates/fdu-py/examples/rollup_adapter.py)
shows one scan serving several application-owned summaries without parsing terminal
output or adopting fdu’s machine schema as the application’s internal model.

```python
from pathlib import Path

import fdu

index = fdu.open(
    Path("/path/to/tree"),
    cache=fdu.CachePolicy.AUTO,
    scan=fdu.ScanOptions(one_filesystem=True),
    analysis=fdu.AnalysisOptions(analyze=fdu.Analysis.ALL),
)
print(index.status.complete)
print(index.total().by_extension)

report = index.report(
    fdu.Query(
        views=(fdu.View.TYPES, fdu.View.FAMILIES, fdu.View.DOCUMENTS),
        selection=fdu.Selection(limit=20, size=fdu.SizeMetric.APPARENT),
    )
)
print(report.provenance.freshness)
print(report.sections)
print(report.as_dict())

mark = index.clock
result = index.refresh()
print(result.status, index.since(mark).changes)
```

The index API (`Index`) returns whole selected results in bulk while retaining the
inventory in Rust. The opened API below also supports continuation-based paging.
Open, scan, and the native reconciliation phase of refresh run with the GIL released, so
unrelated Python threads and independent indexes can progress.
So does a report’s capacity observation, only when it asks for `View.VOLUMES`; every
other `Index.report` holds the GIL. Content analysis streams every eligible file through
EOF. Binary data, invalid UTF-8, and unsupported SLOC languages remain visible as
coverage without making the operation partial; I/O failures and files changed during a
read remain operational errors.
One `Index` object still has `PyO3` runtime borrow exclusion: an overlapping call on
that same object is rejected rather than becoming an unsynchronized shared-index access.
The wheel enables the optional watch dependency and exposes `Index.watch()` as a
closable, query-bound change feed.
It copies the index into an independent session; iteration verifies and applies changes
there and drives persistence.
`Watch.report()` returns its latest applied answer without draining events or updating
the original `Index`. Native watch startup currently holds the GIL, while iteration
waits release it. Content analysis itself remains one-shot: refresh reanalyzes after
metadata reconciliation, while a watch feed reports metadata changes.

`Analysis` names the same analyzers as the command line’s `--analyze`: `lines`, `code`,
and `words`, with `none` and `all` as totals.
`AnalysisOptions` takes one of them or a comma-separated set such as `"code,words"`,
plus a worker count.
A content view requests the analysis it shows, as `--view code` does on the command
line: `fdu.report(root, fdu.Query(views=(fdu.View.CODE, fdu.View.DOCUMENTS)))` runs the
`code` and `words` analyzers without naming them.
An index holds exactly the analyzers it was opened with, so `index.report` never adds
one: the same view on an index opened without `words` raises `InvalidArgumentError`
naming the analyzer.
`AnalysisOptions` is the control for the rest: analysis shown in a metadata view, such
as code lines in `View.LANGUAGES`, and analysis retained for an index’s reports and
rerun by refresh. Typed report sections expose stable type/family groups, exact share
fractions, line and word slots, page denominators, coverage outcomes, analyzer
provenance, detection source and confidence, and generated/vendor/documentation flags.
The original extension grouping remains available as the `extensions` view.
The package supports Python 3.12 and newer and builds one `abi3-py312` extension rather
than separate native payloads for every Python minor release.

## Population, Code Overview, and Cache Destination

`fdu.report()` derives discovery and content analysis from `Selection.ignored`:

```python
import fdu

report = fdu.report(
    ".",
    fdu.Query(
        views=(fdu.View.CODE,),
        selection=fdu.Selection(ignored=fdu.IgnoredEntries.EXCLUDE),
    ),
    cache_dir="/path/to/cache",
)
```

`include` is the default.
`exclude` prunes safely ignored subtrees; `only` discovers ignored matches through
ordinary ancestors and analyzes only ignored bodies.
For retained work, `fdu.open(..., ignored=...)` and `fdu.scan(..., ignored=...)` choose
the initial population.
A default read inherits it.
An Include index can answer a narrower selection; an index that never retained a
population cannot widen its answer.

`View.CODE` runs code analysis and reports a `CodeSection`, the view code analysis also
defaults to, whose overview includes selected source lines, language shares,
ignored/non-ignored contributions, and coverage.
`SortKey.CODE_LINES` ranks files or directories when code analysis was requested.
File rows carry nullable `sort_value` and `classification` evidence; unavailable counts
remain distinct from zero.

`cache_dir` on open/report and cache lifecycle calls names the exact destination and
wins over `FDU_CACHE_DIR`. Otherwise `XDG_CACHE_HOME/fdu` wins over the platform
default: `~/.cache/fdu` on macOS and Linux, `%LOCALAPPDATA%/fdu` on Windows.
`list_caches` and `clear_all_caches` take a keyword cache directory, independently of a
scan root. `cache_directory(cache_dir=...)` resolves that destination without a root.
`cache_path(root, cache_dir=...)` identifies the root’s `.metadata.bin` file; a matching
`.analysis.bin` stores derived metrics without source bodies.

## Directory Inventories and Formats

A default `Query()` keeps the existing directory tree.
Select a flat presentation when requesting a report, so an ordinary tree need not
materialize a complete inventory:

```python
from pathlib import Path

import fdu

index = fdu.open(Path("/path/to/tree"))
for name in (".venv", "venv", "node_modules", "target"):
    report = index.report(
        fdu.Query(
            format=fdu.Format.LONG,
            selection=fdu.Selection(
                kinds=(fdu.EntryKind.DIR,),
                include=(name,),
                before="30d",
                sort=fdu.SortKey.TIME,
                reverse=True,
            ),
        )
    )
    print(report.render())
    print(report.render(fdu.Format.JSON))
```

All four reads use one retained index without rescanning.
`View.LIST` is the metadata default.
`Format.TREE` explicitly requests the current tree; `PATHS` lists matching paths, `LONG`
adds size and signed age, and JSON/JSONL/YAML carry exact metrics.
`TEXT` selects automatic human presentation.
The `View.FILES` preset uses flat name ordering, and `View.TREE` presents structured
directory output. Largest/recent remain regular-file presets, with optional Paths/Long
output.

Every time a report reads is on one clock, `Selection.time`: `TimeBasis.CHANGED` by
default, the newer of modification and status change, so a copy or an extracted archive
that kept an older modification time is still new, or `TimeBasis.MODIFIED`, content age
as `ls -l` shows. The `since`/`before` window, `SortKey.TIME`, every age, `recent`, and
`ages` read it, and `Report.request.time` names it.

## Recent Writes, Ages, and Volumes

One walk says where the data written in a window now lives, how the selection divides by
age, and how full each filesystem is:

```python
from pathlib import Path

import fdu

index = fdu.open(Path("/path/to/tree"))
recent = index.report(
    fdu.Query(
        views=(fdu.View.TREE, fdu.View.AGES),
        selection=fdu.Selection(kinds=(fdu.EntryKind.FILE,), since="24h"),
    )
)
print(f"{recent.selected.bytes} of {recent.scanned.bytes} bytes written in a day")
for section in recent.sections:
    if isinstance(section, fdu.AgesSection):
        for band in section.bands:
            print(band.band, band.tally.files, band.cumulative.bytes)

disk = index.report(fdu.Query(views=(fdu.View.VOLUMES,)))
for section in disk.sections:
    if isinstance(section, fdu.VolumesSection):
        for row in section.volumes:
            print(row.path, row.capacity_bytes, row.available_bytes, row.counted.bytes)
```

Keep `EntryKind.FILE` with a window: without it a directory with any activity in the
window is selected whole.
Every report carries `Report.selected` and `Report.scanned`, each an `EntryTotals` of
files, directories, and both byte metrics, so a filtered root’s 100% can be read against
the whole scan without parsing a note.
`View.AGES` returns an `AgesSection`: nine `AgeBand` values, newest first, each with an
`AgeTally` of files, bytes, and the ignored part, a `cumulative` tally of it and every
newer band, and a `total` equal to a summary of the same selection.
Where the selection admits files alone, each band is also the files
`kinds=(EntryKind.FILE,)` with the window it names would select.
`View.VOLUMES` returns a `VolumesSection` of `VolumeRow` values: each filesystem the
report touches, its capacity, available space, type, and whether it shares space in a
pool, beside the `VolumeCounted` files and bytes this report counted there.
A filesystem the report entered at several paths is one row, named by the first, with
each other path in `also_entered` as a `VolumeEntered`. Capacity is read when the report
is, and below the root only where the walk listed and the operating system’s mount table
shows a mounted filesystem, not an automount trigger (autofs), at exactly that path, so
reading it mounts nothing, except an automount that expires between the report’s read of
the mount table and its call there, a window as long as the calls before it.
Any other row’s capacity is `None`, with an `unknown_reason` that says why, such as
`not a mount point now, so it was not asked`; `View.FULL` leaves volumes out.

Directory `FileRow` values carry subtree `bytes`, `allocated`, `files`, `dirs`,
`complete`, `time_ns`, `age_ns`, and `time_at`; non-directory counts are `None`. A
directory whose subtree was not listed in full has `complete=False`, lower-bound sizes,
`age_ns=None`, and `time_at=None`. Every `FileRow` also carries both raw clocks,
`mtime_ns` and `changed_ns`, so an arrival carrying old content is a row with a recent
`changed_ns` and an old `mtime_ns`. Each `TreeNode` carries the same `time_ns`,
`complete` (`None` for a file), `age_ns`, and `time_at`: the newest time among the
entries the row counts, of any kind, never the report root’s own time, and `None` when
it counts nothing. Its `newest_file_time_ns` keeps the newest regular file alone.
`Report.age_reference_ns` is the instant every age is measured from, including negative
future ages and pre-epoch mtime, and `Report.age_reference_at` is that instant as a
`datetime`. A `Watch.report()` measures from the instant it is called; the windows its
query selects by stay as the query resolved them.
`time_at` and `age_reference_at` are timezone-aware UTC `datetime` values derived from
the exact nanoseconds, floored to the microsecond.
An unrepresentable reference yields `None` age.
Exclusions win throughout the subtree; nested roots may overlap, while grouped totals
count their union once.
Age describes change, not access or last use.
Native entry/lookup projections retain inode attributes; these report rows deliberately
carry subtree metrics instead.

A Report owns its requested projection.
Re-render it to another serialization or between Paths and Long without querying again;
request another report to change between a bounded tree and complete flat inventory.
An incompatible conversion raises `InvalidArgumentError` rather than silently listing
only visible tree rows.
`Selection(depth=5, min_share="1%", breadth=Bound.ALL, limit=Bound.ALL)` describes the
ordinary tree defaults.
Breadth bounds children per directory; limit bounds data rows per section.
`TreeSection.limits` records the effective controls.
Each `TreeOmission.entries` counts directly hidden children: an omitted directory is one
entry, while its nullable `files` count covers regular files throughout that hidden
subtree. The optional `TreeSection.remainder` combines hidden contents across one tree,
with nullable `files`, `bytes`, and `allocated` totals and bound reasons in stable
order. It is `None` when nothing is hidden.
A displayed root-level directory represents all its descendants, including children
hidden by display bounds.
The remainder counts only usage outside those displayed root-level rows.
Parent totals already include the remainder, so do not add it to them.
The terminal gives that remainder one root-level line, such as
`… and 12,345 more files`, with its combined size and root share in the usual columns.
Use `report.render(bar_size=20)` for wider bars, or `bar_size=0` (also negative values)
to hide them; the default is 10. Machine formats are unchanged.
Machine reports use `fdu.report/12`. Machine List output is complete unless explicitly
limited. Details and exact fields are in the
[usage guide](https://github.com/jlevy/fdu/blob/main/docs/usage.md) and
[machine-output reference](https://github.com/jlevy/fdu/blob/main/docs/machine-output.md).

## Long-Lived Roots

`fdu.opened` exposes the progressive API (`OpenedIndex`) over the shared native engine.
It starts progressive discovery, returns several projections from one coherent version,
exposes exact resumable changes, verifies explicit path sets, accepts discovery
priorities, and joins all native work on close:

```python
from pathlib import Path

from fdu import opened

with opened.OpenedIndex.open(Path("."), opened.OpenedOptions(observe=True)) as index:
    answer = index.read(
        opened.Tree(page=opened.Page(limit=200, max_work=100_000)),
        opened.Diagnostics(),
    )
    cursor = answer.change_cursor
    changed = index.changes(cursor, timeout=1.0)
```

The methods are synchronous.
Opened native open/read/state, change polling, refresh, prioritization, and close
release the GIL. Report serialization and Python result construction currently hold it.
An async application adapts them with its own executor policy, keeping task and shutdown
ownership visible. The package does not run a private event loop or shell out to the
command line. Cancelling an executor future does not itself cancel native work.
Closing an `OpenedIndex` shuts down its shared root for all aliases rather than one
request.

Retain the inventory in Rust and request summaries, selected answers, or bounded pages
for ordinary Python use.
Complete reports remain available when needed, with allocation cost proportional to the
selected result. Public reports eagerly decode JSON into dictionaries and typed Python
models and retain both the wire representation and a native rendering handle;
`as_dict()` copies the wire containers again.
Neither reports nor pages are zero-copy exports.
Complete paged enumeration can bound Python retention when previous pages are released,
subject to continuation invalidation as the engine version changes.

The wheel also exposes the native Rust CLI as the `fdu` console script.
Argument parsing, help, streams, color, errors, broken-pipe handling, and exit status
all use the same Rust process boundary as the Cargo-installed binary; there is no Python
CLI reimplementation.

**Status:** 0.x. A new minor release may change the Python API;
[the release process](https://github.com/jlevy/fdu/blob/main/docs/project/guides/release-process.md)
states the compatibility rules.
Building and testing the package from a checkout is covered in
[the repository README](https://github.com/jlevy/fdu#development).

License: MIT.

<!-- This document follows common-doc-guidelines.md.
See github.com/jlevy/practical-prose and review guidelines before editing.
-->

