Metadata-Version: 2.4
Name: nfs4fs
Version: 0.3.7
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Classifier: Typing :: Typed
Requires-Dist: fsspec>=2024.12.0,<2027.0.0
Requires-Dist: vfsi-fsspec>=0.1.7,<0.2
Requires-Dist: black ; extra == 'dev'
Requires-Dist: build ; extra == 'dev'
Requires-Dist: hypothesis>=6.100 ; extra == 'dev'
Requires-Dist: mypy ; extra == 'dev'
Requires-Dist: pytest ; extra == 'dev'
Requires-Dist: ruff ; extra == 'dev'
Requires-Dist: twine ; extra == 'dev'
Requires-Dist: hypothesis>=6.100 ; extra == 'test'
Requires-Dist: pytest ; extra == 'test'
Provides-Extra: dev
Provides-Extra: test
License-File: LICENSE-APACHE
License-File: LICENSE-MIT
License-File: LICENSE-libntirpc
Summary: fsspec filesystem backed by the vectorized VFSI NFSv4 client
Keywords: vfsi,fsspec,nfs,nfs4,filesystem,vectorized-io
Home-Page: https://github.com/vmingchen/vnfs
Author-email: Ming Chen <v.mingchen@gmail.com>
License-Expression: MIT OR Apache-2.0
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/vmingchen/vnfs/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/vmingchen/vnfs/tree/main/adapters/nfs4fs
Project-URL: Homepage, https://github.com/vmingchen/vnfs
Project-URL: Issues, https://github.com/vmingchen/vnfs/issues
Project-URL: Repository, https://github.com/vmingchen/vnfs

# nfs4fs

`nfs4fs` is a Linux `fsspec` implementation backed by
[VFSI](https://github.com/vmingchen/vnfs)'s vectorized NFSv4 client. Bulk
`fsspec` operations are translated into NFS compounds, reducing round trips
for workloads with many small files.

## Install

```sh
python -m pip install nfs4fs
```

Published wheels use CPython's stable ABI and support standard CPython 3.9 and
newer on Linux x86-64 and AArch64. A separate CPython 3.14 free-threaded wheel
is built and tested on x86-64.

A source build additionally requires Rust, CMake, a C compiler, Clang,
`pkg-config`, Kerberos/GSS development headers, and userspace-RCU development
headers. Source and editable builds compile the pinned ntirpc source packaged
in `libntirpc-sys` into a position-independent static archive. A system
libntirpc is not required at build or runtime.

Release wheels are self-contained. They statically include the pinned ntirpc
source revision, and `auditwheel` copies its
Kerberos (`libgssapi_krb5`/`libkrb5`) and userspace-RCU (`liburcu`)
dependencies into the wheel's `.libs/` directory, rewriting RPATH so the
package imports without system native dependencies. CI verifies those bundled
libraries and rejects any wheel that dynamically links or bundles a second
libntirpc.

The package installs `fsspec.specs` entry points for both `nfs4` and `vfsi`, so
normal use does not require `import nfs4fs` before calling `fsspec`.

## NFSv4

Pass the server as an option or as the URL authority:

```python
import fsspec

fs = fsspec.filesystem(
    "nfs4",
    host="nfs.example",
    root="exports/project",
    auth="auth_sys",
    auto_mkdir=False,
)

fs.pipe({"/file-1": b"hello", "/file-2": b"world"})
files = fs.cat(["/file-1", "/file-2"])

with fsspec.open(
    "nfs4://nfs.example/exports/project/file-1", "rb", auth="auth_sys"
) as file:
    print(file.read())
```

The `pipe()` call gives both writes to VFSI at once. For these two small files,
it encodes both independent write chains in one NFSv4 COMPOUND and completes
them in one network round trip. The following `cat()` also preserves the batch;
it currently uses one compound for bounded size discovery and one for the
reads.


### Append and concurrent writers

Append mode writes at the file size observed by the client before its WRITE.
That sequence is not atomic across independent NFS sessions or clients.
Use one writer for each append file, or coordinate writers with an external
lock. This includes writers opened through different filesystem instances,
processes, or pooled sessions; a successful append call does not guarantee
that another client's bytes were preserved.

The client negotiates NFSv4.2 and falls back to NFSv4.1. Set
`minor_version=1` or `minor_version=2` to require a specific version. Select
`auth="auth_sys"` explicitly to use AUTH_SYS. For Kerberos-backed RPCSEC_GSS,
obtain a ticket in the
process credential cache and select `auth="krb5"` or `auth="krb5i"`; the latter
integrity-protects RPC payloads. Selecting either mode is explicit and fails
closed if Kerberos authentication cannot be established; the client does not
downgrade to AUTH_SYS. An optional `service_principal` overrides the default
`nfs@<host>` principal.

```sh
kinit user@EXAMPLE.COM
```

```python
fs = fsspec.filesystem(
    "nfs4",
    host="nfs.example.com",
    root="exports/project",
    auth="krb5i",
)
```

### Connect from an existing mount

If the directory is already mounted on Linux, infer the connection instead:

```python
fs = fsspec.filesystem("nfs4", mount="/mnt/data/git/some/tree")
fs.pipe({"/file-1": b"hello", "/file-2": b"world"})
files = fs.cat(["/file-1", "/file-2"])
```

`mount` must be an absolute path to an existing directory inside the mount.
Relative paths are rejected to avoid working-directory-dependent instance
caching. If `server:/export` is mounted at `/mnt/data`, this example uses
`/export/git/some/tree` as its
remote root. Discovery supports NFSv4.1/4.2 TCP `sec=sys` mounts and infers
AUTH_SYS from that explicit mount configuration. Read-only mounts remain
read-only through the direct client. Unsupported security modes, bind-root
mappings, and directories containing nested mounts fail with a descriptive
error. Do not combine `mount` with `host`, `root`, `auth`, `minor_version`, or
other explicit connection settings; tuning such as timeouts and pool size
remains available.

**Kerberos mounts are not supported by mount discovery yet.** A mount using
RPCSEC_GSS (`sec=krb5`, `sec=krb5i`, or `sec=krb5p`) is rejected; `mount=`
never silently falls back to AUTH_SYS. To use Kerberos, configure the server
and remote root explicitly with `auth="krb5"` or `auth="krb5i"`, as shown
above. This establishes an independent GSS context using the process's
Kerberos credentials, not the kernel mount's context. RPCSEC_GSS privacy
(`krb5p`) is not currently supported by nfs4fs.

Discovery happens during construction. All operations, pooled connections,
and reconnects use the same pinned remote configuration. `fs.mount` exposes
the resolved local directory and `fs.read_only` reports its write restriction.
The client uses direct NFS for all operations, with its own caches and open
state. It does not share or invalidate the mounted kernel client's caches.

`root` is a client-side path prefix, not a security boundary. A relative
symlink inside it can resolve elsewhere in the same NFS export. Configure a
server export rooted at the intended directory when callers must be confined
to that directory.

`compound_size_limit` caps the request-byte budget for each compound, including
reserved RPC/authentication overhead, rather than just file data. It defaults to
1 MiB and cannot exceed the server's negotiated request maximum. Response bytes
and operation counts have separate negotiated limits; lowering this setting
does not lower the read-response budget:

```python
fs = fsspec.filesystem(
    "nfs4", host="nfs.example", auth="auth_sys", compound_size_limit=256 * 1024
)
```

Connection setup and individual NFS RPCs are independently bounded. Defaults
are 10 seconds and 5 seconds, respectively:

```python
fs = fsspec.filesystem(
    "nfs4",
    host="nfs.example",
    auth="auth_sys",
    connect_timeout=5.0,
    request_timeout=15.0,
)
```

## Operations that vectorize

The largest benefit comes from giving `fsspec` multiple paths at once:

- `cat`, `cat_ranges`, and `open_files` batch reads;
- `pipe` and `put` batch writes;
- recursive `get`, `put`, `cp`, `rm`, and tree walks use vector metadata and
  mutation operations;
- `cp_file`/`copy` retain the low-round-trip vector path for bounded small-file
  batches. Files beyond the read allocation budget use NFSv4.2 server-side
  copy when available, in bounded 16 MiB extents, with a streaming client-side
  fallback.

`minor_version()`, `capabilities()`, `server_copy_enabled()`,
`compound_stats()`, and `rpc_stats()` are available on the native client for
feature inspection and diagnostics.

## Small-file benchmark

A local benchmark compared nfs4fs directly with vanilla fsspec
`LocalFileSystem` accessing the same export through a Linux kernel NFS mount.
The server was NFS-Ganesha V15.3 over NFSv4.2 on an AArch64 Ubuntu VM, using
nfs4fs 0.3.1 and fsspec 2026.7.0. Linux `netem` added 500 microseconds to each
loopback traversal, approximately 1 ms of network round-trip delay; ping
averaged 1.43 ms including VM scheduling overhead. Each result is the median
of 30 fresh-path trials over 20 files of 4 KiB, with connection setup excluded,
fresh paths used for every trial, and client execution order alternated.

| Operation | nfs4fs | Kernel NFS + `LocalFileSystem` | Speedup | nfs4fs RPCs |
| --- | ---: | ---: | ---: | ---: |
| `pipe` 20 files | 38.80 ms | 151.31 ms | 3.90x | 1 |
| `cat` 20 files | 4.16 ms | 83.19 ms | 20.01x | 2 |

The same benchmark was repeated after an untimed warm-up, reusing the exact
same files for every measured trial. Ping averaged 1.35 ms during this run:

| Warm operation | nfs4fs | Kernel NFS + `LocalFileSystem` | Speedup | nfs4fs RPCs |
| --- | ---: | ---: | ---: | ---: |
| `pipe` 20 files | 38.03 ms | 143.30 ms | 3.77x | 1 |
| `cat` 20 files | 3.65 ms | 51.94 ms | 14.25x | 2 |

The kernel's metadata and page caches reduced its repeated-read time, but a
normal NFSv4 open and close still involves server state. `LocalFileSystem`
opens the files serially, so those latency-bearing operations remain serial as
well. nfs4fs does not retain file data across separate `cat()` calls; its warm
improvement comes from server-side caches while it continues to batch the
remote operations. Repeated writes changed little because both clients still
have to send the new contents to the server.

The benchmark measures a latency-bound small-file workload, not sequential
bandwidth, and results will vary with the server, client, mount options, and
compound limits. The exact driver is
[`benchmarks/small_files.py`](https://github.com/vmingchen/vnfs/blob/main/adapters/nfs4fs/benchmarks/small_files.py);
it verifies returned data and reports VFSI's compound and RPC counters so
batching regressions are visible alongside timing changes. Pass
`--reuse-paths` to reproduce the warm-cache variant.

## Directory and large-file benchmarks

The [benchmark drivers](https://github.com/vmingchen/vnfs/tree/main/adapters/nfs4fs/benchmarks)
also measure a metadata-heavy directory tree (`find` and recursive `rm`) and
bounded sequential reading of a single large file. Each compares nfs4fs with
fsspec `LocalFileSystem` on the same kernel NFS mount, checks results, and
reports timings alongside nfs4fs RPC counts. The large-file driver explicitly
disables per-open read caching so it measures the streaming path.

For stable large files, opt-in `read_stream_pipelined()` overlaps positional
reads across independent native sessions while delivering chunks in order:

```python
fs = fsspec.filesystem(
    "nfs4", host="nfs.example", auth="auth_sys", connection_pool_size=3
)
with open("download.bin", "wb") as output:
    fs.read_stream_pipelined(
        "/large.bin", lambda offset, data: output.write(data), workers=3
    )
```

Read-ahead is capped at one chunk per worker-owned descriptor, up to
`max_in_flight` chunks and `max_buffered_bytes` total. With the default three
workers and 1 MiB chunks, at most 3 MiB is in flight. This does not change
`fs.open()` and is not a coherent snapshot if another client modifies the file
during the read.
On the development VM, a 64 MiB warm-file read with approximately 5 ms RTT
fell from a 607 ms sequential median to 262 ms with three workers; the kernel
client's warm page-cache read was still faster. See the
[benchmark methodology](https://github.com/vmingchen/vnfs/blob/main/adapters/nfs4fs/benchmarks/README.md)
for the measured workload and limits.

All data-transfer entry points accept fsspec's `callback=` argument. Bulk
operations (`cat`, `cat_ranges`, `pipe`, `get`, `put`, and `copy`) report item
progress on the parent callback and byte progress on a branched callback for
each file. Single-file operations report bytes directly, and oversized
streaming transfers update after every chunk.

## File buffering

Read handles use fsspec's standard per-open caches by default. The default
1 MiB block size matches the VFSI range-I/O path and can be changed per
filesystem or per direct `fs.open()` call:

```python
fs = fsspec.filesystem(
    "nfs4",
    host="nfs.example",
    auth="auth_sys",
    block_size=256 * 1024,
    cache_type="readahead",
)

with fs.open("/large.parquet", "rb", cache_type="blockcache") as file:
    header = file.read(4096)
    file.seek(-8192, 2)
    footer = file.read()
```

All cache types registered by fsspec are accepted. Use `cache_type="none"`
for unbuffered reads, especially when another client can modify an already
open file. Buffered data belongs to one open handle and is discarded on close.
Blocks already fetched by that handle remain cached, but fsspec does not
revalidate the source while the handle is open, so uncached blocks may reflect
a concurrent writer. This is a per-open cache, not a persistent client cache.

fsspec also provides a persistent sparse-file wrapper under the same
`blockcache` name:

```python
cached = fsspec.filesystem(
    "blockcache",
    target_protocol="nfs4",
    target_options={"host": "nfs.example", "auth": "auth_sys"},
    cache_storage="/var/tmp/nfs4fs-cache",
    check_files=True,
)
```

With `check_files=True`, each open compares the saved source identity with
nfs4fs's `ukey()` and starts a fresh local cache generation when it changes.
For NFS this identity uses `FATTR4_CHANGE` and the file ID; the dummy backend
uses file ID, nanosecond timestamps, and size. Validation happens at
open, not on every read. With fsspec's default `check_files=False`, persistent
entries are reused without source validation until `expiry_time`, so that mode
is appropriate only when external clients use immutable or versioned paths.
Mutations made through the same nfs4fs target invalidate registered persistent
block-cache generations automatically, including renamed or recursively
deleted subtrees. Already-open handles retain their per-handle cached blocks;
new persistent generations use separate backing files so refreshes cannot
truncate an active mmap. Retired files are unlinked, with their storage released
when existing handles close.
`cache_check` controls how often local cache metadata is reloaded and does not
validate the server. `invalidate_cache()` controls directory listings; use
`pop_from_cache()` to evict persistent file data explicitly.

Files opened together with `fsspec.open_files()` share a coordinator. A cache
miss reads the same aligned range from a bounded group of sibling descriptors
through one VFSI vector operation. A requested whole-file read keeps its fast
path for the default read-ahead cache; `cache_type="blockcache"` fills its
bounded LRU block-by-block. Speculative siblings are capped to one block (which
may contain a whole small file). Retained group prefetch is also bounded by
the smaller of `max_batch_bytes` and `read_all_max_total_bytes`; older speculative
entries are evicted when necessary. Set `vectorized_buffering=False` to disable
this adaptive fan-out. The persistent `blockcache` wrapper participates in the
same coordinator: it validates source identities in a metadata batch, serves
complete hits locally, and fills sparse misses with vector reads.

Scalar `fs.open()` pins a native descriptor immediately so its handle remains
bound to the opened file after a rename or pathname replacement. Path-based
`cat()` and grouped `open_files()` retain their batched read paths.

Writes remain write-through by default. Set `write_buffering=True` to delay
small writes until `flush()` or `close()`; an `open_files()` write group drains
one block from each file per vector operation. Explicit `flush()` always sends
that file's staged data to the server. Update modes containing `+` retain the
unbuffered seekable implementation, and fsspec transactions retain their
existing disk-spooled commit behavior.

A failed buffered flush makes the writer unusable, including semantic errors
such as a full server. Close it to release its descriptor; `close()` reports the
failed writer without replaying any bytes that might already have been written.

## Directory listing cache

Directory listing caching is disabled by default so that NFS namespaces
modified by other clients remain immediately visible. Enable a
bounded-staleness cache with the standard fsspec options:

```python
fs = fsspec.filesystem(
    "nfs4",
    host="nfs.example",
    root="exports/project",
    auth="auth_sys",
    use_listings_cache=True,
    listings_expiry_time=1.0,
    max_paths=1024,
)
```

`listings_expiry_time` is the TTL in seconds and `max_paths` bounds the number
of directory paths tracked by fsspec's `DirCache`; it does not bound the number
of entries inside one directory. `ls(..., refresh=True)` bypasses the cached
listing. Native vectorized walks populate the cache for the complete subtree,
so repeated `walk`, `find`, `glob`, and `du` operations can reuse it without
turning a miss into one request per directory.

nfs4fs invalidates affected listings after its own writes and namespace
mutations, including uncertain failures. Changes made by another client become
visible when the TTL expires or when the caller requests a refresh. Use
`use_listings_cache=False` when any period of staleness is unacceptable.

## Production notes

- One filesystem instance owns `connection_pool_size` native sessions (one by
  default), each protected by its own mutex. Operations on one session are
  serialized, while a larger pool permits concurrent independent requests.
  Use separate instances with `skip_instance_cache=True` when independent
  filesystem lifetimes or configuration are required.
- Native calls release the CPython interpreter lock while waiting for storage.
  The extension also declares free-threaded CPython support; each filesystem's
  native-session mutex still serializes operations assigned to that session.
- On a transport failure, safe idempotent path reads and positional reads in
  `read_stream_pipelined()` reconnect their affected session and retry once by
  default. Mutations are never replayed automatically because the server may
  already have completed an ambiguously failed request. Set
  `auto_reconnect=False` to disable automatic read recovery in both the Python
  and native layers. This requires a native adapter with recovery control;
  an older adapter raises `ImportError` with upgrade instructions before
  connecting. Upgrade `nfs4fs` and `vfsi-fsspec` together. If native recovery
  cannot complete a read on an update-mode
  handle (`rb+`, `wb+`, `ab+`, or `xb+`), the handle becomes unusable and must be
  closed; Python never reopens it with potentially destructive creation flags.
- A process fork is detected before the next operation and creates a fresh
  native session in the child without sending protocol teardown over the
  parent's inherited connection. Do not fork while an operation is in flight
  or share an already-open `Nfs4File` across a fork. Once application or RPC
  helper threads exist, use multiprocessing's `spawn` method. Prefer creating
  filesystem instances after worker processes start.
- Call `close()` or use `with fsspec.filesystem(...) as fs:` to release the
  native session deterministically. Closing also removes the instance from
  fsspec's instance cache.
- `exists`, `isfile`, and `isdir` return `False` for missing paths but propagate
  authentication and connectivity failures. This prevents outages from being
  mistaken for absent data.
- `auto_mkdir` is `False` by default. Enable it only when write operations are
  allowed to create missing parents.
- `root` rejects `.` and `..` components. Use it to keep all paths under an
  export-relative prefix; it is not a substitute for server-side
  authorization.
- Vector batches are bounded by both `batch_size` (128 items) and
  `max_batch_bytes` (64 MiB). Files larger than that byte threshold stream in
  `transfer_chunk_size` chunks (8 MiB). Tune these per server and workload;
  every value must be positive.
- One native session preserves maximum batching by default. For thread-pool
  workloads, set `connection_pool_size` above one; independent path operations
  are distributed across sessions while every open descriptor remains pinned
  to its owning session. Each additional session consumes one server client
  identity and connection, so size the pool deliberately.
- APIs that materialize complete remote results have independent safety
  limits. `read_all_max_total_bytes` bounds every bytes-returning read,
  including whole-file reads, buffered `read(-1)`, binary `readline`/`readuntil`,
  `cat`, and `cat_ranges` (16
  MiB); `directory_max_entries` (100,000), `directory_max_path_bytes` (16
  MiB), and `walk_max_depth` (128) bound listings and recursive walks. These
  limits are configurable per filesystem instance and survive reconnects.
- `with fs.transaction:` stages scalar and grouped write/append/exclusive
  opens, `pipe`, and file uploads through `put`, independently of file size.
  Update modes containing `+` and directory uploads are rejected within a
  transaction before mutation. Other namespace operations such as `mkdir`
  and `rm` are not transactional. Remote staging files are created privately
  with mode `0600`; replacement preserves the destination's ordinary permission
  bits before publication. New transactional files remain private. Replacement
  creates a new inode; ownership and ACL preservation are not provided.
  Transaction writes require a native adapter with private-create support;
  an older adapter raises an upgrade error before allocating a write spool.
  Ordinary reads and writes remain available with otherwise compatible adapters.
- Transaction writes use a disk-backed spool after
  `transaction_spool_threshold` (8 MiB), upload to unpredictable temporary
  names in each destination directory, then expose the batch with a vectorized
  rename. This prevents partial file contents from becoming visible. Like
  fsspec transactions generally, a multi-file commit is staged rather than a
  server-wide atomic transaction: a server failure during the final rename
  batch can expose a prefix of the batch. Transactional `xb` writes and
  `touch(truncate=False)` of missing files publish through an atomic hard link
  from the staging file, so a concurrent creator causes `FileExistsError`
  instead of being overwritten. Backends without hard
  links reject that publication; they do not fall back to overwriting rename.
- `pipe_file(..., mode="create")` and create-mode `put` use an exclusive server
  create, so concurrent creators cannot silently overwrite one another.
  Outside a transaction, a later write or callback failure can leave an empty
  or partially written file. Cleanup closes the descriptor but does not unlink
  the destination, which may have been replaced by another writer.

## Local development

On Ubuntu/Debian:

```sh
sudo apt-get install build-essential cmake clang libclang-dev pkg-config \
  libkrb5-dev liburcu-dev
python -m venv .venv
.venv/bin/pip install hypothesis maturin fsspec pytest
.venv/bin/pip install -e adapters/vfsi-fsspec
cd adapters/nfs4fs
../../.venv/bin/maturin develop --locked
../../.venv/bin/python -m pytest tests
```

Tests use the local dummy backend unless NFS integration variables are
provided. CI runs the same suite against NFSv4.1, NFSv4.2, and patched
server-side COPY. Samba coverage lives with the `vsmbfs` adapter.

The default suite includes a deterministic Hypothesis state machine that runs
randomized filesystem histories against both nfs4fs's dummy backend and
fsspec's `LocalFileSystem` oracle. It compares public results, exception
classes, directory trees, and file contents after every step. Increase its
budget for a longer local fuzz run:

```sh
NFS4FS_FUZZ_EXAMPLES=1000 NFS4FS_FUZZ_STEPS=100 \
  ../../.venv/bin/python -m pytest tests/test_fuzz_parity.py
```

