Metadata-Version: 2.4
Name: spark-rust-assets
Version: 0.42.1a1
Summary: Spark Rust multi-environment alpha distribution
Requires-Python: >=3.9
License-Expression: Apache-2.0
License-File: LICENSE
Description-Content-Type: text/markdown

# Spark Rust — multi-environment alpha

The Python distribution is `spark-rust`; import it as `sparkrust`. The alpha
combines an embedded CPU SQL engine with compiled cluster runtimes, deployment
assets, documentation, examples, browser WASM and language-package archives.
It does **not** include the Rust engine source, install Apache Spark, or claim
complete PySpark compatibility.

## Install and check

The release candidate is `0.42.1a1`, with engine version `0.42.1`. Publication
status is recorded in the release receipt; a local candidate is not proof of a
public PyPI upload. Once published:

```bash
python3 -m pip install 'spark-rust==0.42.1a1'
sparkrust version
sparkrust doctor
sparkrust sql 'SELECT 40 + 2 AS answer'
```

For a locally supplied release wheelhouse instead:

```bash
python3 -m pip install --find-links /path/to/wheels 'spark-rust==0.42.1a1'
```

The main wheel requires exact-version companion wheels automatically. Native
alpha scope is **Linux x86_64, glibc >= 2.36**, such as Debian 12 or Ubuntu 24.04.
Other operating systems/architectures and GPU execution are not covered. The
CPython extension uses the stable ABI from Python 3.9; consult release acceptance
results for the interpreter versions actually tested. Python dependencies are
installed by pip; the pandas extra is available with `spark-rust[pandas]`.

## In-process Python

```python
import sparkrust

con = sparkrust.connect()
assert con.sql("SELECT 40 + 2 AS answer").collect() == [(42,)]
empty = con.sql("SELECT CAST(NULL AS INT) AS id WHERE false").arrow()
assert empty.num_rows == 0 and empty.column_names == ["id"]
con.close()
```

Use `.arrow()` for Arrow, `.collect()` for tuples and `.toPandas()` for pandas.
`sparkrust demo` and `sparkrust tour` use bundled sample data.

## Local server and native tools

```bash
sparkrust runtime list
sparkrust serve --port 15002 --ui-port 4040
# Run in a separate shell when desired:
sparkrust runtime run spark-sql -- --help
```

`serve` resolves the packaged matching Connect server; no Rust build is needed.
The server's bind/security defaults are not a production exposure policy. Use
private networks, authentication and TLS as described in the cluster guide.
The native `spark-sql` shell is local, not a remote Connect client.

## VM clusters and AKS

Export the installed binaries, installers, chart, documentation and notices:

```bash
sparkrust runtime export ./spark-rust-cluster-alpha
cd spark-rust-cluster-alpha
sha256sum -c SHA256SUMS
# Safe unprivileged installation check; does not create users or start services:
DESTDIR="$PWD/staged" bash vm/install.sh
```

For actual VMs, copy this verified bundle to each node and follow `README.md`
for prerequisites, roles, addresses, shared storage, authentication, installation,
health checks, upgrade and rollback. Installation and service startup remain
explicit operator actions. The alpha installs under `/opt/spark-rust/0.42.1a1`.

The export also includes a runtime-only Dockerfile and Helm chart:

```bash
docker build --platform linux/amd64 -f Dockerfile -t spark-rust-cluster:v0.42.1a1 .
helm lint ./helm -f ./helm/values-aks.yaml
```

This builds locally. Push/import into your authorized registry separately, obtain
its real image digest, and follow the AKS guide with `./helm` as the chart path.
No public container image is implied by PyPI publication. Real AKS networking,
identity, storage and remote-query acceptance must be validated in your target
cluster. `pip install` does not create Azure resources or deploy workloads.

## Browser WASM

```bash
sparkrust wasm serve --demo bi --port 8000
```

Open `http://127.0.0.1:8000/`. The compiled engine and JavaScript are packaged;
no Rust/wasm-bindgen build is needed. Other demos are `browser`, `geospatial`,
`mission-control`, `ml-classification` and `notebook-explorer`. Some interfaces
or optional language modes use external CDN/map/Pyodide dependencies and require
network access; this is not a claim of a fully offline browser site.

## Documentation, examples and language clients

```bash
sparkrust resources list
sparkrust resources path assets/docs/install-cluster.md
sparkrust resources export ./spark-rust-resources
```

Exports refuse to overwrite an existing directory. Documentation and example
source live under `assets/`; Node, Ruby, TypeScript Connect, JVM/Scala, SparkR and
WASM SDK archives live under `clients/`. Install archives with the corresponding
language package manager. Node requires Node 20+, the native Ruby build targets
Ruby 3.1 on Linux x86_64, and JVM/Scala examples require Java 17. SparkR also needs
R and its JVM bridge. Notebook examples require their documented language kernels,
data and service prerequisites; pip does not install those external runtimes.

These archives have their own API/engine versions recorded in the release
manifest; their filenames do not imply independent registry publication. Small
Rust example files and notebooks are teaching material, not engine source.

## Release safety

Each wheel has a RECORD, license text and provenance; the release directory
contains exact artifact hashes and acceptance evidence. Do not substitute old
artifacts or retag an incompatible native binary. No pip installation step starts
services, modifies kubeconfig, runs sudo, deploys Helm, or publishes a registry
artifact. See the bundled release/cluster guides for limitations and security.
