Metadata-Version: 2.4
Name: torchlight-spark
Version: 0.1.0
Summary: Scan Python files and Jupyter notebooks for Spark method calls.
Author: Kyle Bush
License-Expression: Apache-2.0
Project-URL: Repository, https://github.com/Kyle-j-bush/torchlight
Project-URL: Issues, https://github.com/Kyle-j-bush/torchlight/issues
Project-URL: Changelog, https://github.com/Kyle-j-bush/torchlight/blob/main/CHANGELOG.md
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Dynamic: license-file

# Torchlight

[![CI](https://github.com/Kyle-j-bush/torchlight/actions/workflows/ci.yml/badge.svg)](https://github.com/Kyle-j-bush/torchlight/actions/workflows/ci.yml)
[![CodeQL](https://github.com/Kyle-j-bush/torchlight/actions/workflows/codeql.yml/badge.svg)](https://github.com/Kyle-j-bush/torchlight/actions/workflows/codeql.yml)

Find `.collect()` and `.distinct()` calls without running your code. Uses only
the Python standard library (Python 3.12+).

## Install and run

```sh
pip install torchlight-spark
torchlight path/to/file.py path/to/notebook.ipynb path/to/directory
```

For an isolated CLI installation, use `pipx install torchlight-spark` instead.
The distribution is named `torchlight-spark`, but the command and Python package
are named `torchlight`. Do not install it alongside the unrelated `torchlight`
PyPI distribution in the same environment.

You can also run directly from a checkout:

```sh
python main.py path/to/file.py path/to/notebook.ipynb path/to/directory
# Or: python -m torchlight <paths...>
```

- Directories are scanned recursively, skipping virtual environments, version
  control folders, notebook checkpoints, `__pycache__`, and `node_modules`.
- Notebooks support top-level `await`, line magics, shell escapes, and Python
  cell magics (`capture`, `debug`, `prun`, `time`, `timeit`). Other cell magics
  are skipped. Cell numbers count code cells only.
- Detection is based on method names, not Spark type information.

Findings go to stdout; errors go to stderr:

```text
pipeline.py:4: spark method .collect()
report.ipynb:cell 3,line 2: spark method .distinct()
```

Exit codes: **0** = no findings or errors, **1** = findings (even with errors),
**2** = errors only.

Run the regression tests:

```sh
python -m unittest discover -s tests
```

## Development and security

See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, linting, and the pull request
workflow. CI tests Python 3.12–3.14 on Linux and Python 3.12 on macOS and Windows.
It also builds and tests both the wheel and source distribution in clean environments.
The `main` branch requires passing CI and CodeQL checks through pull requests;
direct pushes and protection bypasses are disabled.

See [SECURITY.md](SECURITY.md) for safe-use guidance and private vulnerability
reporting. Never share credentials or private notebooks in issues.

Release notes are in [CHANGELOG.md](CHANGELOG.md).

## License

Copyright 2026 Kyle Bush. Licensed under the [Apache License 2.0](LICENSE)
(`Apache-2.0`). See [NOTICE](NOTICE) for attribution.
