Metadata-Version: 2.4
Name: tg-engine
Version: 1.0.0
Summary: TigerGate scan engine — static analysis for many languages, packaged for the TigerGate platform. Based on Semgrep (LGPL-2.1).
Author-email: TigerGate <support@tigergate.dev>
License-Expression: LGPL-2.1-or-later
Project-URL: Homepage, https://semgrep.dev
Project-URL: Documentation, https://semgrep.dev/docs
Project-URL: Repository, https://github.com/semgrep/semgrep
Project-URL: Issues, https://github.com/semgrep/semgrep/issues
Project-URL: Changelog, https://github.com/semgrep/semgrep/blob/develop/CHANGELOG.md
Classifier: Environment :: Console
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: attrs>=21.3
Requires-Dist: boltons~=21.0
Requires-Dist: click-option-group~=0.5
Requires-Dist: click~=8.1.8
Requires-Dist: colorama~=0.4.0
Requires-Dist: exceptiongroup~=1.2.0
Requires-Dist: glom>=23.3
Requires-Dist: jsonschema~=4.25.1
Requires-Dist: mcp==1.23.3
Requires-Dist: opentelemetry-api~=1.37.0
Requires-Dist: opentelemetry-sdk~=1.37.0
Requires-Dist: opentelemetry-exporter-otlp-proto-http~=1.37.0
Requires-Dist: opentelemetry-instrumentation-requests~=0.58b0
Requires-Dist: opentelemetry-instrumentation-threading~=0.58b0
Requires-Dist: packaging>=21.0
Requires-Dist: peewee~=3.14
Requires-Dist: pyjwt[crypto]~=2.12.0
Requires-Dist: requests~=2.22
Requires-Dist: rich>=13.5.2
Requires-Dist: ruamel.yaml>=0.18.15
Requires-Dist: ruamel.yaml.clib==0.2.14
Requires-Dist: semantic-version~=2.10.0
Requires-Dist: tomli~=2.0.1
Requires-Dist: typing-extensions~=4.2
Requires-Dist: urllib3~=2.0
Requires-Dist: wcmatch~=8.3
Requires-Dist: pywin32==311; sys_platform == "win32"
Dynamic: description
Dynamic: description-content-type
Dynamic: license-file

# tg-engine

A renamed fork of [semgrep](https://github.com/semgrep/semgrep) (pinned to upstream tag `v1.159.0`) published under a different PyPI distribution name for the TigerGate platform.

The internal Python module name is still `semgrep` — only the **distribution name** (`pip install ...`) and the **CLI binary** are renamed. This minimises the diff against upstream so future rebases stay near-trivial. All outbound telemetry to `semgrep.dev`-owned endpoints is also disabled at six call sites in the Python source.

| | |
|---|---|
| Upstream | https://github.com/semgrep/semgrep |
| Pinned upstream tag | `v1.159.0` |
| Package version | `1.0.0` |
| Distribution name (PyPI) | `tg-engine` |
| Console script | `tg-engine` |
| Internal Python module | `semgrep` (unchanged) |
| Telemetry | stripped (see `tg-engine: telemetry stripped` markers in source) |
| License | LGPL-2.1-or-later (inherited from upstream — see [LICENSE](LICENSE)) |

The full diff vs. upstream is small and intentional:

- [cli/pyproject.toml](cli/pyproject.toml) — distribution name, console script, description, authors, version.
- 4 Python source files — six telemetry call sites early-return so no traffic reaches `metrics.semgrep.dev`, `telemetry.semgrep.dev`, `pyroscope-receive.semgrep.dev`, `fail-open.prod.semgrep.dev`, or `semgrep.dev/api/check-version`. Each edit is marked with the comment `tg-engine: telemetry stripped` so it's grep-able and rebase-aware.

Everything else (OCaml `semgrep-core`, Python source under `cli/src/semgrep/`, LSP message types, tests, CI workflows) is untouched.

---

## Build the wheel (locally, current platform)

A self-contained release wheel needs three things in `cli/src/semgrep/bin/`:
- The `semgrep-core` OCaml binary for the target platform.
- Its companion shared libraries under `bin/libs/`.
- `__init__.py` (already present).

Use the helper script — it pulls upstream's prebuilt binary for *your current platform*, drops it into the source tree, and builds the wheel:

```bash
./scripts/build-wheel.sh
```

Output: `cli/dist/tg_engine-1.0.0-<python-tags>-<platform>.whl`, ~70 MB.

The shipped binary is upstream's compiled `semgrep-core` — LGPL-compliant redistribution. The source tree's `cli/src/semgrep/bin/{semgrep-core, libs/}` is gitignored by upstream's `.gitignore` so the large binaries never land in git.

### Manual build (if you need finer control)

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip build
cd cli
python -m build --wheel
```

A wheel built this way **does not** include `semgrep-core` — `tg-engine` will install but any scan will fail with `Failed to find semgrep-core in PATH or in the semgrep package.` Use `scripts/build-wheel.sh` for anything beyond rename-only smoke testing.

---

## Smoke test the wheel

```bash
python3 -m venv /tmp/tg-smoke
source /tmp/tg-smoke/bin/activate
pip install cli/dist/tg_engine-*.whl

pip show tg-engine               # Name: tg-engine, Version: 1.0.0
which tg-engine                  # /tmp/tg-smoke/bin/tg-engine
tg-engine --version              # prints semgrep-core's version (1.16x)
python -c "import semgrep"       # internal module still importable
tg-engine scan --config p/python /path/to/some.py   # produces findings
```

---

## Release setup

The high-level shape:

```
For each target platform: build a per-platform wheel
   └── (optional) Build sdist on one platform
       └── twine check
           └── twine upload --repository testpypi      ← rehearsal
               └── pip install from TestPyPI; verify
                   └── twine upload                    ← real publish
                       └── git tag tg-engine-<ver>
```

### One-time setup

1. **PyPI accounts.** Create one at https://pypi.org and one at https://test.pypi.org. Generate an API token for each (Account → API tokens → scope = entire account or specific project).
2. **Local tooling.** In your release venv:
   ```bash
   pip install --upgrade build twine keyring
   ```
3. **Credentials.** Put tokens in `~/.pypirc` (permissions 0600):
   ```ini
   [distutils]
   index-servers =
       pypi
       testpypi

   [pypi]
   username = __token__
   password = pypi-<your-token>

   [testpypi]
   repository = https://test.pypi.org/legacy/
   username = __token__
   password = pypi-<your-testpypi-token>
   ```
   Or use `keyring` (more secure): `keyring set https://upload.pypi.org/legacy/ __token__`.

### Per-release flow

1. **Bump version.** Edit two files in lockstep:
   - `cli/pyproject.toml` — `[project] version = "X.Y.Z"`
   - `cli/src/semgrep/__init__.py` — `__VERSION__ = "X.Y.Z"`
2. **Build per-platform wheels.** A single tg-engine release needs one wheel per OS/CPU customers run. Locally you can only build for your current platform. The rest needs CI (see below).
   ```bash
   ./scripts/build-wheel.sh
   ```
3. **Verify.**
   ```bash
   twine check cli/dist/*
   ```
4. **TestPyPI rehearsal.**
   ```bash
   twine upload --repository testpypi cli/dist/*
   python3 -m venv /tmp/tg-release-test
   /tmp/tg-release-test/bin/pip install --index-url https://test.pypi.org/simple/ \
                                        --extra-index-url https://pypi.org/simple/ \
                                        tg-engine
   /tmp/tg-release-test/bin/tg-engine --version
   ```
5. **Real publish.**
   ```bash
   twine upload cli/dist/*
   ```
6. **Tag the release.**
   ```bash
   git tag -a tg-engine-X.Y.Z -m "tg-engine X.Y.Z"
   git push origin main --tags
   ```

### CI release via GitHub Actions

The full multi-arch build + publish flow lives in [.github/workflows/release.yml](.github/workflows/release.yml). It builds five wheels in parallel (Linux x64, Linux arm64, macOS x64, macOS arm64, Windows x64), then publishes them in a single approved-gated job.

**Trigger map:**

| Tag pushed | Build | Publish | GitHub release |
|---|---|---|---|
| `tg-engine-1.0.0` | all 5 archs | **PyPI** | yes |
| `tg-engine-1.0.0-rc.1` (also `-alpha`, `-beta`) | all 5 archs | **TestPyPI** | yes (prerelease) |
| `tg-engine-1.0.0-dev` | all 5 archs | none (artifacts only) | yes (prerelease) |
| manual `workflow_dispatch` | all 5 archs | chosen via input dropdown | no |

**Cutting a release:**

```bash
# Lockstep version bump
sed -i 's/version = "1\.0\.0"/version = "1.0.1"/' cli/pyproject.toml
sed -i 's/__VERSION__ = "1\.0\.0"/__VERSION__ = "1.0.1"/' cli/src/semgrep/__init__.py
git commit -am "Release 1.0.1"

git tag -a tg-engine-1.0.1 -m "tg-engine 1.0.1"
git push origin main --tags
# → CI builds all archs, waits at the `release` environment for approval,
#   then publishes to PyPI + cuts a GitHub release with the wheels attached.
```

### One-time CI setup (do this before the first release)

1. **PyPI OIDC trusted publisher.** No API token needed — PyPI verifies GitHub's OIDC claim directly. Go to https://pypi.org/manage/account/publishing/ → Add a new pending publisher:
   - PyPI Project Name: `tg-engine`
   - Owner: your GitHub org/user
   - Repository name: your repo name
   - Workflow filename: `release.yml`
   - Environment name: `release`

   For TestPyPI, repeat at https://test.pypi.org/manage/account/publishing/.

2. **GitHub Environment with required reviewer.** This is the approval gate before any upload happens. In repo Settings → Environments:
   - Click "New environment", name it `release`.
   - Under "Deployment protection rules", enable "Required reviewers" and add yourself (or the relevant team).
   - Save.

3. **Push a test tag** to verify the wiring:
   ```bash
   git tag -a tg-engine-1.0.0-dev -m "wiring test"
   git push origin tg-engine-1.0.0-dev
   ```
   This builds all 5 wheels and creates a GitHub prerelease without uploading anywhere. Check that all 5 wheels are attached to the release before doing a real publish.

### Publishing to a private index

Replace the `twine upload` call with the private index URL (or add to `~/.pypirc`):

```bash
twine upload --repository-url https://pypi.tigergate.dev/simple/ cli/dist/*
```

Or via `~/.pypirc`:

```ini
[tigergate]
repository = https://pypi.tigergate.dev/simple/
username = __token__
password = <token>
```

Then: `twine upload --repository tigergate cli/dist/*`.

### Versioning policy

This fork uses SemVer (`MAJOR.MINOR.PATCH`) independent of upstream's calendar-y numbering. Upstream version is tracked in the README's "Pinned upstream tag" row. Bump:

- **MAJOR** on any breaking change to the public CLI/LSP contract.
- **MINOR** when rebasing to a newer upstream tag (new features inherited).
- **PATCH** for telemetry-strip extensions, bug fixes, or internal-only changes.

---

## Rebasing onto a newer upstream tag

This repo was initialised as a fresh `git init` with no upstream remote, so `git rebase upstream/<tag>` is not directly available. Two options:

**Option A — re-fetch upstream and replay the rename.** Clone the new upstream tag into a temporary directory, copy the new tree over this repo's working tree (preserving `README.md`, `scripts/`, `.git/`), re-apply the rename in `cli/pyproject.toml` + the six `tg-engine: telemetry stripped` patches + version bump, commit.

**Option B — restore an upstream remote.**

```bash
git remote add upstream https://github.com/semgrep/semgrep.git
git fetch upstream --tags
# Inspect the diff and cherry-pick or merge as appropriate.
```

The rename and telemetry strips are intentionally clustered into single commits so conflicts during rebase are localised. Update the "Pinned upstream tag" row in this README and bump the MINOR version.

---

## Credit and license

This project is a redistribution of [semgrep](https://github.com/semgrep/semgrep) (LGPL-2.1-or-later) authored by Semgrep Inc. All credit for the scanner engine belongs to the upstream authors. The full upstream license is preserved verbatim in [LICENSE](LICENSE). See the upstream README at https://github.com/semgrep/semgrep for documentation, rule syntax, and contribution guidelines.
