tg-engine 1.0.0

TigerGate scan engine — static analysis for many languages, packaged for the TigerGate platform. Based on Semgrep (LGPL-2.1).

tg-engine

A renamed fork of semgrep (pinned to upstream tag v1.159.0) published under a different PyPI distribution name for the TigerGate platform.

The internal Python module name is still semgrep — only the distribution name (pip install ...) and the CLI binary are renamed. This minimises the diff against upstream so future rebases stay near-trivial. All outbound telemetry to semgrep.dev-owned endpoints is also disabled at six call sites in the Python source.

Upstream	https://github.com/semgrep/semgrep
Pinned upstream tag	v1.159.0
Package version	1.0.0
Distribution name (PyPI)	tg-engine
Console script	tg-engine
Internal Python module	semgrep (unchanged)
Telemetry	stripped (see tg-engine: telemetry stripped markers in source)
License	LGPL-2.1-or-later (inherited from upstream — see LICENSE)

The full diff vs. upstream is small and intentional:

• cli/pyproject.toml — distribution name, console script, description, authors, version.
• 4 Python source files — six telemetry call sites early-return so no traffic reaches metrics.semgrep.dev, telemetry.semgrep.dev, pyroscope-receive.semgrep.dev, fail-open.prod.semgrep.dev, or semgrep.dev/api/check-version. Each edit is marked with the comment tg-engine: telemetry stripped so it's grep-able and rebase-aware.

Everything else (OCaml semgrep-core, Python source under cli/src/semgrep/, LSP message types, tests, CI workflows) is untouched.

---

Build the wheel (locally, current platform)

A self-contained release wheel needs three things in cli/src/semgrep/bin/:

• The semgrep-core OCaml binary for the target platform.
• Its companion shared libraries under bin/libs/.
• __init__.py (already present).

Use the helper script — it pulls upstream's prebuilt binary for your current platform, drops it into the source tree, and builds the wheel:

./scripts/build-wheel.sh

Output: cli/dist/tg_engine-1.0.0-<python-tags>-<platform>.whl, ~70 MB.

The shipped binary is upstream's compiled semgrep-core — LGPL-compliant redistribution. The source tree's cli/src/semgrep/bin/{semgrep-core, libs/} is gitignored by upstream's .gitignore so the large binaries never land in git.

Manual build (if you need finer control)

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip build
cd cli
python -m build --wheel

A wheel built this way does not include semgrep-core — tg-engine will install but any scan will fail with Failed to find semgrep-core in PATH or in the semgrep package. Use scripts/build-wheel.sh for anything beyond rename-only smoke testing.

---

Smoke test the wheel

python3 -m venv /tmp/tg-smoke
source /tmp/tg-smoke/bin/activate
pip install cli/dist/tg_engine-*.whl

pip show tg-engine               # Name: tg-engine, Version: 1.0.0
which tg-engine                  # /tmp/tg-smoke/bin/tg-engine
tg-engine --version              # prints semgrep-core's version (1.16x)
python -c "import semgrep"       # internal module still importable
tg-engine scan --config p/python /path/to/some.py   # produces findings

---

Release setup

The high-level shape:

For each target platform: build a per-platform wheel
   └── (optional) Build sdist on one platform
       └── twine check
           └── twine upload --repository testpypi      ← rehearsal
               └── pip install from TestPyPI; verify
                   └── twine upload                    ← real publish
                       └── git tag tg-engine-<ver>

One-time setup

1. PyPI accounts. Create one at https://pypi.org and one at https://test.pypi.org. Generate an API token for each (Account → API tokens → scope = entire account or specific project).
2. Local tooling. In your release venv:

   pip install --upgrade build twine keyring
   

3. Credentials. Put tokens in ~/.pypirc (permissions 0600):

   [distutils]
   index-servers =
       pypi
       testpypi
   
   [pypi]
   username = __token__
   password = pypi-<your-token>
   
   [testpypi]
   repository = https://test.pypi.org/legacy/
   username = __token__
   password = pypi-<your-testpypi-token>
   

   Or use keyring (more secure): keyring set https://upload.pypi.org/legacy/ __token__.

Per-release flow

1. Bump version. Edit two files in lockstep:
   • cli/pyproject.toml — [project] version = "X.Y.Z"
   • cli/src/semgrep/__init__.py — __VERSION__ = "X.Y.Z"
2. Build per-platform wheels. A single tg-engine release needs one wheel per OS/CPU customers run. Locally you can only build for your current platform. The rest needs CI (see below).

   ./scripts/build-wheel.sh
   

3. Verify.

   twine check cli/dist/*
   

4. TestPyPI rehearsal.

   twine upload --repository testpypi cli/dist/*
   python3 -m venv /tmp/tg-release-test
   /tmp/tg-release-test/bin/pip install --index-url https://test.pypi.org/simple/ \
                                        --extra-index-url https://pypi.org/simple/ \
                                        tg-engine
   /tmp/tg-release-test/bin/tg-engine --version
   

5. Real publish.

   twine upload cli/dist/*
   

6. Tag the release.

   git tag -a tg-engine-X.Y.Z -m "tg-engine X.Y.Z"
   git push origin main --tags
   

CI release via GitHub Actions

The full multi-arch build + publish flow lives in .github/workflows/release.yml. It builds five wheels in parallel (Linux x64, Linux arm64, macOS x64, macOS arm64, Windows x64), then publishes them in a single approved-gated job.

Trigger map:

Tag pushed	Build	Publish	GitHub release
tg-engine-1.0.0	all 5 archs	PyPI	yes
tg-engine-1.0.0-rc.1 (also -alpha, -beta)	all 5 archs	TestPyPI	yes (prerelease)
tg-engine-1.0.0-dev	all 5 archs	none (artifacts only)	yes (prerelease)
manual workflow_dispatch	all 5 archs	chosen via input dropdown	no

Cutting a release:

# Lockstep version bump
sed -i 's/version = "1\.0\.0"/version = "1.0.1"/' cli/pyproject.toml
sed -i 's/__VERSION__ = "1\.0\.0"/__VERSION__ = "1.0.1"/' cli/src/semgrep/__init__.py
git commit -am "Release 1.0.1"

git tag -a tg-engine-1.0.1 -m "tg-engine 1.0.1"
git push origin main --tags
# → CI builds all archs, waits at the `release` environment for approval,
#   then publishes to PyPI + cuts a GitHub release with the wheels attached.

One-time CI setup (do this before the first release)

1. PyPI OIDC trusted publisher. No API token needed — PyPI verifies GitHub's OIDC claim directly. Go to https://pypi.org/manage/account/publishing/ → Add a new pending publisher:

   • PyPI Project Name: tg-engine
   • Owner: your GitHub org/user
   • Repository name: your repo name
   • Workflow filename: release.yml
   • Environment name: release

   For TestPyPI, repeat at https://test.pypi.org/manage/account/publishing/.

2. GitHub Environment with required reviewer. This is the approval gate before any upload happens. In repo Settings → Environments:

   • Click "New environment", name it release.
   • Under "Deployment protection rules", enable "Required reviewers" and add yourself (or the relevant team).
   • Save.

3. Push a test tag to verify the wiring:

   git tag -a tg-engine-1.0.0-dev -m "wiring test"
   git push origin tg-engine-1.0.0-dev
   

   This builds all 5 wheels and creates a GitHub prerelease without uploading anywhere. Check that all 5 wheels are attached to the release before doing a real publish.

Publishing to a private index

Replace the twine upload call with the private index URL (or add to ~/.pypirc):

twine upload --repository-url https://pypi.tigergate.dev/simple/ cli/dist/*

Or via ~/.pypirc:

[tigergate]
repository = https://pypi.tigergate.dev/simple/
username = __token__
password = <token>

Then: twine upload --repository tigergate cli/dist/*.

Versioning policy

This fork uses SemVer (MAJOR.MINOR.PATCH) independent of upstream's calendar-y numbering. Upstream version is tracked in the README's "Pinned upstream tag" row. Bump:

• MAJOR on any breaking change to the public CLI/LSP contract.
• MINOR when rebasing to a newer upstream tag (new features inherited).
• PATCH for telemetry-strip extensions, bug fixes, or internal-only changes.

---

Rebasing onto a newer upstream tag

This repo was initialised as a fresh git init with no upstream remote, so git rebase upstream/<tag> is not directly available. Two options:

Option A — re-fetch upstream and replay the rename. Clone the new upstream tag into a temporary directory, copy the new tree over this repo's working tree (preserving README.md, scripts/, .git/), re-apply the rename in cli/pyproject.toml + the six tg-engine: telemetry stripped patches + version bump, commit.

Option B — restore an upstream remote.

git remote add upstream https://github.com/semgrep/semgrep.git
git fetch upstream --tags
# Inspect the diff and cherry-pick or merge as appropriate.

The rename and telemetry strips are intentionally clustered into single commits so conflicts during rebase are localised. Update the "Pinned upstream tag" row in this README and bump the MINOR version.

---

Credit and license

This project is a redistribution of semgrep (LGPL-2.1-or-later) authored by Semgrep Inc. All credit for the scanner engine belongs to the upstream authors. The full upstream license is preserved verbatim in LICENSE. See the upstream README at https://github.com/semgrep/semgrep for documentation, rule syntax, and contribution guidelines.