fidesplus 2.87.4a49

Fidesplus

Fidesplus

Fidesplus is a monorepo containing the various premium offerings of the Fides ecosystem. These offerings are primarily distributed as Docker images, and therefore each published image is considered its own application. The available applications are as follows:

• Slim: This is the standard extension of Fides that includes the visual datamap, lightweight-masking regex engine, lightweight Fidescls (classify) implementation, and more.
• Fidesplus: This is a superset of slim. It contains all of Fidesplus as well as some more advanced Natural Language Processing libraries to increase the accuracy and functionality of the classification related features.
• Masking: (DEPRECATED) This is a lightweight wrapper around PrivateAI, an application that provides additional functionality around masking PII. This application has been deprecated as of August 2025, but as it is still in use by one Ethyca client, we've not fully removed it from the codebase yet.

Note: Most local development is done using the slim application for performance reasons, especially on M1 machines.

All of these applications can be built and tested using uv and the nox sessions in this repo (uv run nox …).

Monorepo layout

This repository is a monorepo: core OSS Fides and Fidesplus live in one tree. You do not need a separate checkout of ethyca/fides.

Component	Path
Fides (Python package fides)	src/fides/
Fidesplus server	src/fidesplus/
Admin UI	clients/admin-ui/
Privacy Center	clients/privacy-center/
Tests (Fidesplus)	tests/fidesplus/
Tests (Fides core)	tests/fides/

Dependencies are managed with uv from a single root pyproject.toml (uv.lock). Install everything with:

uv sync --all-groups

Getting Started

Follow these steps for our standard development environment:

1. Install these system requirements locally:

   • Docker (version 20.10.11 or later)
   • Python (version 3.13)
   • uv
   • Cursor or VSCode as your IDE
   • 1Password CLI (op) — required for automatic environment seeding (see Environment Setup & Seeding below). If you don't have 1PW CLI, you can still run the dev environment without seeding by passing -- no-seed.

2. Clone ethyca/fidesplus and sync dependencies from the repo root:

git clone https://github.com/ethyca/fidesplus.git
cd fidesplus
uv sync --all-groups

3. Open the repository root in your editor.

4. Build and run the stack in hot-reloading dev mode (slim backend + Admin UI + Privacy Center):

uv run nox -s 'build(slim)' 'dev(slim)' -- dev fides-admin-ui fides-privacy-center

Or use the command palette (Cmd+Shift+P) → Tasks: Run Task → Run Fidesplus & Fides (DEV) if your editor is configured with the repo tasks.

Project	Source directory	Local URL
fides package	src/fides/	n/a
fidesplus server	src/fidesplus/	http://localhost:8080
Admin UI	clients/admin-ui/	http://localhost:3000
Privacy Center	clients/privacy-center/	http://localhost:3001

Changes under src/fides, src/fidesplus, and the mounted client trees reload in the running containers.

WARNING: The client applications (fides-admin-ui and fides-privacy-center) are built on-demand when you first visit them in your web browser. You will see log messages in the Docker logs like the example below, and these can be quite slow to compile within the Docker container:

2025-08-18 13:26:09 admin-ui:dev:  ○ Compiling /login ...
2025-08-18 13:26:49 admin-ui:dev:  ✓ Compiled /login in 40.4s (6498 modules)

...as a result, you may find this approach too cumbersome to quickly work on the client applications. For instructions on how to build and run the UIs outside Docker, see clients/admin-ui/README.md and clients/privacy-center/README.md.

NOTE: When using uv run nox -s "dev(slim)" -- demo, the Admin UI and Privacy Center run from pre-built Docker images that do not hot-reload or rebuild automatically. To incorporate local changes, you must manually rebuild the images:

docker compose -f docker-compose.yml -f demo/docker-compose.demo.yml build fides-admin-ui fides-privacy-center

The Admin UI also runs a Next.js production build on container startup that takes a few minutes, so it won't be immediately accessible even after the container reports healthy — look for ✓ Ready in Xms in the container logs. This build step is memory-intensive — allocate at least 12 GB of memory to Docker Desktop.

Developing Fidesplus

This section is a brief overview of the available development commands and resources available in this repository. Remember that you can also rely on uv run nox (no args) and uv run nox -s usage -- <session_name> to learn more about the available commands.

Shell Setup (optional, zsh)

Tired of typing uv run nox? Install the shell integration once:

uv run nox -s shell_setup

This adds a nox wrapper function to your ~/.zshrc that runs uv run nox under the hood (so sessions always see a synced project venv), plus tab-completion for session names — nox -s <TAB> lists every available session. The integration lives in scripts/nox-uv.zsh, so improvements ship automatically with git pull; re-run shell_setup only if you move your clone.

Listing the Applications and their Information

The full Application list, including all related information, can be viewed with the following command:

uv run nox -s display_applications

Building the Applications

For the sake of simplicity, one can build all permutations of a specific Application by running:

uv run nox -s "build(<application>)" -- all

Real Example:

uv run nox -s "build(slim)" -- all

Or if you want to build a specific version:

uv run nox -s "build(<application>)" -- <build_name>

Real Example:

uv run nox -s "build(slim)" -- dev

Running the Applications

To run the default local dev stack (slim image + worker), use:

uv run nox -s dev

dev always starts the slim monorepo stack — it does not iterate over every application image. For other images (full fidesplus, masking, sidecar), use:

uv run nox -s "dev_app(<application>)"

dev(slim) remains a backward-compatible alias for dev.

By default, uv run nox -s dev starts services in the background (detached mode), waits for them to become healthy, and then automatically invokes the seed API to configure the environment. Once complete, it prints a "DEV ENVIRONMENT READY" banner with useful commands (tail logs, stop containers, re-seed, etc.) and exits — leaving the containers running.

The automatic seeding is best-effort: if 1Password CLI is not available or seeding fails for any reason, a warning is printed but the environment stays up and usable. To skip seeding entirely:

uv run nox -s dev -- no-seed

See Environment Setup & Seeding below for details on how seeding and .env validation work.

Due to dev no longer handling the build process, you can chain commands together for ease-of-use. The following command will build and then spin up the slim application.

Note: order matters here!

uv run nox -s "build(slim)" dev -- dev

---

Running Multiple Instances in Parallel (Git Worktrees)

When working across multiple git worktrees simultaneously — for example with agentic coding workflows or parallel feature branches — each worktree can run its own fully isolated stack with no port conflicts.

It just works from a worktree. When nox -s "dev(slim)" is run from a linked worktree (any path other than the main fidesplus/ checkout), it automatically detects the worktree, derives a unique WORKTREE_ID from the branch name, and assigns isolated ports and Docker resource names. No configuration required.

# In your main checkout — uses default ports (8080, 5432, 6379, …)
cd ~/<where-your-code-lives>/fidesplus
nox -s dev

# In a linked worktree — auto-detected, gets its own ports
cd ~/<where-your-code-lives>/.worktrees/fidesplus-my-feature
nox -s dev
# Logs will show:
#   WORKTREE_ID:          my-feature
#   PORT_OFFSET:          <hash-derived>
#   COMPOSE_PROJECT_NAME: fidesplus-my-feature
#   API port:             <offset port>

All Docker resources are fully isolated per worktree:

• Named volumes (fidesplus-my-feature_postgres) — each worktree has its own database
• Container names (fidesplus-slim-my-feature)
• Docker network (fidesplus-my-feature_default)
• Host ports — offset from defaults so nothing collides

Manual override (pin a specific slot or resolve a hash collision):

# in fidesplus/.env within the worktree
WORKTREE_ID=my-feature   # short slug; auto-detected from branch if omitted
PORT_OFFSET=100          # explicit slot; hash-derived from WORKTREE_ID if omitted

Slot reference table:

Slot	PORT_OFFSET	API	Admin UI	Privacy Center	DB	Redis
0 (main)	0	8080	3000	3001	5432	6379
1	100	8180	3100	3101	5532	6479
2	200	8280	3200	3201	5632	6579
3	300	8380	3300	3301	5732	6679

Each worktree seeding is independent — run nox -s seed in each worktree after first start.

---

Connecting to the Application

To connect to the running slim dev container:

uv run nox -s shell

For other application images:

uv run nox -s "shell_app(<application>)"

Working on Fides core (src/fides)

The Fidesplus server reloads when you change src/fidesplus or data/. Edits to src/fides/ are picked up the same way in dev: the repo is bind-mounted into the container and uvicorn reload watches the tree. No sibling fides checkout or FIDES_REPO_PATH is required.

Start the stack as usual:

uv run nox -s dev

You can use pdb.set_trace() in either package while the slim container is running. Fides core and Fidesplus changes ship together in one PR on this monorepo.

Environment Setup & Seeding

The default uv run nox -s dev session enforces a reference .env as the baseline local dev state. This ensures that all required environment variables — API keys, service credentials, feature flags — are present before seeding runs. The flow works as follows:

1Password CLI (prerequisite)

The 1Password CLI (op) is effectively a prerequisite for local Fidesplus development with seeding enabled. It is used for two things:

1. .env validation — On startup, the dev session fetches a reference .env from 1Password (vault: fidesplus-demo-seed, item: fidesplus-demo-dot-env) and validates that every variable it defines is present in your local environment (either in your shell or in the repo-root .env file).
2. Seed API invocation — The seed API reads secrets (e.g., Mailgun API keys, S3 credentials) from 1Password at runtime to configure the environment.

To install and configure:

# Install (macOS)
brew install 1password-cli

# Sign in to your Ethyca account
op signin

If you don't have 1PW CLI installed or configured, you can bypass seeding entirely:

uv run nox -s "dev(slim)" -- no-seed

For full details on seed prerequisites, troubleshooting, and the seed API, see src/fidesplus/seed/README.md.

Reference .env and validation

When uv run nox -s dev runs (without no-seed), it:

1. Fetches the reference .env from 1Password
2. Compares its variables against your local .env file and shell environment
3. If variables are missing, offers to pull the full reference .env from 1Password automatically (backing up your existing .env first)
4. If you decline or the pull fails, aborts with a clear error listing the missing variables

This ensures your local environment matches the expected configuration before containers start and seeding runs.

Note: After a uv run nox -s teardown (or docker compose down), the next uv run nox -s dev will re-seed the environment automatically, since the seed state is not persisted across container recreations.

Demo mode

The demo posarg starts a full demo environment that includes the Fidesplus backend, Admin UI, Privacy Center, sample databases, and the Cookie House sample application — everything needed for a working end-to-end demonstration. Unlike the default dev mode, demo mode attaches to container logs after startup (Ctrl+C to stop). See demo/README.md for demo infrastructure details and src/fidesplus/seed/README.md for seed configuration and troubleshooting.

# Full demo environment with seeding
uv run nox -s dev -- demo

# Demo environment without seeding
uv run nox -s dev -- demo no-seed

Quick reference

Command	Behavior
uv run nox -s dev	Start detached, validate .env, seed (best-effort), print ready banner
uv run nox -s dev -- no-seed	Start detached, skip all seeding and .env validation
uv run nox -s dev -- demo	Full demo environment, seed (required), attach to logs
uv run nox -s dev -- demo no-seed	Full demo environment, skip seeding, attach to logs
uv run nox -s "dev_app(fidesplus)"	Dev stack for a non-slim application image
uv run nox -s seed	Manually seed a running environment
uv run nox -s teardown	Stop containers
uv run nox -s teardown -- volumes	Stop containers and wipe all state

Running tests

Note: The pytest suite in Fidesplus currently relies on a BIGQUERY_CREDENTIALS environment variable being set in the test runtime. At the time of this writing, this should be set to the value of the BigQuery (bqextract-service-acct-2) service account private key vault entry found in OnePassword. Search for this vault in OnePassword, copy the BIGQUERY_CREDENTIALS environment variable value, and then add it to your local .env file like this:

BIGQUERY_CREDENTIALS='{ "type": "service_account", ...

To run the pytest test suite locally, you can either use the dedicated pytest nox session, or manually run pytest on your already-running local application container (see "Running the Application" section above)

Examples: Running fidesplus tests

Fidesplus backend tests use the pytest nox session (slim Docker stack, tests/fidesplus/):

uv run nox -s pytest

Merged fides core tests (tests/fides/) use separate sessions — build the CI image first, then run one group:

docker build -f Dockerfile --target slim_test -t ethyca/fidesplus:local-slim .
uv run nox -s "pytest_fides(api)"

To run only a specific set of tests, you can supply a test path as an argument. For example, to run just the classification tests:

uv run nox -s pytest tests/cls/

You can also pass individual test files or use any pytest supported selectors:

uv run nox -s pytest tests/api/test_plus.py

To rerun only the tests that failed in the previous pytest run (uses .pytest_cache on the bind-mounted repo):

uv run nox -s pytest_unit -- failed
# or
uv run nox -s pytest -- --lf
# or
PYTEST_LAST_FAILED=1 uv run nox -s "pytest_fides(api)"

The same failed / --lf shorthands work on other nox pytest sessions (pytest_local, pytest_fides, etc.).

Examples: Running tests inside the application container

There are two common approaches for running the backend test suite directly on your running Fidesplus backend container:

1. Run tests using a single command from your host machine

This approach runs tests via a one-liner without needing to manually shell into the container. For the default backend container (fidesplus-slim), you can do:

docker exec fidesplus-slim bash -c "pytest --no-cov"

To run a specific test file (example: only run tests/ops/schemas/test_partitioning_schema.py):

docker exec fidesplus-slim bash -c "pytest --no-cov tests/ops/schemas/test_partitioning_schema.py"

2. Run tests by shelling into the container

Alternatively, you can open a shell in the running backend container, and then invoke pytest directly within that shell. First, shell into the container

uv run nox -s "shell(<application>)"

Once inside the container, you can run any pytest command. For example:

pytest --no-cov

Or to run a specific test file or folder:

pytest --no-cov tests/api/test_plus.py

Configuration

Some of the additional functionality within Fidesplus requires additional, specific configuration. The following section explains what is required and how to go about setting up the various additional services available.

Dictionary service

Fidesplus can be pointed to a dictionary service (Fides Compass). There are three relevant variables:

FIDESPLUS__DICTIONARY__ENABLED=true
FIDESPLUS__DICTIONARY__DICTIONARY_SERVICE_URL=http://host.docker.internal:8081
FIDESPLUS__DICTIONARY__DICTIONARY_SERVICE_API_KEY=sample_api_key

These can alternatively be set in fides.toml:

[dictionary]
enabled = false
dictionary_service_url = "http://host.docker.internal:8081"
dictionary_service_api_key = "sample_api_key"

Hosted dictionary

To run against the hosted dictionary in our staging environment, you can find an API key in 1p under "Staging API Key".

The service URL for the environment is https://compass.dev.ethyca.com.

Local dictionary

Alternatively, you can run the dictionary service locally Once you have the service running, the default credentials provided in the fides.toml file should work.

Core (OSS) fides in this monorepo

Fides core lives in-tree at src/fides/ as part of the unified fidesplus Python package. You do not maintain a separate requirements.txt pin or a sibling ethyca/fides checkout for day-to-day development.

Typical workflow

1. Change code under src/fides/ and/or src/fidesplus/ on one branch.
2. Sync deps when pyproject.toml / uv.lock change: uv sync --all-groups.
3. Build and run locally: uv run nox -s "build(slim)" "dev(slim)" -- dev.
4. Run tests:
   • Fidesplus: uv run nox -s pytest
   • Fides core: docker build -f Dockerfile --target slim_test -t ethyca/fidesplus:local-slim . then uv run nox -s "pytest_fides(api)" (see Running tests).

CI uses the same monorepo layout (backend_checks.yml for tests/fides/, backend_pr_checks.yml for tests/fidesplus/).

Prerelease Docker images (optional)

For integration testing against published prerelease builds (not the in-tree workspace), use nox dev_prerelease / prod_prerelease targets and uv run nox -s "push(slim)" -- prerelease after tagging. See uv run nox -s usage -- push and the docker_fidesplus_prerelease workflow.

Note: Nightly double-edge images (Fidesplus main plus a separate Fides main checkout) are no longer built; both trees ship in one revision in this repo.