racf-generator 0.1.0

Synthetic RACF IRRDBU00 unload generator

racf-generator

A synthetic RACF IRRDBU00 unload generator.

racf_generator builds a fully synthetic, internally consistent enterprise (users, groups, datasets, general resources, started tasks, OMVS/TSO segments, ACLs) and serializes it into a RACF IRRDBU00 unload file — the same flat-file format produced by running IRRDBU00 on z/OS and downloading it via FTP/ASCII transfer.

No real RACF data is ever used. Everything is generated from a seed, so the same seed always produces byte-identical output.

The generator is schema-driven: it reads the record layout (offsets.json) from the mfpandas package at runtime instead of hardcoding field positions, so it stays compatible as that schema evolves. Output produced by this generator is verified (in this project's own test suite) to parse cleanly through the real mfpandas.IRRDBU00 parser.

Why

Useful anywhere you need a RACF unload but can't (or shouldn't) use a real one:

• unit/integration tests for tools that consume IRRDBU00 unloads
• CI pipelines
• documentation and screenshots
• demos and sales environments
• training environments
• benchmarking parsers against different data volumes

Install

pip install racf-generator

Requires Python 3.10+. Runtime dependencies are mfpandas (schema source) and Faker (realistic human names/emails/phone numbers) — both installed automatically.

Development install

If you've cloned this repository instead (to run the tests, use scripts/try_it.py, or work on the generator itself):

pip install -e ".[dev]"
# or: make install

Quick start

from racf_generator import IRRDBU00Generator

generator = IRRDBU00Generator(seed=42, profile="medium")
generator.write("dummy.unload")

That's it — dummy.unload is now a plain-text IRRDBU00 file you can hand to any RACF unload parser, including mfpandas.IRRDBU00:

from mfpandas import IRRDBU00

r = IRRDBU00(irrdbu00="dummy.unload")
r.parse_t()
print(r.errors)        # []
print(len(r._users))   # 500
print(len(r._groups))  # 60

Reusing a generator

.build() builds the in-memory enterprise model once and caches it — calling .write() multiple times (or .build() directly to inspect the model) never regenerates different data:

generator = IRRDBU00Generator(seed=42, profile="small")

model = generator.build()          # build once
print(len(model.users))            # 50
print(len(model.groups))           # 10

generator.write("first.unload")    # uses the cached model
generator.write("second.unload")   # same model, same bytes as first.unload

Determinism

The same seed always produces the same enterprise, down to the byte:

IRRDBU00Generator(seed=42, profile="small").write("a.unload")
IRRDBU00Generator(seed=42, profile="small").write("b.unload")
# a.unload and b.unload are byte-identical

IRRDBU00Generator(seed=7, profile="small").write("c.unload")
# c.unload differs — different seed, different (but still internally
# consistent) enterprise

Try it from a clone, no typing required

scripts/try_it.py is a convenience script for this repository — it's not part of the published PyPI package, so it only works if you've cloned the repo (see Development install above):

./scripts/try_it.py                                            # medium profile, verified
./scripts/try_it.py --profile small
./scripts/try_it.py --profile large --out /tmp/big.unload
./scripts/try_it.py --profile enterprise --users 200 --groups 25
./scripts/try_it.py --seed 7 --no-verify                       # skip the mfpandas round-trip

It writes the file, prints a summary (user/group/dataset/resource counts, ...), and by default parses the result back with the real mfpandas.IRRDBU00 to confirm it's valid.

Pointing at a different schema

By default the generator loads offsets.json from whatever version of mfpandas is installed. If you need to generate against a different/newer schema file without upgrading mfpandas, pass schema_path explicitly:

IRRDBU00Generator(seed=42, profile="small", schema_path="/path/to/offsets.json").write("dummy.unload")

Profiles

A profile just controls how big the generated enterprise is — how many users and groups get created. Everything else (naming conventions, privilege rarity, dataset/resource realism, referential integrity) is identical across profiles; only the scale changes.

Profile	Users	Groups	Typical use
small	50	10	fast unit tests, quick local checks
medium	500	60	demos, docs, "realistic-looking" screenshots
large	5000	400	load testing, parser benchmarking
enterprise	you choose	you choose	anything the three presets don't cover

small, medium, and large are fixed presets (see racf_generator/profiles.py) — you just name them:

IRRDBU00Generator(seed=42, profile="small").write("small.unload")
IRRDBU00Generator(seed=42, profile="medium").write("medium.unload")
IRRDBU00Generator(seed=42, profile="large").write("large.unload")

enterprise is the escape hatch: instead of picking a preset, you supply user_count and group_count yourself. Use it when you need a size the presets don't offer — say, 150 users for a specific test fixture, or 50,000 users to stress-test a downstream parser well beyond what large covers:

IRRDBU00Generator(
    seed=42,
    profile="enterprise",
    user_count=150,
    group_count=20,
).write("custom.unload")

profile="enterprise" requires both user_count and group_count — it raises ValueError if either is missing, since there's no sensible default size for "enterprise" the way there is for the three presets.

One constraint applies to every profile including enterprise: group_count must be at least 9. That's because 9 groups (SYS1, SECURITY, OPERATIONS, PAYROLL, HR, FINANCE, DBA, NETWORK, AUDIT) are a fixed catalog that always gets created first — they're what everything else (dataset ownership, resource access, started tasks) hangs off of. Any additional groups beyond those 9 are generated as subgroups underneath them (e.g. PAYROLL2, FINANCE3, ...).

Similarly, user_count must be large enough to fit the fixed catalog of service IDs (CICSA, DB2MSTR, MQM, TCPIP, WASUSER, OMVSKERN, JES2) and admin IDs (SECADM01, SYSADM1, AUDITOR1) — 10 users at minimum. Every user above that count is a Faker-generated human.

What gets generated

• Groups — a fixed department hierarchy (SYS1 → SECURITY, OPERATIONS, PAYROLL, HR, FINANCE, DBA, NETWORK, AUDIT), plus extra subgroups if group_count exceeds 9.
• Users — a mix of:
  • human users, with Faker-generated full names and RACF-style userids derived from them (Maria Johnson → MJOHNSON), plus DEPT/COSTCTR/EMAIL/ PHONE installation data (also Faker-driven)
  • service IDs (CICSA, DB2MSTR, MQM, TCPIP, WASUSER, OMVSKERN, JES2)
  • admin IDs (SECADM01, SYSADM1, AUDITOR1)
  • realistic privilege rarity: SPECIAL is rare, OPERATIONS is uncommon, AUDITOR is very rare, and SPECIAL/AUDITOR users belong to SECURITY
  • internally consistent lifecycle state: active, revoked (locked out, no recent logon), or expired (stale password, still logs on)
• Group connects — every user connected to their default group, plus some human users connected to a second group
• OMVS segments — unique UIDs, home paths, and shell programs (always present for service/admin users, probabilistic for human users)
• TSO segments — probabilistic, human users only
• Datasets — under SYS1.**, SYS2.**, PAYROLL.**, HR.**, FINANCE.**, USER.<userid>.**, DB2.<subsystem>.**, CICS.<region>.**, with realistic access patterns (e.g. PAYROLL group gets ALTER on payroll datasets, HR group gets UPDATE on HR datasets)
• General resources — FACILITY, OPERCMDS, XFACILIT, SURROGAT, PROGRAM, SERVER, EJBROLE profiles with group-based access
• Started tasks — CICSA, DB2MSTR, MQM, TCPIP, OMVS, JES2, each mapped to a real backing user

Every reference resolves — no dangling groups, users, datasets, or resources — and it's enforced by the test suite (tests/test_generator_integration.py).

Testing

pip install -e ".[dev]"
pytest tests/ -v
# or: make test

The suite includes a real round-trip: it writes a file with this generator and parses it back with the actual installed mfpandas.IRRDBU00, asserting zero parse errors.

Releasing

make check       # build sdist + wheel, validate with twine
make upload-test # dry run against TestPyPI
make upload      # publish to PyPI

upload/upload-test need PyPI credentials — either a ~/.pypirc, or TWINE_USERNAME=__token__ TWINE_PASSWORD=<pypi-api-token> in the environment. Run make help for the full target list.

Architecture

Two layers:

• racf_generator.model / racf_generator.builder — the enterprise domain model and the generators that populate it. No knowledge of the IRRDBU00 file format at all.
• racf_generator.writer — schema-driven serialization only. Loads offsets.json from mfpandas, maps domain objects to field dicts, and renders fixed-width lines. No business logic — if a record type is missing from the schema, it's skipped with a warning instead of crashing.