BBI Assessment

bbi-assess is a client-hosted, read-only assessment package. It collects screened evidence from selected repositories, AWS, PostgreSQL, Snowflake, approved documents, and owner answers. Deterministic rules produce foundation findings, an investigation screen for six AI use types, and readiness gates only for owner-confirmed use cases. bbi-bucky is a dependency and powers optional, grounded follow-up through the generated assessment host and the bucky CLI. Scanning client repositories never installs Bucky into them or edits them.

Install and run

Install the core package and only the connectors required for the approved scope. The aws extra also supports AWS CodeCommit; analysis enables deeper static SQL and frontend parsing. The all extra includes the optional DIP adapter.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade 'bbi-assess[aws,analysis,openlineage]==0.7.9'
bbi-assess init
# Review assess.yaml and bucky_policy.yaml; select only approved sources.
bbi-assess scope aws add --id default --profile default --regions all --account-id 123456789012
bbi-assess doctor
bbi-assess run                 # prints a scope preview and asks once before collection
# bbi-assess run --yes         # non-interactive trial after reviewing the config
# bbi-assess run --smart       # same flow, plus screened Bucky interpretation
# bbi-assess report --smart    # enrich the most recent saved snapshot

For GitHub, PostgreSQL, or Snowflake collection, add their named extras to the install command and configure their scopes in assess.yaml. Run bbi-assess serve --port 8765 for the local assessment host and Bucky widget. Configure model.provider, model.name, and model.egress before asking bucky questions or using --smart; without model egress, the CLI refuses natural-language questions. Bucky answers questions grounded in the saved, permitted assessment evidence. It cannot answer arbitrary questions or fill gaps that the collection did not establish.

init creates one assess.yaml, a masking policy, owner-answer template, and a package-owned FastAPI/Bucky host under .bbi_assess/host. doctor previews scopes, access, and the model destination plus categories of screened material that could be sent. run collects a new immutable snapshot, assesses it with rubric 4, and renders report schema 6. The separate collect, assess, and report commands let a reviewer inspect each stage. Prior snapshots and report files are retained without reinterpretation.

The bundle under .bbi_assess/reports/<snapshot>-<digest>/ contains index.html, a compact executive.pdf, assessment.html with foundation and AI-use explanations, lineage.html with runtime job/data-flow evidence, report.json, searchable evidence.html, a criteria appendix, component reports, flows.html, and linked .drawio/SVG diagrams. Start at index.html: read the current decision, observed AI/data/delivery signals, candidate AI opportunities, implications of collected evidence, and ordered preparation work. Follow its links to foundation details, component or repository dossiers, evidence citations, and technical diagrams. Each selected repository gets a dossier with its candidate role, detected entry points, data operations, and cited paths. The data platform uses one linked asset register rather than a page per dataset. The six-profile matrix lives in the detailed assessment; it says Blocked only for a recorded failed answer or conflicting evidence, Gaps for missing proof in an accessible source, and Not assessed for sources outside or inaccessible to the scope. It never grants readiness. Evidence references such as AWS-NET-ABC123 link to the register.

The optional analysis extra uses SQLGlot to parse SQL table reads and writes. On Python 3.13 and earlier it also installs the official Tree-sitter TypeScript and JavaScript grammars to locate calls. The Python 3.14 trial exposed a native parser crash on a large TSX module, so 3.14 uses a bounded lexical fallback for literal frontend API calls until the binding is stable. Python uses its built-in AST to resolve FastAPI routes, dependency aliases, method calls, selected data-client construction, and bounded propagation through connection-pool returns and arguments, including asyncpg pools. Bucky's screened GitIngest wrapper inventories approved source without changing the client repository; its count is not treated as a call graph. Exact API paths and cloud resource names create candidate links, not claims that services communicated at runtime. A two-part schema.table SQL reference can link to one unique Glue table in the same account. A three-part catalog.schema.table reference can link only when the repository explicitly declares that Spark catalog alias as Glue and the table match is unique; REST and Hive aliases are not treated as Glue. Source bodies and query text are not stored in the assessment graph.

Scope and evidence

assess.yaml selects local_repositories, exact github.repositories, codecommit.repositories (all or named repositories), named aws.targets, documents paths or globs, data_sources, owner_answers, domains, profiling limits, model egress, and report audience. The generated bucky_policy.yaml adds exclusions and masking. GitHub uses GITHUB_TOKEN from the process environment; PostgreSQL uses a named DSN environment variable; Snowflake uses a selected connection alias. AWS targets use local profiles and optionally an assumed role. --regions all discovers enabled regions, and --account-id guards against collecting from the wrong account. These credentials are never written into reports. For CodeCommit, the collector lists repositories in the selected regions and reads eligible files from the selected branch into a temporary screened copy; each repository is capped at 1,000 files of at most 100 KB. Repositories with no usable branch and files beyond these limits are reported as coverage gaps. The AWS collector reads metadata only: VPCs, subnets, security groups, EC2 instances, EKS/ECS clusters, load balancers, RDS instances, Lambda functions, Glue database and table schemas, CloudTrail trails, S3 bucket names, IAM role names, and selected CodePipeline/CodeBuild topology. Glue columns become individually cited data_column evidence (bounded to 30 per table) for candidate term matching; schema metadata cannot approve a business definition. Pipeline source and build configuration can provide candidate links between a CodeCommit repository and an AWS deployment target; this does not prove a pipeline ran successfully. It does not inspect S3 object bodies, instance storage, IAM policy documents, database rows, or Kubernetes workloads. Database connectors issue bounded read-only metadata and aggregate queries without exporting raw rows. Missing permissions become coverage gaps.

Approved TXT, Markdown, JSON, CSV, PDF, DOCX, PPTX, and XLSX files are locally extracted with section/page/slide/sheet citations, size limits, and screening. Deterministic relationship claims remain documented or disputed until reviewed; a matching resource identifier alone does not approve a business definition. collect --with-llm-claims can draft additional proposed claims from screened excerpts only when model.egress.enabled: true and doc_excerpts is allowed. Model claims cannot pass gates.

Runtime lineage from OpenLineage

OpenLineage is an existing event standard, not a new client-side scanner. If the client already emits OpenLineage RunEvents from Airflow, dbt, Spark, or another producer, export selected local .json or .jsonl event files and add them to assess.yaml:

openlineage:
  files: [lineage/approved-events.jsonl]

The assessment reads those files locally. It joins START and COMPLETE events by run ID, records completed job inputs and outputs, and cites the run events in lineage.html and an editable diagram. It discards event facets, environment variables, payload bodies, and raw run IDs. Failed or incomplete runs are recorded without asserting a completed data flow. File selection is explicit and bounded; no remote OpenLineage or Marquez service is queried. bbi-assess[openlineage] installs the official Python producer client if needed, while reading existing exports needs no extra dependency. An exact S3 bucket name or SQL table name creates only a candidate cross-source match. A completed event is evidence of emitted lineage metadata, not proof of data quality, legal rights, or a production deployment.

bbi-assess questionnaire --output owner-questions.xlsx
# Owner fills answers, 0–4 ratings, and reviewer names.
bbi-assess ingest-answers --file owner-questions.xlsx
bbi-assess collect
bbi-assess diff <older-snapshot-id> <newer-snapshot-id>

The questionnaire focuses on unresolved foundation criteria and document disputes. An imported answer needs a reviewer and rating. A rating of zero is a recorded gap; a positive assertion does not by itself verify a control. diff compares evidence content, relationships, readiness gates, and work packages while ignoring collection timestamps.

AI use cases and Bucky

With no selected use case, the package pre-screens semantic layer/text-to-SQL, knowledge chatbot, workflow agent, engineering accelerator, business decision/automation, and predictive AI as investigations only. Owner-selected cases get common and profile-specific gates. Hybrids inherit the union of requirements. Unknown or disputed mandatory proof blocks progression; production release is a separate implemented-system decision.

bbi-assess use-case questions
bbi-assess use-case template --output case.json
# Have the owner complete the form.
bbi-assess use-case add --file case.json
bbi-assess collect
bbi-assess explain --use-case <id>

For a model, set model.provider, model.name, model.region for Bedrock, and explicitly review model.egress.enabled and model.egress.allow in assess.yaml. bbi-assess init generates the host; bbi-assess serve --port 8765 starts it locally. bucky opens the terminal analyst; bucky --question "..." asks one question. The CLI prints screened text as provisional while streaming, hides citation trailers, and validates evidence IDs at completion. It corrects an unverified response rather than saving it as verified. run --smart and report --smart draft a marked executive paragraph, foundation-by-foundation and AI-use explanations, and repository-purpose drafts from bounded screened metadata. With explicit doc_excerpts egress permission, the optional repository reader can also use selected masked README prose lines and validates any returned source quote against the cited line. The former collects a new snapshot; the latter reuses a saved snapshot. The older --with-llm spelling remains an alias. Every accepted section cites supplied evidence and is marked proposed. The model cannot change relationships, rules, scores, or readiness decisions. No model call occurs in the default deterministic assessment. Bedrock is an AWS-managed model endpoint, but bbi-assess still runs on the machine where the CLI is launched. Its Bedrock endpoint region is independent of the AWS regions being assessed; doctor prints the configured destination. Selecting Bedrock does not keep a locally generated report entirely inside the client VPC.

Bucky's assessment tools are allowlisted reads over saved, screened evidence and snapshot history. It does not mutate client cloud, repositories, or databases. Its local MCP client starts the generated stdio tool server on demand. The assessment host and widget are installed only within .bbi_assess/host by package commands.

Internal and DIP boundaries

Set report.audience: internal to include draft Blackbuck capability matches. The catalog seed needs product-owner review before client distribution; client reports focus on the verified gap and preparation work. dip-export creates a screened, project-scoped local artifact with the same assessment and snapshot IDs. dip-serve exposes authenticated, read-only MCP capabilities for a later DIP integration. It does not register a release, publish an artifact, or provide a remote deployment gateway.

Verification and limits

Run python -m pytest tests. Tests cover pre-screen states, documents and contradictions, masking, owner-answer round trips, snapshot diffs, source-to-data paths, report links and diagrams, Bucky streaming, and prior report compatibility. A full local trial collected read-only AWS metadata across the configured regions and screened source from 47 CodeCommit repositories. Provider permissions must still be validated in each client environment. Static analysis cannot prove a deployed or executed flow. OpenLineage events, OpenTelemetry traces, runtime configuration, and owner review are possible sources for stronger evidence. The local service binds to loopback and needs an identity and authorization gateway before any network deployment. A live model call with this trial's AWS evidence has not been run because the session's approval review rejected that egress.