TianGong TIDAS Tools User Guide

English | 中文

This toolkit is used for conversion and validation of TianGong TIDAS and eILCD/ILCD data formats.

1. Introduction

This toolkit contains these independent tools:

• TIDAS and eILCD Data Format Conversion Tool
• External LCA Format Import Tool
• TIDAS and eILCD/ILCD Data Validation Tool
• TIDAS and eILCD Data Export Tool

2. TIDAS and eILCD Data Format Conversion Tool Usage

(1) Installation Instructions

# Install this toolkit
pip install tidas-tools

(2) Tool Functionalities

This tool supports mutual conversion between the following two data formats:

• TIDAS data format → eILCD data format (default mode)
• eILCD data format → TIDAS data format

(3) Command-line Arguments

(4) Usage Examples

# Convert TIDAS data to eILCD format
tidas-convert --input-dir <TIDAS_data_directory> --output-dir <eILCD_output_directory> --to-eilcd

# Convert eILCD data to TIDAS format
tidas-convert --input-dir <eILCD_data_directory> --output-dir <TIDAS_output_directory> --to-tidas

3. External LCA Format Import Tool Usage

(1) Current Scope

tidas-import is the staged entry point for importing external LCA formats into TIDAS and optionally ILCD/eILCD. The current implementation provides CLI dispatch, source format detection, .zolca rejection, machine-readable conversion reports, and minimal validated adapters for openLCA JSON-LD, EcoSpold 1, SimaPro CSV, EcoSpold 2, and openLCA process XLSX.

Current source status:

• openLCA JSON-LD zip/directory: minimal import to TIDAS and ILCD/eILCD
• EcoSpold 1 XML/zip: minimal import to TIDAS and ILCD/eILCD
• SimaPro CSV block format: minimal import to TIDAS and ILCD/eILCD
• EcoSpold 2 .spold/zip: minimal import to TIDAS and ILCD/eILCD
• openLCA process XLSX: minimal import to TIDAS and ILCD/eILCD

.zolca is intentionally out of scope.

Imported JSON-LD actors and sources are written as TIDAS contacts and sources. Source units from EcoSpold, SimaPro CSV, and process XLSX inputs are propagated into generated unit groups and flow properties when no explicit reference data is available.

When downstream AI/import workers need to handle each TIDAS process independently, the importer writes per-process bundles by default. The normal <output_directory>/tidas package is still written unchanged; the importer also writes <output_directory>/process-bundles/<process_uuid>/ folders containing the process JSON plus referenced flow, flow property, unit group, contact, and source JSON files. --process-bundles-dir <dir> overrides the bundle location, and --no-process-bundles disables bundle output.

The expert mapping CSV is disabled by default because large imports can produce very large field-level mapping files. Use --write-mapping-csv to write <output_directory>/mapping.csv.gz.

(2) Usage Example

tidas-import --input <source_file_or_dir> --output-dir <output_directory> --detect-only
tidas-import --input <source_file_or_dir> --output-dir <output_directory> --target both --validation-jobs 0
tidas-import --input <source_file_or_dir> --output-dir <output_directory> --no-process-bundles
tidas-import --input <source_file_or_dir> --output-dir <output_directory> --write-mapping-csv

4. Deterministic TIDAS/ILCD Release Packaging

tidas-release-tool consumes a finalized canonical TIDAS dataset tree plus its tiangong.release.canonical-dataset-index.v1. It never assigns UUIDs or versions. The tool validates exact transitive references, converts and validates ILCD, checks normalized semantic round-trip, and builds the two self-contained release profiles with byte-stable ZIP metadata.

tidas-release-tool validate-tidas --input-dir <canonical-tidas-dir>
tidas-release-tool convert-ilcd --input-dir <canonical-tidas-dir> --output-dir <ilcd-dir>
tidas-release-tool validate-ilcd --input-dir <ilcd-dir>
tidas-release-tool semantic-roundtrip --tidas-dir <canonical-tidas-dir> --ilcd-dir <ilcd-dir>
tidas-release-tool build-packages \
  --tidas-dir <canonical-tidas-dir> \
  --ilcd-dir <ilcd-dir> \
  --dataset-index <canonical-dataset-index.json> \
  --output-dir <package-dir>

The package command emits canonical TIDAS and derived ILCD variants for unit-process-full-closure.v1 and standalone-lifecyclemodel-result-full-closure.v1. Missing UUID/version references fail closed. JSON stdout is stable machine output; --report <path> also persists the same result.

5. TIDAS and eILCD/ILCD Data Validation Tool Usage

(1) Tool Functionalities

This tool validates whether TIDAS JSON data or eILCD/ILCD XML data complies with the packaged schema standards. TIDAS JSON validation uses a compiled schema fast path and falls back to complete error collection when a schema issue is found.

(2) Command-line Arguments

(3) Usage Example

# Validate TIDAS data format
tidas-validate --input-dir <TIDAS_data_directory> --data-format tidas

# Validate eILCD/ILCD XML data format
tidas-validate --input-dir <eILCD_data_directory> --data-format ilcd

# Validate large packages with all CPU cores
tidas-validate --input-dir <TIDAS_data_directory> --data-format tidas --jobs 0

# Inspect the reproducibility handshake used by closure-preflight workers
tidas-validate --describe --format json

# Stream deterministic validation evidence for exactly the manifest documents
tidas-validate --protocol document-validation-batch.v1 \
  --input-dir <batch_root> \
  --input-manifest <document-validation-batch.v1.jsonl>

The batch protocol treats document issues as a completed scan: it emits one issue event per finding, a final summary/hash event, and exits zero. Unsafe paths, duplicate keys/paths, symlinks, content-hash drift, malformed input, or missing execution proof are protocol failures. Reference target existence and database visibility are intentionally outside this document-validation layer.

6. TIDAS Export Tool Documentation

(1) Tool Functionalities

This tool exports data records in either TIDAS or eILCD format. It also optionally downloads supplementary files and bundles them into a final zip archive.

(2) Command-line Arguments and Environment Variables

Credentials can also be set via environment variables (defaults to the .env file in the current directory):

DB_USER=
DB_PASSWORD=
DB_HOST=
DB_PORT=5432
DB_NAME=postgres
AWS_REGION=
AWS_ENDPOINT=

(3) Usage Example

# Export records to TIDAS format and produce a ZIP archive.
tidas-export -i <TIDAS_input_directory> -z <TIDAS_ZIP_File> --to-tidas

# Export records to eILCD format without downloading supplementary files
tidas-export -z <eILCD_ZIP_File> --to-eilcd --skip-external-docs

7. Log File Information

Both data conversion and validation tools will automatically generate execution logs. The log file name is:

tidas-{function_name}.log

8. Development Environment Setup and Contribution Guide

If you wish to participate in development, you can set up your environment following these steps:

(1) Ubuntu System Environment Preparation

# Update repositories and install software management tools
sudo apt update
sudo apt install software-properties-common

# Add the official PPA repository for the latest Python version and install Python 3.12
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install -y python3.12

# Install necessary dependency packages
sudo apt install libxml2-dev libxslt-dev
sudo apt-get install build-essential python3-dev

# Upgrade software packages on the system
sudo apt upgrade

(2) Manage Python Environment with uv

# Install uv (if not already available)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Synchronize dependencies (including development tools)
uv sync --dev

# Activate the virtual environment created by uv (optional)
source .venv/bin/activate

# Run project commands without activating the environment
uv run python src/tidas_tools/convert.py --help

9. Code Standards and Testing

(1) Code Formatting Tool (black recommended)

# Automatically format code using black
uv run black .

(2) Testing Instructions

To test data conversion and validation functionalities, run the following commands:

# Test converting TIDAS data to eILCD format
uv run python src/tidas_tools/convert.py -i <TIDAS_data_directory> -o <eILCD_data_directory> --to-eilcd

# Test converting eILCD data to TIDAS format
uv run python src/tidas_tools/convert.py --input-dir <eILCD_data_directory> --output-dir <TIDAS_data_directory> --to-tidas

# Test external LCA format detection
uv run python src/tidas_tools/import_lca/cli.py --input <source_file_or_dir> --output-dir <output_directory> --detect-only

# Test TIDAS and eILCD/ILCD data validation functionality
# Execute automated tests
uv run pytest

# Validate TIDAS data
uv run python src/tidas_tools/validate.py -i <TIDAS_data_directory> --data-format tidas

# Validate eILCD/ILCD data
uv run python src/tidas_tools/validate.py -i <eILCD_data_directory> --data-format ilcd

10. Automatic Building and Publishing (CI/CD)

This project supports automatic building and publishing. When you push a git tag named with the v<version> format to the repository, it will trigger the workflow automatically. For example:

# List existing tags
git tag

# Create a new tag (e.g., version v0.0.1)
git tag v0.0.1

# Push the newly created tag to the remote repository to trigger automatic workflow
git push origin v0.0.1

Schema and methodology updates on main can also trigger a cross-repository SDK sync into tiangong-lca/tidas-sdk through .github/workflows/dispatch-tidas-sdk-sync.yml.

That automation requires the repository secret TIDAS_SDK_AUTOMATION_TOKEN.

11. Contribution

We welcome your contributions! You can participate in the project by submitting issues or pull requests.