Skip to main content

DCS SDK v2.2.3

SDK for DataChecks

Installation

Python version >=3.10,<3.13

$ pip install dcs-sdk[all-dbs]

Supported Databases

Availability Status

Database Code Name Supported
PostgreSQL postgres ✅
Snowflake snowflake ✅
Trino trino ✅
Presto presto ✅
Databricks databricks ✅
Oracle oracle ✅
MSSQL mssql ✅
MySQL mysql ✅
SAP Sybase IQ/ASE sybase ✅
SAP HANA sap_hana ✅
IBM DB2 db2 ✅
Vertica vertica ✅
ClickHouse clickhouse ✅
File file ✅
BigQuery bigquery ✅

Available Commands

Option Short Option Required Default Description Example
--config-path -C Yes None Specify the file path for the configuration dcs-sdk run --config-path config.yaml --compare comp_name
--compare Yes None Run only specific comparison using comparison name dcs-sdk run --config-path config.yaml --compare comp_name
--save-json -j No False Save the data into a JSON file dcs-sdk run --config-path config.yaml --compare comp_name --save-json
--json-path -jp No dcs_report.json Specify the file path for JSON file dcs-sdk run --config-path config.yaml --compare comp_name --save-json --json-path ouput.json
--stats No False Print stats about data diff dcs-sdk run --config-path config.yaml --compare comp_name --stats
--url No None Specify url to send data to server dcs-sdk run --config-path config.yaml --compare comp_name --url=https://comapre/send/data
--html-report No False Save table as HTML dcs-sdk run --config-path config.yaml --compare comp_name --html-report
--report-path No dcs_report.html Specify the file path for HTML report dcs-sdk run --config-path config.yaml --compare comp_name --html-report --report-path table.html
--table No False Display Comparison in table format dcs-sdk run --config-path config.yaml --compare comp_name --html-report --report-path table.html --table

Example Command [CLI]

$ dcs-sdk --version

$ dcs-sdk --help

$ dcs-sdk run -C example.yaml --compare comparison_one --stats -j -jp output.json --html-report --report-path result.html --table --url=https://comapre/send/data

File Comparisons

dcs-sdk supports file-backed comparisons through DuckDB for:

  • .csv
  • .parquet
  • mixed-format comparisons such as csv ↔ parquet

Supported file datasource types:

  • file
  • azure_blob

Notes:

  • File paths must point to concrete .csv or .parquet files or globs.
  • Query-backed file comparisons are supported. When source.query or target.query is provided, the SDK loads the file into DuckDB and compares against the filtered/projected query view.

Local File Example

data_sources:
  - name: source_file
    type: file
    file_path: sample_data/parquet/one_source.parquet

  - name: target_file
    type: file
    file_path: sample_data/parquet/two_target.parquet

comparisons:
  parquet_file_diff:
    source:
      data_source: source_file
      table: one_source
    target:
      data_source: target_file
      table: two_target
    key_columns: [id]
    columns: [customer_name, status, amount, region]

Run it with:

dcs-sdk run -C parquet_file_comparison.yaml --compare parquet_file_diff --stats

Databricks Query-Backed Comparisons

Databricks comparisons can use either:

  • a table name
  • a SQL query

For Parquet files stored on Databricks, use a query with read_files(...).

Databricks Table vs Parquet Example

data_sources:
  - name: databricks_demo
    type: databricks
    connection:
      host: your-workspace.cloud.databricks.com
      port: 443
      http_path: /sql/1.0/warehouses/your-warehouse
      access_token: ${DATABRICKS_TOKEN}
      catalog: dcs_demo_databricks
      schema: source
    temporary_schema: temp_schema

comparisons:
  databricks_table_vs_parquet:
    source:
      data_source: databricks_demo
      table: source_table
    target:
      data_source: databricks_demo
      query: |
        SELECT *
        FROM read_files(
          '/Volumes/dcs_demo_databricks/source/dcs-test-volumne/two_target.parquet',
          format => 'parquet'
        )
      view_name: datachecks_target_file
      materialization_type: table
    key_columns: [id]
    columns: [customer_name, status, amount, region]

Notes:

  • Query-backed Databricks comparisons require temporary_schema.
  • Generated temp views/tables use the datachecks_ prefix.
  • Prefer Unity Catalog volume paths such as /Volumes/... for Databricks file queries.
  • Legacy DBFS root paths such as dbfs:/raw/... are not the recommended path for this flow.

Release files for dcs-sdk 2.2.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dcs-sdk 2.2.3
File Size Uploaded
dcs_sdk-2.2.3.tar.gz 1.4 MB Details

Built distribution (wheel)

Table of built distributions (wheels) for dcs-sdk 2.2.3
File Interpreter ABI Platform
dcs_sdk-2.2.3-py3-none-any.whl Python 3 none any Details

Total release size: 3.0 MB

Release files / dcs_sdk-2.2.3.tar.gz

Download URL dcs_sdk-2.2.3.tar.gz
Size 1.4 MB
Tags Source
SHA-256 checksum
How to use checksums
3b0f71117a8d61ec983cbe1243f2a817ed7626f7e722ebe1f91a7f3864d4800e
BLAKE2b-256 checksum
How to use checksums
4b927c0cc6625b40fb0561522113251be6052a6cea30bf3710481958b05bc244
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.5.1 CPython/3.12.14 Linux/6.17.0-1022-azure

Release files / dcs_sdk-2.2.3-py3-none-any.whl

Download URL dcs_sdk-2.2.3-py3-none-any.whl
Size 1.6 MB
Tags Python 3
SHA-256 checksum
How to use checksums
cec1f4b35a982d3bb9cef2c2ce1896a294b5d06f65fe3a9e834fc637068c8641
BLAKE2b-256 checksum
How to use checksums
c412ab77e5da5b4d0f477f4cf00941eaad9f2ab35e12eb71384dac5c8eb3789d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.5.1 CPython/3.12.14 Linux/6.17.0-1022-azure

Release history Release notifications | RSS feed

This release

2.2.3 This release

2 release files

2.2.2

2 release files

2.2.1

2 release files

2.2.0

2 release files

2.1.0

2 release files

2.0.0

2 release files

1.9.9

2 release files

1.9.8

2 release files

1.9.7

2 release files

1.9.6

2 release files

1.9.5

2 release files

1.9.4

2 release files

1.9.3

2 release files

1.9.2

2 release files

1.9.1

2 release files

1.9.0

2 release files

1.8.9

2 release files

1.8.8

2 release files

1.8.7

2 release files

1.8.6

2 release files

1.8.5

2 release files

1.8.4

2 release files

1.8.3

2 release files

1.8.2

2 release files

1.8.1

2 release files

1.8.0

2 release files

1.7.9

2 release files

1.7.8

2 release files

1.7.7

2 release files

1.7.6

2 release files

1.7.5

2 release files

1.7.4

2 release files

1.7.3

2 release files

1.7.2

2 release files

1.7.1

2 release files

1.7.0

2 release files

1.6.9

2 release files

1.6.8

2 release files

1.6.7

2 release files

1.6.6

2 release files

1.6.5

2 release files

1.6.4

2 release files

1.6.3

2 release files

1.6.2

2 release files

1.6.1

2 release files

1.6.0

2 release files

1.5.9

2 release files

1.5.8

2 release files

1.5.7

2 release files

1.5.6

2 release files

1.5.5

2 release files

1.5.4

2 release files

1.5.3

2 release files

1.5.2

2 release files

1.5.1

2 release files

1.5.0

2 release files

1.4.9

2 release files

1.4.8

2 release files

1.4.7

2 release files

1.4.6

2 release files

1.4.5

2 release files

1.4.4

2 release files

1.4.3

2 release files

1.4.2

2 release files

1.4.1

2 release files

1.4.0

2 release files

1.3.9

2 release files

1.3.8

2 release files

1.3.7

2 release files

1.3.6

2 release files

1.3.5

2 release files

1.3.4

2 release files

1.3.3

2 release files

1.3.2

2 release files

1.3.1

2 release files

1.3.0

2 release files

1.2.1

2 release files

1.2.0

2 release files

1.1.0

2 release files

1.0.0

2 release files

0.3.3

2 release files

0.3.2

2 release files

0.3.1

2 release files

0.3.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page