Skip to main content

AutoData Python Client

Official Python SDK for the AutoData ML data preparation pipeline API.

Installation

pip install datatoolpack

Or install from source:

git clone https://github.com/datatoolpack/datatoolpack
cd datatoolpack
pip install .

Quick Start

from datatoolpack import AutoDataClient

with AutoDataClient(api_key="dtpk_YOUR_API_KEY") as client:
    result = client.process(
        file_path="data.csv",
        target_columns=["price"],
        output_rows=10000,
    )
    print(result["files"])

Get your API key from the AutoData dashboard → API Keys tab.


Authentication

with AutoDataClient(api_key="dtpk_YOUR_API_KEY") as client:
    ...

Access Code (passcode)

If you have an access code instead of an API key:

with AutoDataClient(passcode="123456789012") as client:
    ...

Environment Variables

Set environment variables to avoid hardcoding credentials:

export AUTODATA_API_KEY="dtpk_YOUR_API_KEY"
# or
export AUTODATA_PASSCODE="123456789012"
# optional
export AUTODATA_BASE_URL="https://autodata.datatoolpack.com"
# No need to pass api_key — reads from AUTODATA_API_KEY automatically
with AutoDataClient() as client:
    result = client.process("data.csv", target_columns="price")

Supported File Formats

Format Extensions
CSV .csv
Excel .xlsx, .xls
Parquet .parquet
JSON .json
Feather .feather
ORC .orc

Reference

AutoDataClient(api_key, passcode, base_url, timeout, max_retries)

Parameter Type Default Env Variable Description
api_key str None AUTODATA_API_KEY API key starting with dtpk_
passcode str None AUTODATA_PASSCODE Access code (alternative to api_key)
base_url str "https://autodata.datatoolpack.com" AUTODATA_BASE_URL Server URL (no trailing slash)
timeout int 120 Request timeout in seconds
max_retries int 3 Auto-retries for 429 / 502 / 503 / 504

Either api_key or passcode is required (via argument or env variable).

Implements context manager — use with with to auto-close connections:

with AutoDataClient(api_key="dtpk_...") as client:
    ...
# session is closed automatically

Or close manually:

client = AutoDataClient(api_key="dtpk_...")
# ... use client ...
client.close()

Core Pipeline

client.process(...) — Upload & run pipeline

result = client.process(
    file_path="data.csv",           # Path to input file (CSV, XLSX, Parquet, ...)
    target_columns=["price"],       # y-column(s) for ML
    output_rows=10000,              # Target row count in the synthetic output (default 10000)
    tools={                         # Toggle pipeline steps (all optional)
        "anomaly": False,           # Anomaly detection (off by default)
        "dtc": True,                # Data Type Conversion
        "mdh": True,                # Missing Data Handler
        "dor": False,               # Dimensionality Reduction (off by default)
        "cds": True,                # Column Scaling
        "dsm": True,                # Data Split Manager
        "dsg": True,                # Synthetic Data Generator
    },
    advanced_params={               # Fine-grained parameters (all optional)
        "excluded_columns": ["id"], # Columns to drop before processing
        "text_mode": 0,             # 0=none, 1=neural, 2=tfidf
        "text_cleaning": True,      # Clean text before encoding
        "zscore_limit": 3.0,        # Z-score outlier threshold
        "dsg_mode": "copula",       # "copula" or "gan"
        "similarity_p": 95,         # Similarity percentile for DSG
    },
    wait=True,                      # Block until complete (default True)
    poll_interval=2,                # Status poll interval in seconds
    download_path="./outputs/",     # Where to save files (default auto)
    auto_download=True,             # Set False to skip download when wait=True
    output_preferences=["dsg.csv"], # Which files to download (default all)
    compressed=True,                # Download as ZIP (default True)
)

Returns a dict:

{
    "session_id": "abc123...",
    "status": "completed",
    "files": [
        {"name": "dsg.csv", "url": "/download/.../dsg.csv", "size": 2097152, "description": "..."},
        {"name": "dsm_train.csv", ...},
    ],
    "row_count": 10000,
    "duration_seconds": 42.1,
}

Set wait=False to get back immediately with just session_id and status:

result = client.process(file_path="data.csv", target_columns="price", wait=False)
session_id = result["session_id"]

client.get_status(session_id) — Poll progress

status = client.get_status(session_id)
# {
#   "status": "running",           # queued | running | completed | error | cancelled
#   "message": "Running MDH...",
#   "current_step": 3,
#   "total_steps": 6,
#   "progress_percent": 50,
#   "duration_seconds": 15.3,
# }

client.get_result(session_id) — Fetch completed results

result = client.get_result(session_id)
# {"status": "completed", "files": [...], "row_count": ..., "duration_seconds": ...}

client.wait_for_completion(session_id, poll_interval) — Block until done

result = client.wait_for_completion(session_id, poll_interval=3)

Prints live progress to stdout. Raises AutoDataError if processing fails.


client.cancel(session_id) — Cancel a running job

cancelled = client.cancel(session_id)  # True if acknowledged

client.retry_session(session_id) — Retry a failed pipeline

Retry from the last checkpoint — no need to re-upload or re-configure:

result = client.retry_session(session_id="failed-session-id")
# {'success': True, 'session_id': '...', 'message': 'Retrying from last checkpoint'}

client.download_results(session_id, ...) — Download output files

path = client.download_results(
    session_id,
    download_path="./my_outputs/",      # Directory to save into
    output_preferences=["dsg.csv"],     # Specific files only (None = all)
    compressed=True,                    # ZIP download (default) or individual files
)
print(f"Saved to {path}")

client.download_file(url, output_path) — Download a single file

client.download_file("/api/v1/download/abc123.../dsg.csv", "dsg.csv")

Feature Selection (F4)

Rank a completed session's columns by predictive importance — without re-running the pipeline. Handy for trimming wide datasets before training.

client.recommend_features(session_id, target_columns, methods, top_k)

ranking = client.recommend_features(
    session_id="abc123...",            # a completed session you own
    target_columns=["price"],          # rank features against these target(s)
    methods=["mutual_information"],    # one or more methods (see below); default ["mutual_information"]
    top_k=20,                          # number of top features to return (default 20)
)
# {
#   "ranking":    [{"feature": "sqft", "score": 0.81}, ...],   # combined across methods
#   "per_method": {"mutual_information": [...], "shap_importance": [...]},
#   "warnings":   [...],
# }

Available methods: variance_threshold, correlation_pruning, mutual_information, rfe, shap_importance, pca. Pass several to blend their rankings, e.g. methods=["mutual_information", "shap_importance"]. (SHAP-based ranking uses a fast LightGBM model on the server; it degrades gracefully to mutual_information if those libraries are unavailable.)


Inference & Retraining (v0.11.0+)

Apply a completed session's fitted transforms to new data — the missing half of the ML lifecycle: train once, then score fresh production rows with exactly the same encoding, imputation and scaling.

client.infer(...) — Score new data

Lightweight pipeline (Anomaly Detection → DTC → MDH → CDS). Outputs features.csv (+ y_columns.csv if targets are present in the new file).

result = client.infer(
    original_session_id="5d3bc0f0-...",   # a completed training session you own
    file_path="new_rows.csv",             # columns must match the training data
    download_path="./scored",             # optional: save outputs locally
    return_dataframes=True,               # optional: get pandas DataFrames back
)
features_df = result["dataframes"]["features"]

client.retrain(...) — Refresh a training set with new rows

Full pipeline including Split → CDS → DSM → DSG; produces retraining_output.csv (features + targets combined).

result = client.retrain(
    original_session_id="5d3bc0f0-...",
    file_path="new_rows.csv",
    run_dsg=True, output_rows=20000,      # synthetic augmentation target
    download_path="./retrained",
)

Both calls are synchronous (the response arrives when processing ends; default per-request timeout is ≥ 900 s — raise timeout= for very large files), require the original session to be completed and owned by you, and are free of credit charges in v1.


API Keys & Usage

client.list_keys() — List API keys

keys = client.list_keys()
# [{"id": "...", "name": "My Key", "prefix": "dtpk_abc123", "created_at": "..."}]

client.get_usage() — Usage statistics

usage = client.get_usage()
# {
#   "daily_credits_used": 500,
#   "daily_credit_limit": 10000,
#   "daily_remaining": 9500,
#   "lifetime_credits_used": 12340,
#   "lifetime_credit_limit": 1000000,
#   "lifetime_remaining": 987660,
#   "daily_request_count": 3,
#   "last_used_at": "2026-04-12T10:30:00Z",
# }

Connectors

Read data directly from databases and cloud storage instead of uploading files.

Supported connector types

Type Description Required secrets
sql PostgreSQL, MySQL, SQL Server connection_string
snowflake Snowflake Data Cloud account, user, password, database, schema, warehouse
bigquery Google BigQuery credentials_json, project_id, dataset
mongodb MongoDB / Atlas connection_string, database
s3 Amazon S3 bucket, access_key_id, secret_access_key, region
gcs Google Cloud Storage bucket, credentials_json
databricks Databricks Lakehouse host, token, catalog, schema
delta Delta Lake path (+ cloud credentials if remote)
fabric Microsoft Fabric workspace_id, lakehouse_id, tenant_id, client_id, client_secret
kafka Apache Kafka bootstrap_servers, topic, group_id
kinesis Amazon Kinesis stream_name, region, access_key_id, secret_access_key

More connectors: AutoData supports 30+ source types. Beyond the common ones above you can also pass oracle, redshift, cassandra, dynamodb, elasticsearch, clickhouse, timescaledb, synapse, azure_blob, adls, sap_hana, mqtt, opcua, influxdb, pi_web, salesforce, hubspot, stripe, netsuite, google_sheets, ga4, sharepoint, and http/rest as connector_type. See Dashboard → Documentation → Connector Data Sources for each type's required secrets fields.

Quick connector example

with AutoDataClient(api_key="dtpk_...") as client:
    # Test the connection
    client.test_connector("sql", secrets={
        "connection_string": "postgresql://user:pass@host/db"
    })

    # List tables
    tables = client.discover("sql", secrets={
        "connection_string": "postgresql://user:pass@host/db"
    })
    print(tables)  # ["users", "orders", ...]

    # Preview columns
    info = client.preview("sql", table="orders", secrets={
        "connection_string": "postgresql://user:pass@host/db"
    })
    print(info["columns"])  # ["id", "price", "date", ...]

    # Run the full pipeline from the connector
    result = client.process_from_connector(
        connector_type="sql",
        table="orders",
        target_columns=["price"],
        secrets={"connection_string": "postgresql://user:pass@host/db"},
        output_rows=10000,
    )
    print(result["files"])

Connector examples by type

Snowflake:

secrets = {
    "account": "abc123.us-east-1",
    "user": "analyst",
    "password": "...",
    "database": "PROD",
    "schema": "PUBLIC",
    "warehouse": "COMPUTE_WH",
}
tables = client.discover("snowflake", secrets=secrets)
result = client.process_from_connector("snowflake", table="ORDERS",
    target_columns=["AMOUNT"], secrets=secrets)

BigQuery:

secrets = {
    "credentials_json": '{"type":"service_account",...}',
    "project_id": "my-project",
    "dataset": "analytics",
}
tables = client.discover("bigquery", secrets=secrets)

Amazon S3:

secrets = {
    "bucket": "my-data-lake",
    "access_key_id": "AKIA...",
    "secret_access_key": "...",
    "region": "us-east-1",
}
result = client.process_from_connector("s3", table="data/sales.csv",
    target_columns=["revenue"], secrets=secrets, output_rows=50000)

MongoDB:

secrets = {
    "connection_string": "mongodb+srv://user:pass@cluster.mongodb.net",
    "database": "analytics",
}
tables = client.discover("mongodb", secrets=secrets)

client.test_connector(connector_type, secrets, credential_id) — Test connection

result = client.test_connector("s3", secrets={...})
# {"success": True, "message": "Connection successful"}

client.discover(connector_type, ...) — List tables / files

tables = client.discover("sql", secrets={...})
# ["users", "orders", "transactions"]

client.preview(connector_type, table, ...) — Preview columns & row count

info = client.preview("sql", table="orders", secrets={...})
# {"success": True, "columns": ["id", "price", ...], "row_count": 50000}

Supports optional custom_query parameter for SQL-based connectors:

info = client.preview("sql", table="orders", custom_query="SELECT * FROM orders WHERE date > '2025-01-01'", secrets={...})

client.process_from_connector(...) — Run pipeline from connector

result = client.process_from_connector(
    connector_type="sql",
    table="orders",
    target_columns=["price"],
    secrets={"connection_string": "postgresql://..."},
    output_rows=10000,
    incremental=True,          # Only fetch new rows since last sync
    type_casts={"age": "int"}, # Override column types before processing
    wait=True,
    download_path="./outputs/",
)

Supports all the same options as client.process(): tools, advanced_params, wait, poll_interval, auto_download, output_preferences, compressed.

client.write_output(session_id, ...) — Write results to a target

client.write_output(
    session_id="abc123...",
    connector_type="sql",
    table_name="ml_prepared_data",
    secrets={"connection_string": "postgresql://..."},
    output_stage="dsg",        # which pipeline output to write
    if_exists="replace",       # "replace", "append", "fail", or "merge"
    merge_keys=["id"],         # column names for merge matching (when if_exists='merge')
)
# {"success": True, "rows_written": 10000, "table": "ml_prepared_data"}

Credential Management

Save credentials on the server so you don't need to pass secrets every time:

# Save a new credential
cred = client.save_credential(
    name="Production DB",
    connector_type="sql",
    secrets={"connection_string": "postgresql://..."},
)
cred_id = cred["id"]

# Use credential_id instead of secrets
tables = client.discover("sql", credential_id=cred_id)
result = client.process_from_connector("sql", table="orders",
    target_columns=["price"], credential_id=cred_id)

# List saved credentials (secrets are never exposed)
creds = client.list_credentials()

# Update a credential
client.update_credential(cred_id, name="Prod DB v2", secrets={...})

# Delete a credential
client.delete_credential(cred_id)

Schema Mapping

Auto-suggest and apply column mappings between source and target schemas:

client.suggest_mapping(source_columns, target_columns) — Auto-suggest mapping

mapping = client.suggest_mapping(
    source_columns=["customer_name", "order_amt", "order_dt"],
    target_columns=["name", "amount", "date"],
    threshold=0.6,  # minimum similarity score (0-1)
)
# {
#   "mapping": {"customer_name": "name", "order_amt": "amount", "order_dt": "date"},
#   "scores": {"customer_name": 0.85, "order_amt": 0.72, "order_dt": 0.68}
# }

client.apply_mapping(session_id, mapping) — Rename columns

result = client.apply_mapping(
    session_id="abc123...",
    mapping={"old_col_name": "new_col_name", ...},
    output_stage="dsg",  # which pipeline stage to apply to
)
# {"columns": ["new_col_name", ...], "preview": [...]}

Quality Alerts

Monitor pipeline metrics and get notified when thresholds are breached:

# Create a quality alert rule
rule = client.create_quality_alert(
    name="High row loss",
    metric="row_loss_pct",    # row_loss_pct, null_pct, column_drop_count, duration_seconds
    operator=">",             # >, <, >=, <=, ==
    threshold=10.0,
    severity="critical",      # warning or critical
    stage="mdh",              # optional: anomaly, dtc, mdh, cds, dsm, dsg
)

# List rules
rules = client.list_quality_alerts()

# Update a rule
client.update_quality_alert(rule_id="...", threshold=15.0, severity="warning")

# Get fired alert events
events = client.get_alert_events(session_id="optional-filter")

# Delete a rule
client.delete_quality_alert(rule_id="...")

Available metrics:

Metric Description
row_loss_pct Percentage of rows lost in a step
null_pct Percentage of null values remaining
column_drop_count Number of columns dropped
duration_seconds Time taken by a pipeline step

Sync Watermarks

Track incremental sync progress for connector-based pipelines:

# List all watermarks
watermarks = client.list_watermarks()
# [{"id": "wm-123", "connector_type": "sql", "table_name": "orders",
#   "watermark_column": "updated_at", "last_value": "2026-04-12T00:00:00Z"}]

# Reset a watermark (re-sync from beginning)
client.reset_watermark(watermark_id="wm-123")

Scheduled Runs

Automate recurring pipeline executions:

# Create an interval-based schedule (every 24 hours)
schedule = client.create_scheduled_run(
    name="Daily ETL",
    schedule_type="interval",
    interval_minutes=1440,
    connector_type="snowflake",
    table="orders",
    credential_id="cred-id",
    y_columns=["target"],
    output_rows=50000,
)

# Create with cron expression (weekdays at 8 AM)
schedule = client.create_scheduled_run(
    name="Weekday ETL",
    schedule_type="cron",
    cron_expression="0 8 * * 1-5",
    connector_type="snowflake",
    table="orders",
    credential_id="cred-id",
    y_columns=["target"],
    tools={"anomaly": True, "dsg": False},  # customize pipeline steps
)

# List all schedules
schedules = client.list_scheduled_runs()

# Enable/disable a schedule
client.toggle_scheduled_run(run_id="run-id")

# Trigger immediately (ignore schedule)
client.run_scheduled_now(run_id="run-id")

# Delete a schedule
client.delete_scheduled_run(run_id="run-id")

Triggers (F7)

Auto-start a pipeline when an external event fires — an inbound webhook, a connector watermark advancing, another pipeline completing, or a quality alert. Triggers are feature-flagged on the server (FEATURE_FLAG_TRIGGERS); if the flag is off, these endpoints return 404.

client.create_trigger(name, trigger_type, config, target_pipeline_config, enabled)

trigger = client.create_trigger(
    name="Reprocess on new S3 batch",
    trigger_type="inbound_webhook",   # inbound_webhook | watermark_advance | pipeline_completion | quality_alert
    config={},                         # type-specific (see table below)
    target_pipeline_config={           # the pipeline to run when the trigger fires
        "source_type": "s3",
        "credential_id": "cred-uuid",
        "table": "incoming/sales.csv",
        "target_columns": ["revenue"],
        "output_rows": 10000,
    },
    enabled=True,
)
# For inbound_webhook the secret is returned ONLY once — save it now:
print(trigger["webhook_secret"], trigger["inbound_url"])

Trigger types & their config:

trigger_type config keys
inbound_webhook optional {"hmac_key": "..."} — require HMAC-SHA256-signed requests
watermark_advance {"credential_id", "table", "watermark_column", "min_advance"}
pipeline_completion {} — chain off another pipeline finishing
quality_alert {} — fire when a quality-alert rule trips
# List / fetch / update / delete
triggers = client.list_triggers()
t        = client.get_trigger(trigger_id)
client.update_trigger(trigger_id, enabled=False)   # name | enabled | config | target_pipeline_config
client.delete_trigger(trigger_id)

# Dry-run: validate config + target WITHOUT firing
report = client.test_trigger(trigger_id)
# {"would_fire": True, "checks": {...}}

Firing an inbound webhook from your own system:

curl -X POST "https://autodata.datatoolpack.com/trigger/inbound/<webhook_secret>" \
     -H "Content-Type: application/json" \
     -d '{"any": "payload"}'
# If you set an hmac_key, also sign the raw body:
#   X-Signature: sha256=<hex HMAC-SHA256 of the request body>

Inbound firing is rate-limited and HMAC-verified server-side; repeated failures auto-disable the trigger.


Folder Listeners

Automatically trigger pipelines when new files appear in cloud storage:

# Create an S3 folder listener
listener = client.create_listener(
    name="S3 Ingest",
    source_type="s3",            # s3, gcs, azure_blob, local, sftp
    watch_path="s3://bucket/incoming/",
    credential_id="cred-id",
    y_columns=["target"],
    pipeline_config={"enable_dsg": False},
    output_rows=10000,
)

# List all listeners
listeners = client.list_listeners()

# Update a listener
client.update_listener(listener_id="...", watch_path="s3://bucket/new-path/", enabled=False)

# Delete a listener
client.delete_listener(listener_id="...")

Supported source types: s3, gcs, azure_blob, local, sftp


SFTP Upload

Upload files to the AutoData server via SFTP for processing:

# Get SFTP server connection info
info = client.sftp_info()
# {"host": "...", "port": 22, "username": "..."}

# Manage SFTP credentials
cred = client.create_sftp_credential(name="My Upload Key")
# {"id": "...", "username": "...", "password": "..."} (password shown only once)

creds = client.list_sftp_credentials()
client.delete_sftp_credential(credential_id="...")

Spark (Large Data)

For datasets too large for pandas, AutoData can use PySpark on the server:

client.spark_status() — Check Spark availability

status = client.spark_status()
# {"available": True, "spark_version": "3.5.0", ...}

client.spark_read(file_path, sample_rows) — Read large files

info = client.spark_read(
    file_path="/data/huge_dataset.parquet",
    sample_rows=20,
)
# {"row_count": 50000000, "columns": [...], "dtypes": {...}, "sample": [...]}

client.spark_transform(...) — Apply Spark transformations

result = client.spark_transform(
    file_path="/data/huge_dataset.parquet",
    output_path="/data/transformed.csv",
    operations=[
        {"op": "cast", "type_casts": {"age": "int"}},
        {"op": "drop_nulls", "subset": ["col1"]},
        {"op": "fill_nulls", "strategy": "mean"},
        {"op": "normalize", "method": "minmax"},
        {"op": "sample", "n": 50000},
    ],
    output_format="csv",  # csv or parquet
)
# {"row_count": 50000, "columns": [...]}

Worker Status

Check the processing worker fleet:

status = client.worker_status()
# {"backend": "local", "active_jobs": 2, "queue_size": 0, ...}

Error Handling

All API errors raise AutoDataError:

from datatoolpack import AutoDataClient, AutoDataError

with AutoDataClient() as client:
    try:
        result = client.process("data.csv", target_columns="price")
    except AutoDataError as e:
        print(f"API error {e.status_code}: {e}")
    except FileNotFoundError as e:
        print(f"File not found: {e}")
    except ValueError as e:
        print(f"Invalid input: {e}")  # e.g. unsupported file format

AutoDataError attributes:

  • str(e) — human-readable error message from the server
  • e.status_code — HTTP status code (e.g. 401, 429, 500), or None for non-HTTP errors

Transient errors (429, 502, 503, 504) are automatically retried up to max_retries times with exponential back-off.


Advanced Example: Non-blocking with manual polling

import time
from datatoolpack import AutoDataClient, AutoDataError

with AutoDataClient() as client:
    # Start job without blocking
    job = client.process(
        "large_dataset.csv",
        target_columns=["churn"],
        wait=False,
    )
    session_id = job["session_id"]
    print(f"Job started: {session_id}")

    # Poll manually
    while True:
        status = client.get_status(session_id)
        print(f"  {status['progress_percent']}% — {status['message']}")
        if status["status"] == "completed":
            break
        elif status["status"] in ("error", "cancelled"):
            raise AutoDataError(f"Job {status['status']}: {status['message']}")
        time.sleep(5)

    # Download results
    path = client.download_results(session_id, download_path="./outputs/")
    print(f"Results saved to {path}")

Complete Method Reference

Core Pipeline

Method Description
process(file_path, target_columns, ...) Upload a file and run the pipeline
get_status(session_id) Poll processing progress
get_result(session_id) Get completed session results
wait_for_completion(session_id) Block until session finishes
cancel(session_id) Cancel a running job
retry_session(session_id) Retry a failed session from checkpoint
download_results(session_id, ...) Download result files
download_file(url, output_path) Download a single file

API Keys & Usage

Method Description
list_keys() List all API keys for the account
get_usage() Get credit usage statistics

Feature Selection

Method Description
recommend_features(session_id, target_columns, ...) Rank a completed session's features (F4)

Connectors

Method Description
test_connector(connector_type, ...) Test connectivity to a data source
discover(connector_type, ...) List tables/files in a data source
preview(connector_type, table, ...) Preview columns and row count
process_from_connector(connector_type, table, ...) Run pipeline from a connector source
write_output(session_id, connector_type, ...) Write results to a database/storage target

Credentials

Method Description
list_credentials() List saved credentials
save_credential(name, connector_type, secrets) Save a new credential
update_credential(credential_id, ...) Update a saved credential
delete_credential(credential_id) Delete a saved credential

Schema Mapping

Method Description
suggest_mapping(source_columns, target_columns) Auto-suggest column mapping
apply_mapping(session_id, mapping) Rename columns using a mapping

Quality Alerts

Method Description
list_quality_alerts() List quality alert rules
create_quality_alert(name, metric, operator, threshold, ...) Create an alert rule
update_quality_alert(rule_id, ...) Update an alert rule
delete_quality_alert(rule_id) Delete an alert rule
get_alert_events(session_id=None) List fired alert events

Sync Watermarks

Method Description
list_watermarks() List all sync watermarks
reset_watermark(watermark_id) Reset a watermark to re-sync

Scheduled Runs

Method Description
list_scheduled_runs() List all scheduled runs
create_scheduled_run(name, schedule_type, ...) Create a scheduled run
delete_scheduled_run(run_id) Delete a scheduled run
toggle_scheduled_run(run_id) Enable/disable a scheduled run
run_scheduled_now(run_id) Trigger a scheduled run immediately

Triggers

Method Description
list_triggers() List your triggers
create_trigger(name, trigger_type, ...) Create an event trigger (F7)
get_trigger(trigger_id) Fetch one trigger
update_trigger(trigger_id, ...) Update a trigger
delete_trigger(trigger_id) Delete a trigger
test_trigger(trigger_id) Dry-run a trigger without firing

Folder Listeners

Method Description
list_listeners() List folder listeners
create_listener(name, source_type, watch_path, ...) Create a folder listener
update_listener(listener_id, ...) Update a folder listener
delete_listener(listener_id) Delete a folder listener

SFTP

Method Description
sftp_info() Get SFTP server connection info
list_sftp_credentials() List SFTP credentials
create_sftp_credential(name) Create SFTP credential
delete_sftp_credential(credential_id) Delete SFTP credential

Spark (Large Data)

Method Description
spark_status() Check if Spark is available
spark_read(file_path, sample_rows) Read a large file via Spark
spark_transform(file_path, output_path, operations, ...) Apply Spark transformations

Worker Status

Method Description
worker_status() Get worker fleet status

Requirements

  • Python >= 3.8
  • requests >= 2.25.0

License

MIT

Metadata

Release files for datatoolpack 0.11.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for datatoolpack 0.11.0
File Size Uploaded Provenance Actions
datatoolpack-0.11.0.tar.gz 49.2 kB Without provenance Details for datatoolpack-0.11.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for datatoolpack 0.11.0
File Interpreter ABI Platform Actions
datatoolpack-0.11.0-py3-none-any.whl Python 3 none any Details for datatoolpack-0.11.0-py3-none-any.whl

Total release size: 79.9 kB

Release files / datatoolpack-0.11.0.tar.gz

Download URL datatoolpack-0.11.0.tar.gz
Size 49.2 kB
Tags Source
SHA-256 checksum
How to use checksums
94c6aec8be028cc38cd20d1b6c353b6bbdb525cbbada1c8d0a8b66a02f0e02e9
BLAKE2b-256 checksum
How to use checksums
dca994d9096e1e16ea6cc1da40cc9ff3ed5fcbf7833a94d762219767a7947170
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.9

Release files / datatoolpack-0.11.0-py3-none-any.whl

Download URL datatoolpack-0.11.0-py3-none-any.whl
Size 30.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
f9f2c66cf2b35c17c2d7216e7310e70ce5974f9f60fa83f3f23a763545178f05
BLAKE2b-256 checksum
How to use checksums
041402ce05c53dbfcc853d5151a4ac3501ec58a8b595cff2723466eb5ebc3d3e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.9

Security for datatoolpack 0.11.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page