Skip to main content

A client library for DXR

Project description

Data X-Ray Python Library

Unofficial Python library for the Data X-Ray API.

Warning This library is unofficial and unsupported and may change at any time. Not for production use.

Installation

pip install dxrpy

Usage

Initializing the Client

from dxrpy import DXRClient

client = DXRClient(
    api_url="https://your-dxr-instance.example.com",
    api_key="your_api_key",
)

Credentials can also be loaded from environment variables (or a .env file):

DXR_BASE_URL=https://your-dxr-instance.example.com
DXR_API_KEY=your_api_key

On-Demand Classifier

Run files through the on-demand classification pipeline.

from dxrpy.utils import File

files = [File("path/to/file1.txt"), File("path/to/file2.txt")]
hits = client.on_demand_classifier.run_job(files, datasource_ids=[123])

for hit in hits:
    print(hit.labels)

Datasources

Create and manage datasources programmatically.

# List all datasources
datasources = client.datasources.list()

# Create a new datasource
ds = client.datasources.create(
    name="benchmark-experiment-1",
    connector_type_id=7,          # on-demand / file upload connector
    settings_profile_id=50217,
)
print(ds.id)

# Find by name
ds = client.datasources.find_by_name("benchmark-experiment-1")

# Update
client.datasources.update(ds.id, name="benchmark-experiment-1-v2")

# Delete
client.datasources.delete(ds.id)

Smart Labels

Smart labels (tags) automatically tag documents matching a query condition. They are defined globally and reference one or more datasources.

from dxrpy.index.json_search_query import JsonSearchQueryItem

# Create a label that fires whenever an SSN annotation exists
label = client.smart_labels.create(
    name="Has-SSN",
    datasource_ids=[50148],
    query_items=[
        JsonSearchQueryItem(
            parameter="annotators",
            value="annotation.42",
            type="text",
            match_strategy="exists",
        ),
    ],
    color="#FF5733",
)

# List / find
labels = client.smart_labels.list()
label = client.smart_labels.find_by_name("Has-SSN")

# Update conditions
client.smart_labels.update(label.id, query_items=[...])

# Delete
client.smart_labels.delete(label.id)

Extractors

LLM metadata extractors define a prompt and optional target data types.

extractor = client.extractors.create(
    name="PII extractor",
    prompt="Extract all personal information from the text. Return a JSON object with keys: name, email, phone.",
    data_types=["PERSON", "EMAIL_ADDRESS", "PHONE_NUMBER"],
)
print(extractor.id)

# CRUD
extractors = client.extractors.list()
extractor = client.extractors.find_by_name("PII extractor")
client.extractors.update(extractor.id, prompt="Updated prompt...")
client.extractors.delete(extractor.id)

Settings Profiles & Extraction Workflows

Settings profiles bundle classification configuration. Use set_extraction_workflow to attach an extractor (unconditionally or with conditions) to a profile.

from dxrpy import WorkflowStep
from dxrpy.index.json_search_query import JsonSearchQueryItem

# Extractor that always runs
client.settings_profiles.set_extraction_workflow(
    profile_id=50217,
    enabled=True,
    steps=[WorkflowStep(extractor_id=594)],
)

# Extractor that only runs when an SSN annotation already exists
client.settings_profiles.set_extraction_workflow(
    profile_id=50217,
    enabled=True,
    steps=[
        WorkflowStep(
            extractor_id=594,
            condition=[
                JsonSearchQueryItem(
                    parameter="annotators",
                    value="annotation.16",
                    type="text",
                    match_strategy="exists",
                )
            ],
        )
    ],
)

# Disable extraction
client.settings_profiles.set_extraction_workflow(
    profile_id=50217,
    enabled=False,
)

Searching the Index

from dxrpy.index.json_search_query import JsonSearchQuery, JsonSearchQueryItem

query = JsonSearchQuery(
    datasource_ids=["123"],
    query_items=[
        JsonSearchQueryItem(parameter="match_all", value={}, type="match")
    ],
)
results = client.index.search(query)

print(f"Total hits: {results.total_hits}")
for hit in results.hits:
    print(hit.file_name, hit.labels)

Changelog

See CHANGELOG.md.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dxrpy-0.3.1.tar.gz (17.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dxrpy-0.3.1-py3-none-any.whl (23.0 kB view details)

Uploaded Python 3

File details

Details for the file dxrpy-0.3.1.tar.gz.

File metadata

  • Download URL: dxrpy-0.3.1.tar.gz
  • Upload date:
  • Size: 17.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.7

File hashes

Hashes for dxrpy-0.3.1.tar.gz
Algorithm Hash digest
SHA256 f07374d453cd262ccec6daa395edb53cda73d228ae97a1be8deacecf688cfffd
MD5 11f3a2496e4baebb9f7949832cfb2fef
BLAKE2b-256 dcb300f2fc753164b246330750d6a9cd6e36e929a6bff0a309e40c34ecc4b93b

See more details on using hashes here.

File details

Details for the file dxrpy-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: dxrpy-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 23.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.7

File hashes

Hashes for dxrpy-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 dd936e346808f874caef8a71c193460a0761b78086b957ee5122eb55e6c765e7
MD5 7a602e5b106c9ed34e6d5866ab49e194
BLAKE2b-256 d4e1c1909f200a5a3bb2f9652cb26e5bae17b9a639e5ac96135df6618c091365

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page