thds.adls

A high-performance Azure Data Lake Storage (ADLS Gen2) client for the THDS monorepo. It wraps the Azure SDK with hash-aware caching, azcopy acceleration, and shared client/credential plumbing so applications can transfer large blob datasets quickly and reliably.

Highlights

Environment-aware paths first: Almost every consumer starts by importing fqn, AdlsFqn, and defaults.env_root() to build storage-account/container URIs that follow the current THDS environment.
Cache-backed reads: download_to_cache is the standard entry point for pulling blobs down with a verified hash so local workflows, tests, and pipelines can operate on read-only copies.
Bulk filesystem helpers: ADLSFileSystem powers scripts and jobs that need to walk directories, fetch batches of files, or mirror hive tables without re-implementing Azure SDK plumbing.
Spark/Databricks bridges: abfss and uri conversions keep analytics code agnostic to whether it needs an adls://, abfss://, https://, or dbfs:// view of the same path.
Composable utilities: Higher-level modules (cache, upload, copy, list) layer on top of those imports so teams can opt into more advanced behavior without leaving the public API surface.

Key Modules

Component	Typical usage in the monorepo
`fqn`	Parse, validate, and join ADLS paths; used when materializing model datasets and configuring pipelines.
`AdlsFqn`	Strongly typed value passed between tasks and tests to represent a single blob or directory.
`defaults` / `named_roots`	Resolve environment-specific storage roots (`defaults.env_root()`, `named_roots.require(...)`).
`download_to_cache` (`cached` module)	Bring a blob down to the shared read-only cache before analytics, feature builds, or test fixtures run.
`ADLSFileSystem` (`impl` module)	Fetch or list entire directory trees and integrate with caching inside scripts and notebooks.
`abfss`	Translate `AdlsFqn` objects into `abfss://` URIs for Spark/Databricks jobs.
`uri`	Normalize `adls://`, `abfss://`, `https://`, and `dbfs://` strings into `AdlsFqn` values (and vice versa).
`global_client` / `shared_credential`	Shared, fork-safe Azure clients and credentials backing the public helpers above.

Example Usage

Use the caching helpers and Source integration:

from thds.adls import cached, upload, source

cache_path = cached.download_to_cache("adls://acct/container/path/to/file")
src = upload("adls://acct/container/path/out.parquet", cache_path)
verified = source.get_with_hash(src.uri)

For CLI usage, run (from repo root):

uv run python -m thds.adls.tools.download adls://acct/container/path/file

Operational Notes

Hash metadata: Uploads attach hash_xxh3_128_b64 automatically when the bytes are known. Download completion back-fills missing hashes when permissions allow.
Locks and concurrency: Large transfers acquire per-path file locks to keep azcopy instances cooperative. Global HTTP connection pools default to 100 but are configurable via thds.core.config.
Error handling: BlobNotFoundError and other ADLS-specific exceptions translate into custom error types to simplify retries and diagnostics.
Extensibility: Additional hash algorithms can be registered by importing dependent packages (e.g., blake3). Named roots can be populated dynamically via environment-specific modules (thds.adls._thds_defaults hook).

Release files for thds.adls 4.5.20260921185128

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for thds.adls 4.5.20260921185128
File	Interpreter	ABI	Platform	Reset
thds_adls-4.5.20260921185128-py3-none-any.whl 66.8 kB Sep 21, 2026	Python 3	none	any	Details

Release files / thds_adls-4.5.20260921185128-py3-none-any.whl

Download URL	thds_adls-4.5.20260921185128-py3-none-any.whl
Size	66.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`b7145e8b351047d404a137146f047c919d4a5b500c8698820e16d05da3d66983`
BLAKE2b-256 checksum How to use checksums	`f5451e725f71f5061422a91f75feff05d996f392067b90ecf44a874c4beebf86`
Upload date	Sep 21, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.13.3`