🏔️ aspect
aspect is a Python library and command-line tool for reproducible featurisation of tabular machine-learning datasets.
It lets you define independent transformation pipelines for different input columns, including molecular structures, taxonomy, categorical variables, and numerical measurements. Pipelines can be serialised, applied to new datasets, checkpointed, and restored.
Aspect uses Providence for dataset loading, source provenance, and checkpointing, and Hugging Face Datasets for Arrow-backed data processing and caching.
The core idea:
Input dataset
│
├── SMILES ──────→ Molecular fingerprints
├── Species ─────→ Taxonomic fingerprints
├── Assay ───────→ Categorical encoding
├── MW ──────────→ Log transformation
└── Description ─→ Language-model embeddings
│
▼
Aspect DataPipeline
│
├── Reusable feature specification
├── Reproducible dataset checkpoint
└── Optional source materialisation
│
▼
Providence DataResolver
│
├── Hugging Face Dataset
├── Export: CSV / TSV / JSON / Parquet
└── Model-specific collation / DataLoader
Feature representations are stored in Arrow-compatible columns. Model-specific collators can convert these stored values into tensors, molecular graphs, or other runtime representations without embedding model-specific logic in dataset preprocessing.
Contents
- Quick start
- Installation
- Command-line interface
- Python API
- Feature specifications
- Available transforms
- Saving and loading pipelines
- Supported input and output formats
- Working with output columns
- Custom transforms
- Model integration
- Development
- Issues and documentation
Quick start
1. Featurise a dataset
Create a small CSV:
cat > compounds.csv <<'CSV'
compound_id,assay,mwt,pIC50
cmpd-1,MIC,46.07,5.0
cmpd-2,MIC,60.10,6.2
cmpd-3,IC50,78.11,4.8
CSV
Generate numeric and categorical features:
aspect featurize compounds.csv \
--features mwt:log@log_mwt assay:hash@assay_hash pIC50@target \
--extras compound_id \
--output features.parquet
The feature specification defines the transformations:
| Specification | Meaning |
|---|---|
mwt:log@log_mwt |
Apply natural logarithm to mwt |
assay:hash@assay_hash |
Hash the assay category into a fixed-length vector |
pIC50@target |
Copy pIC50 into a feature named target |
--extras compound_id |
Explicitly retain the compound identifier |
By default, Aspect also retains original input columns.
2. Save a reproducible checkpoint
The same command can save a checkpoint alongside the exported table:
aspect featurize compounds.csv \
--features mwt:log@log_mwt assay:hash@assay_hash \
--checkpoint features.as \
--output features.parquet
The checkpoint records the feature specification, data source, and information needed to reconstruct the transformed dataset.
3. Restore and export
Reconstruct the transformed dataset from its checkpoint:
aspect write features.as \
--output restored.parquet
Export it in another format:
aspect write features.as \
--output restored.csv.gz
4. Python API
from aspect.data import DataPipeline
data = {
"compound_id": ["cmpd-1", "cmpd-2", "cmpd-3"],
"assay": ["MIC", "MIC", "IC50"],
"mwt": [46.07, 60.10, 78.11],
}
pipe = DataPipeline(
column_transforms={
"log_mwt": ("mwt", "log"),
"assay_hash": (
"assay",
{"name": "hash", "kwargs": {"ndim": 8}},
),
},
columns_to_keep=["compound_id"],
)
resolved = pipe(data, drop_unused_columns=True)
# Access the underlying Hugging Face Dataset
features = resolved.data
print(features.column_names)
print(features[:2])
The transformed dataset contains:
['compound_id', 'log_mwt', 'assay_hash']
log_mwt contains a numeric feature with shape (n, 1), while assay_hash contains an (n, 8) feature matrix.
Save and reload the pipeline:
pipe.save("features.as")
restored = DataPipeline.load("features.as")
print(restored.data_out.data[:2])
Installation
Aspect requires Python 3.11 or later.
Basic installation
pip install aspect-data
Optional dependencies
Install only the features you need:
| Extra | Functionality | Installation |
|---|---|---|
chem |
Molecular fingerprints and descriptors via Schemist | pip install "aspect-data[chem]" |
bio |
Taxonomic fingerprints via Vectome | pip install "aspect-data[bio]" |
torch |
PyTorch integration | pip install "aspect-data[torch]" |
chemprop |
Chemprop molecular features | pip install "aspect-data[chemprop]" |
chemeleon |
Frozen CheMeleon molecular embeddings | pip install "aspect-data[chemeleon]" |
atom-jepa |
Frozen Atom-JEPA and Atom-JEPA-Bio 3D molecular embeddings | pip install "aspect-data[atom-jepa]" |
deep |
Transformer-based SMILES/text embeddings (includes chem) |
pip install "aspect-data[deep]" |
esm |
ESM2 protein embeddings | pip install "aspect-data[esm]" |
Install from source
git clone https://github.com/scbirlab/aspect.git
cd aspect
pip install -e ".[dev]"
Aspect depends on Providence (>=0.0.2), which provides the common dataset-resolution and checkpointing infrastructure.
Command-line interface
Aspect provides three commands:
aspect serialize
aspect featurize
aspect write
View available arguments with:
aspect --help
aspect featurize --help
aspect serialize --help
aspect write --help
aspect serialize
Create a reusable feature specification without processing a dataset:
aspect serialize \
--features mwt:log@log_mwt assay:hash@assay_hash \
--extras compound_id \
--output feature-spec.as
The specification can then be applied to other datasets:
aspect featurize new-compounds.csv \
--config feature-spec.as \
--output new-features.parquet
aspect featurize
Apply a feature specification or saved pipeline to a dataset.
aspect featurize compounds.csv \
--features mwt:log@log_mwt assay:hash@assay_hash \
--output features.parquet
A saved specification can be supplied with --config, either as a checkpoint directory or as a pipeline JSON file (as written by DataPipeline.to_config):
aspect featurize compounds.csv \
--config feature-spec.as \
--output features.parquet
Missing values
Columns are written out unchanged, nulls included. Transforms see numeric nulls as NaN, which propagates (e.g. log of a missing value is NaN), and string nulls as "", which molecular featurizers treat as an invalid SMILES (usually a zero vector). Other string transforms featurize the empty string, so hash gives a non-zero vector; use missing indicators if you need to tell these rows apart.
The missing transform gives a Boolean column, True where its input is null, NaN or the empty string (empty strings count as missing, which suits SMILES and other identifiers, but not categories where "" is a real value) (e.g. --features smiles:missing@smiles_missing). --missing-indicators (or "missing_indicators": true in a pipeline config that uses the column_transforms key) is shorthand that adds a missing:<name> branch on the input of every transform output <name>. Saved configurations contain the expanded branches. The indicators can be used as features downstream, e.g. in Arvore. Integer columns with nulls are read by transforms as float64, so their values above 2^53 are rounded; integer columns without nulls stay exact.
Checkpoint without exporting
aspect featurize compounds.csv \
--config feature-spec.as \
--checkpoint features.as
Save source and transformed data
aspect featurize compounds.csv \
--config feature-spec.as \
--checkpoint features.as \
--force-write-data
This requests materialisation of both source and transformed data inside the checkpoint, making it less dependent on the original dataset remaining available.
Without this option, Providence can retain immutable remote datasets by reference, avoiding unnecessary duplication. Local inputs are generally packaged by the current implementation.
Process a subset of rows
aspect featurize large-table.parquet \
--start 1000 \
--end 2000 \
--features mwt:log@log_mwt \
--checkpoint subset.as
--start is inclusive and --end is exclusive. The example selects rows 1000–1999.
Configure the cache
aspect featurize compounds.csv \
--features assay:hash@assay_hash \
--cache .cache \
--output features.parquet
The cache directory is --cache (or DataPipeline(cache_dir=...)) if given, else $PROVIDENCE_CACHE, else the platform user cache directory (on Linux, $XDG_CACHE_HOME/providence/<version>, defaulting to ~/.cache). Datasets, Hugging Face models, and other downloaded model weights (e.g. CheMeleon) all live there.
Cached features are keyed on the transform configuration and on the installed versions of the feature backends (schemist, rdkit, descriptastorus, vectome, chemprop, dimorphite_dl, transformers, torch). Upgrading any of these recomputes features rather than reusing a stale cache.
aspect write
Export data from a saved Aspect checkpoint:
aspect write features.as --output features.csv
aspect write features.as --output features.csv.gz
aspect write features.as --output features.parquet
aspect write features.as --output features.hf
The checkpoint is loaded through DataPipeline.load() and the output is delegated to Providence.
If the checkpoint contains only a feature specification and no source dataset, there is no transformed table to export.
Python API
The main class is DataPipeline.
from aspect.data import DataPipeline
A pipeline maps input columns to named output features.
pipe = DataPipeline({
"log_mwt": ("mwt", "log"),
"assay_hash": (
"assay",
{"name": "hash", "kwargs": {"ndim": 16}},
),
})
Input data
Pipelines can be applied to Python mappings, Pandas DataFrames, Hugging Face datasets, local files, or Providence resolvers.
# Dictionary
resolved = pipe({
"mwt": [46.07, 60.10],
"assay": ["MIC", "IC50"],
})
# Pandas DataFrame
resolved = pipe(df)
# Local file
resolved = pipe("compounds.csv")
# Hugging Face Hub dataset
resolved = pipe(
"hf://datasets/scbirlab/fang-2023-biogen-adme~scaffold-split:train"
)
Providence integration
Aspect uses Providence's DataResolver to manage source datasets and transformed results.
from providence import DataResolver
from aspect.data import DataPipeline
source = DataResolver("compounds.csv")
pipe = DataPipeline({
"log_mwt": ("mwt", "log"),
})
result = pipe(source)
# Underlying datasets
input_ds = pipe.data_in.data
output_ds = result.data
# Source metadata
source_info = pipe.data_in.data_source
print(output_ds.column_names)
DataPipeline.__call__() returns a DataResolver, rather than a raw Hugging Face dataset.
The underlying dataset remains available through .data, and provenance is tracked through the resolver.
Exporting results
pipe.write_table("features.parquet")
pipe.write_table("features.csv.gz")
This delegates serialization to Providence.
Selecting rows
For explicit row selection, create and slice the resolver before passing it to Aspect:
source = DataResolver("compounds.csv")
source = source.select_rows(start=100, end=200)
result = pipe(source)
Reusing transformations
The same feature specification can be applied to multiple datasets:
pipe = DataPipeline({
"log_mwt": ("mwt", "log"),
})
train = pipe("train.csv")
test = pipe("test.csv")
Each call returns the corresponding transformed resolver. The pipeline's data_in and data_out attributes refer to the most recent call.
Feature specifications
Aspect supports both structured Python specifications and compact CLI strings.
Python specifications
The basic format is:
{
"output_column": ("input_column", "transform_name")
}
Example:
pipe = DataPipeline({
"log_mwt": ("mwt", "log"),
})
Parameterised transforms
Pass a dictionary containing the transform name and keyword arguments:
pipe = DataPipeline({
"assay_hash": (
"assay",
{
"name": "hash",
"kwargs": {
"ndim": 32,
"seed": 123,
},
},
),
})
Chained transforms
Pass a sequence of transformations:
pipe = DataPipeline({
"scaled_log_mwt": (
"mwt",
[
"log",
{
"name": "scale",
"kwargs": {"factor": 0.1},
},
],
),
})
The scale transform in this example is a custom transform that must be registered first.
Chains execute in the supplied order. Each transform must accept the representation produced by the preceding transform.
Branches that start with the same step (same transform, input column and arguments) share that step's output, so it runs once per batch. For example, one clean-smiles step can feed several molecular featurizers, and also be written out by naming it as its own branch:
pipe = DataPipeline({
"clean_smiles": ("smiles", "clean-smiles"),
"morgan": ("smiles", ["clean-smiles", "morgan-fingerprint"]),
"chemeleon": ("smiles", ["clean-smiles", "chemeleon"]),
})
CLI specifications
The CLI format is:
input_column:transform@output_column
| Syntax | Behaviour |
|---|---|
column |
Retain an input column |
column@new_name |
Identity transformation with a new name |
column:transform |
Apply a transform and automatically name the output |
column:transform@new_name |
Apply a transform with an explicit output name |
column:t1:t2@new_name |
Chain transformations |
column:transform(arg=value)@name |
Pass simple transform arguments |
Example:
aspect featurize compounds.csv \
--features \
mwt:log@log_mwt \
assay:hash(ndim=32)@assay_hash \
pIC50@target \
--output features.parquet
Simple integers, floats, and booleans are parsed automatically. For complex or nested parameters, use the Python API.
Available transforms
Base transforms
Available with the standard installation:
| Transform | Input | Output |
|---|---|---|
identity |
Any supported column | Identity representation |
log |
Numeric | Natural logarithm |
hash |
Categorical/string | Deterministic fixed-length vector |
one-hot |
Categorical | One-hot vector |
Example:
pipe = DataPipeline({
"log_mwt": ("mwt", "log"),
"assay_hash": (
"assay",
{"name": "hash", "kwargs": {"ndim": 16}},
),
"assay_onehot": (
"assay",
{
"name": "one-hot",
"kwargs": {"categories": ["MIC", "IC50"]},
},
),
})
Chemistry transforms
Install with:
pip install "aspect-data[chem]"
| Transform | Input | Output |
|---|---|---|
clean-smiles |
SMILES | Canonical SMILES, desalted and protonated |
morgan-fingerprint |
SMILES | Molecular fingerprint |
descriptors-2d |
SMILES | 2D molecular descriptors |
descriptors-3d |
SMILES | 3D molecular descriptors |
conformers |
SMILES | 3D conformers (atomic numbers and coordinates) |
Molecular featurizers use SMILES as given and make no cleaning decisions. To clean, chain clean-smiles first (CLI: smiles:clean-smiles:morgan-fingerprint@morgan). It canonicalizes, keeps the parent fragment and sets the dominant protonation state at pH 7.4 (desalt=True, protonate=True, protonation_pH=7.4); SMILES it cannot parse become null, which featurizers treat as missing.
conformers embeds 3D conformers, with hydrogens, using schemist (ETKDG, then MMFF94). Options: n_conformers (default 1), seed (42), etkdg_version (3), prune_rms_thresh (-1, no pruning), minimize (True), max_iters (500) and n_threads (1); the defaults reproduce the Atom-JEPA protocol. Each row is a struct of atomic_numbers (int8 [N]) and coordinates (float32 [conformers, N, 3], Å), stored exactly in Arrow and Parquet. Rows are null where the SMILES is missing or invalid, or conformers could not be generated; chain missing after conformers (e.g. smiles:conformers:missing@conformers_failed) to flag them.
descriptors-3d averages RDKit 3D descriptors over an ETKDG conformer ensemble and appends their standard deviations across conformers. By default it embeds clip(1 + rotatable bonds, 3, 10) conformers per molecule, prunes near-duplicates (heavy-atom RMSD < 0.5 Å), and skips force-field minimisation. Options (n_conformers, max_conformers, prune_rms_thresh, minimize, with_std, n_threads, seed) go in kwargs and are recorded in the transform config:
pipe = DataPipeline({
"desc3d": ("smiles", {"name": "descriptors-3d", "kwargs": {"n_conformers": 20}}),
})
Example:
data = {
"compound_id": ["ethanol", "ethylamine", "benzene"],
"smiles": ["CCO", "CCN", "c1ccccc1"],
}
pipe = DataPipeline(
{
"morgan": ("smiles", "morgan-fingerprint"),
"desc2d": ("smiles", "descriptors-2d"),
},
columns_to_keep=["compound_id"],
)
features = pipe(data, drop_unused_columns=True).data
Taxonomic transforms
Install with:
pip install "aspect-data[bio]"
The vectome-fingerprint transform encodes species names or taxonomic identifiers as fixed-length taxonomic representations.
data = {
"species": [
"Mycobacterium tuberculosis",
"Escherichia coli",
"Staphylococcus aureus",
],
}
pipe = DataPipeline({
"species_fp": (
"species",
{
"name": "vectome-fingerprint",
"kwargs": {"ndim": 128},
},
),
})
features = pipe(data).data
Deep-learning transforms
Install with:
pip install "aspect-data[deep]"
The hf-bart transform supports transformer-derived text embeddings.
data = {
"description": [
"cell wall inhibitor",
"protein synthesis inhibitor",
"DNA gyrase inhibitor",
],
}
pipe = DataPipeline({
"text_embedding": (
"description",
{
"name": "hf-bart",
"kwargs": {
"ref": "facebook/bart-base",
"aggregator": ["mean", "max"],
},
},
),
})
features = pipe(data).data
Inputs are embedded in length-sorted chunks of batch_size strings (default 64), and aggregators ignore padding tokens, so memory is bounded and embeddings do not depend on batch composition. Model weights may be downloaded on first use.
Inputs are tokenized as given, so hf-bart works on SMILES and free text alike; chain clean-smiles before it to clean SMILES. Missing inputs are embedded as <unk>.
Protein embeddings
Install with:
pip install "aspect-data[esm]"
The esm2 transform embeds amino acid sequences with an ESM2 protein language model, mean-pooling the final layer over residue tokens.
data = {"sequence": ["MKTAYIAKQRQISFVKSHFSRQ", "MSLLTEVETPIRNEWGCRC"]}
pipe = DataPipeline({
"protein_embedding": (
"sequence",
{
"name": "esm2",
"kwargs": {
"model": "facebook/esm2_t12_35M_UR50D", # 480-d; default
"revision": "6fbf070e65b0b7291e7bbcd451118c216cff79d8",
},
},
),
})
features = pipe(data).data
Pin revision to a Hugging Face commit for reproducible embeddings. Other options: layer (default -1), pooling ("mean" or "cls"), batch_size, device, and long_sequences ("truncate" to 1022 residues, the default, or "chunk" for length-weighted overlapping windows). Missing or invalid sequences give zero vectors.
Chemprop transforms
Install with:
pip install "aspect-data[chemprop]"
The chemprop-mol transform prepares molecular datapoints for Chemprop-style graph models.
Molecular graphs generally require model-specific batching. Aspect separates dataset storage from runtime collation so graph representations can be reconstructed when needed.
CheMeleon embeddings
Install with:
pip install "aspect-data[chemeleon]"
The chemeleon transform embeds SMILES with the frozen message-passing encoder of CheMeleon, a Chemprop D-MPNN pre-trained on Mordred descriptors, giving 2048-d mean-aggregated vectors.
pipe = DataPipeline({"chemeleon": ("smiles", "chemeleon")})
features = pipe(data).data
SMILES are used as given, matching upstream CheMeleonFingerprint; chain clean-smiles first, e.g. ("smiles", ["clean-smiles", "chemeleon"]), so that CC(=O)O.[Na+], CC(=O)O and CC(=O)[O-] give the same vector. Options: batch_size (default 256) and device. Missing or invalid SMILES give zero vectors.
Weights are downloaded from Zenodo on first use to chemeleon/chemeleon_mp.pt in the cache directory and verified by SHA-256. Concurrent processes share one download through a file lock, and a cached file failing the checksum is re-downloaded once. Locking across nodes needs a filesystem with working locks (e.g. Lustre mounted with flock, not localflock); otherwise point the cache at node-local storage.
| Environment variable | Default | Purpose |
|---|---|---|
ASPECT_CHEMELEON_LOCK_TIMEOUT |
600 |
Seconds to wait for another process's download |
ASPECT_CHEMELEON_DOWNLOAD_TIMEOUT |
60 |
Socket timeout in seconds for the download |
Atom-JEPA embeddings
Install with:
pip install "aspect-data[atom-jepa]"
The atom-jepa transform embeds 3D conformers with the frozen encoder of Atom-JEPA, averaging each molecule's conformer embeddings. Generate the conformers with the conformers transform, whose defaults reproduce the Atom-JEPA protocol, so ("smiles", ["conformers", "atom-jepa"]) matches upstream Atom-JEPA exactly with the same RDKit version (tested bit-for-bit with RDKit 2026.09 and atom-jepa 0.4.0; conformers can differ across RDKit versions, which are part of the cache key). Branches that share a conformers step generate conformers once:
pipe = DataPipeline({
"conformers": ("smiles", {"name": "conformers", "kwargs": {"n_conformers": 10}}),
"conformers_failed": ("smiles", [{"name": "conformers", "kwargs": {"n_conformers": 10}}, "missing"]),
"jepa_bio": ("smiles", [{"name": "conformers", "kwargs": {"n_conformers": 10}}, "atom-jepa"]),
"jepa": ("smiles", [
{"name": "conformers", "kwargs": {"n_conformers": 10}},
{"name": "atom-jepa", "kwargs": {"model": "molecules"}},
]),
})
features = pipe(data).data
Options: model ("bio", default, for Atom-JEPA-Bio, or "molecules"), layers ("last", default, 256 values; "all", the 8 blocks flattened to 2048; or a block 1-8), batch_size (conformers per forward pass, default 32, which bounds tensor memory), device, precision ("fp32", default, or "bf16" on CUDA) and revision. Null conformers give zero vectors; conformers_failed in the example above marks those rows, so they can be told apart from real embeddings, e.g. when benchmarking representations. Averaging embeddings over conformers is not the same as averaging predictions over them.
Weights are downloaded from Hugging Face on first use, pinned to a commit (revision), into the Hugging Face cache inside the cache directory.
Saving and loading pipelines
Aspect supports reusable feature specifications and provenance-aware dataset checkpoints.
Save a feature specification
Create a pipeline without applying it to any dataset:
pipe = DataPipeline({
"log_mwt": ("mwt", "log"),
"assay_hash": (
"assay",
{"name": "hash", "kwargs": {"ndim": 16}},
),
})
pipe.save("feature-spec.as")
Restore and apply it later:
from aspect.data import DataPipeline
pipe = DataPipeline.load("feature-spec.as")
result = pipe({
"mwt": [46.07, 60.10],
"assay": ["MIC", "IC50"],
})
Checkpoint a transformed dataset
pipe = DataPipeline({
"log_mwt": ("mwt", "log"),
})
result = pipe("compounds.csv")
pipe.save("features.as")
The checkpoint records the pipeline configuration and a Providence checkpoint describing the input dataset.
Load it again:
restored = DataPipeline.load("features.as")
features = restored.data_out.data
By default, when transformed data has not been explicitly saved, loading reconstructs it by applying the feature pipeline to the restored source dataset.
Materialise transformed data
pipe.save(
"features.as",
save_transformed=True,
)
This stores a transformed-data checkpoint alongside the source checkpoint.
Package source data
pipe.save(
"features.as",
save_source_data=True,
save_transformed=True,
)
This requests packaging of the input data and transformed data.
For remote datasets with immutable revisions, Providence can instead retain source references, allowing checkpoints to be substantially smaller.
Local sources are normally materialised to preserve reproducibility.
Load without applying transformations
pipe = DataPipeline.load(
"features.as",
skip_call=True,
)
This restores the specification and available checkpoint artefacts without automatically running the transformation pipeline.
Checkpoint layout
A checkpoint may contain:
features.as/
├── config.json
├── data.pv/
└── transformed.pv/
config.json contains the pipeline specification and checkpoint metadata.
data.pv contains the Providence source checkpoint.
transformed.pv is included when transformed data is explicitly checkpointed.
The contents of Providence checkpoints depend on whether datasets are retained by reference or materialised.
Compatibility
The legacy methods save_checkpoint() and load_checkpoint() remain available as compatibility aliases, but new code should use save() and load().
Supported input and output formats
Input types
Aspect accepts the input types supported by Providence, including:
| Input | Example |
|---|---|
| Python mapping | {"smiles": ["CCO", "CCN"]} |
| Pandas DataFrame | pipe(df) |
| Hugging Face Dataset | pipe(dataset) |
| Hugging Face IterableDataset | pipe(iterable_dataset) |
| Local file | pipe("data.csv") |
| Hugging Face dataset reference | pipe("hf://datasets/org/repo~config:split") |
| Providence DataResolver | pipe(resolved) |
Local files
Commonly supported formats include:
- CSV:
.csv,.csv.gz - Tab-separated:
.tsv,.tsv.gz,.txt,.txt.gz - JSON:
.json - Parquet:
.parquet - Arrow:
.arrow - XML:
.xml - Saved Hugging Face datasets:
.hf
Output formats
The CLI and write_table() use Providence to select an output format from the filename.
| Extension | Format |
|---|---|
.csv |
CSV |
.csv.gz |
Gzip-compressed CSV |
.tsv, .txt |
Tab-separated text |
.tsv.gz, .txt.gz |
Gzip-compressed tab-separated text |
.json |
JSON |
.parquet |
Parquet |
.hf |
Hugging Face dataset directory |
| Unrecognised | Hugging Face dataset directory with .hf appended |
Hugging Face Hub references
Providence supports versioned dataset references:
hf://datasets/<owner>/<repo>~<config>:<split>@<revision>
For example:
hf://datasets/scbirlab/fang-2023-biogen-adme~scaffold-split:train
The revision, configuration, and split are optional where supported by the resolver.
Pinning a dataset to an immutable commit revision improves reproducibility.
Working with output columns
By default, Aspect retains original columns and adds transformed features.
out = pipe(data)
To retain only transformed features and explicitly selected metadata:
pipe = DataPipeline(
{
"log_mwt": ("mwt", "log"),
"assay_hash": ("assay", "hash"),
},
columns_to_keep=["compound_id"],
)
out = pipe(
data,
drop_unused_columns=True,
)
Extra columns can also be specified at call time:
out = pipe(
data,
drop_unused_columns=True,
keep_extra_columns=["compound_id", "target"],
)
Access the underlying output dataset using:
dataset = out.data
print(dataset.features)
print(dataset.column_names)
Custom transforms
Custom transforms can be registered through Aspect's transform registry.
A transform factory receives configuration arguments and returns a callable that operates on data batches.
import numpy as np
from aspect.data import DataPipeline
from aspect.transform.registry import register_function
@register_function("square")
def Square():
def _square(data, input_column):
x = np.asarray(data[input_column], dtype=float)
return x ** 2
return _square
pipe = DataPipeline({
"mwt_squared": ("mwt", "square"),
})
out = pipe({
"mwt": [46.07, 60.10, 78.11],
}).data
Parameterised transforms
@register_function("scale")
def Scale(factor=1.0):
def _scale(data, input_column):
x = np.asarray(data[input_column], dtype=float)
return x * factor
return _scale
pipe = DataPipeline({
"scaled_mwt": (
"mwt",
{"name": "scale", "kwargs": {"factor": 0.001}},
),
})
Custom transforms must be registered in the Python process before loading pipelines that reference them.
Model integration
Aspect separates stored feature representations from model-specific runtime inputs.
The output dataset may contain numeric arrays, chemical fingerprints, taxonomic encodings, or other Arrow-compatible values.
For simple numerical models, extract the dataset:
features = pipe(data).data
For PyTorch models, use the built-in DataLoader integration:
loader = pipe.dataloader(
batch_size=32,
shuffle=True,
)
for batch in loader:
# Batched, collated features
pass
This requires the torch extra.
Rows are read with the pipeline's output_format (default "numpy"), so floating-point features arrive as float32 tensors. A dataset you pass in that already has a format set keeps it.
Transforms requiring specialised runtime representations can provide collators. These are applied when constructing model batches, allowing features such as molecular graphs to be stored independently from the framework-specific objects consumed by a model.
Custom collators can be provided through the collators argument:
loader = pipe.dataloader(
batch_size=32,
collators={
"custom_feature": custom_collator,
},
)
Mixed molecular and biological features
Aspect can combine multiple representations in one dataset:
data = {
"compound_id": ["cmpd-1", "cmpd-2", "cmpd-3"],
"smiles": ["CCO", "CCN", "c1ccccc1"],
"assay": ["solubility", "clearance", "microsome"],
"species": [
"Escherichia coli",
"Mycobacterium tuberculosis",
"Staphylococcus aureus",
],
"mwt": [46.07, 45.08, 78.11],
"label": [0.1, 0.4, 0.8],
}
pipe = DataPipeline(
{
"morgan": ("smiles", "morgan-fingerprint"),
"desc2d": ("smiles", "descriptors-2d"),
"assay_hash": (
"assay",
{"name": "hash", "kwargs": {"ndim": 32}},
),
"species_fp": (
"species",
{"name": "vectome-fingerprint", "kwargs": {"ndim": 64}},
),
"log_mwt": ("mwt", "log"),
"y": ("label", "identity"),
},
columns_to_keep=["compound_id"],
)
features = pipe(
data,
drop_unused_columns=True,
).data
This requires the chemistry and taxonomy extras.
Downstream models can independently consume molecular fingerprints, descriptors, categorical features, and taxonomic encodings, or combine suitable dense features into a common representation.
Development
Install the development dependencies:
pip install -e ".[dev]"
Run the test suite:
pytest
Run module doctests:
pytest aspect --doctest-modules
Run the CLI integration tests:
bash test/scripts/run-tests.sh
Aspect maintains separate GitHub Actions workflows for Python tests and CLI smoke tests.
Related packages
Aspect is part of a small ecosystem for reproducible molecular machine-learning workflows:
- Providence — Dataset loading, source provenance, and checkpointing.
- Eluent — Chemistry-aware grouping, splitting, and cross-validation.
- Schemist — Molecular featurisation and cheminformatics.
- Vectome — Taxonomic and biological representations.
The packages can be used independently or composed into reproducible workflows from source data to model-ready features and evaluation splits.
Issues and documentation
Bug reports, feature requests, and contributions are welcome through the GitHub issue tracker.
Documentation: aspect.readthedocs.org
Source code: github.com/scbirlab/aspect
License: MIT.
Metadata
Release files for aspect-data 0.1.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| aspect_data-0.1.3.tar.gz | 81.6 kB | Without provenance | Details for aspect_data-0.1.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| aspect_data-0.1.3-py3-none-any.whl | Python 3 | none | any | Details for aspect_data-0.1.3-py3-none-any.whl |
Total release size: 134.8 kB
Release files / aspect_data-0.1.3.tar.gz
| Download URL | aspect_data-0.1.3.tar.gz |
|---|---|
| Size | 81.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f0e23c83077db1fc8df4e131dda98f50b5172c223128a325f262e6d7c3c845d7
|
|
BLAKE2b-256 checksum How to use checksums |
4773e89c243b3539eb71f4001f34307c96b03fec09f698b2b2a521d3ffad7651
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
Release files / aspect_data-0.1.3-py3-none-any.whl
| Download URL | aspect_data-0.1.3-py3-none-any.whl |
|---|---|
| Size | 53.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
430a72d92080d8295248caf96b93635a2f6a481420cf03abbba136ac7fe48fda
|
|
BLAKE2b-256 checksum How to use checksums |
cd4a927d91685059640db7791dca615efcfbeaa82f50c0ee8971a1ae6378e478
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
Security for aspect-data 0.1.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.