Timeseriesflattener
Time series from e.g. electronic health records often have a large number of variables, are sampled at irregular intervals and tend to have a large number of missing values. Before this type of data can be used for prediction modelling with machine learning methods such as logistic regression or XGBoost, the data needs to be reshaped.
In essence, the time series need to be flattened so that each prediction time is represented by a set of predictor values and an outcome value. These predictor values can be constructed by aggregating the preceding values in the time series within a certain time window.
timeseriesflattener aims to simplify this process by providing an easy-to-use and fully-specified pipeline for flattening complex time series.
🔧 Installation
To get started using timeseriesflattener simply install it using pip by running the following line in your terminal:
pip install timeseriesflattener
⚡ Quick start
import datetime as dt
import numpy as np
import polars as pl
# Load a dataframe with times you wish to make a prediction
prediction_times_df = pl.DataFrame(
{"id": [1, 1, 2], "date": ["2020-01-01", "2020-02-01", "2020-02-01"]}
)
# Load a dataframe with raw values you wish to aggregate as predictors
predictor_df = pl.DataFrame(
{
"id": [1, 1, 1, 2],
"date": ["2020-01-15", "2019-12-10", "2019-12-15", "2020-01-02"],
"predictor_value": [1, 2, 3, 4],
}
)
# Load a dataframe specifying when the outcome occurs
outcome_df = pl.DataFrame({"id": [1], "date": ["2020-03-01"], "outcome_value": [1]})
# Specify how to aggregate the predictors and define the outcome
from timeseriesflattener import (
MaxAggregator,
MinAggregator,
OutcomeSpec,
PredictionTimeFrame,
PredictorSpec,
ValueFrame,
)
predictor_spec = PredictorSpec(
value_frame=ValueFrame(
init_df=predictor_df, entity_id_col_name="id", value_timestamp_col_name="date"
),
lookbehind_distances=[dt.timedelta(days=1)],
aggregators=[MaxAggregator(), MinAggregator()],
fallback=np.nan,
column_prefix="pred",
)
outcome_spec = OutcomeSpec(
value_frame=ValueFrame(
init_df=outcome_df, entity_id_col_name="id", value_timestamp_col_name="date"
),
lookahead_distances=[dt.timedelta(days=1)],
aggregators=[MaxAggregator(), MinAggregator()],
fallback=np.nan,
column_prefix="outc",
)
# Instantiate TimeseriesFlattener and add the specifications
from timeseriesflattener import Flattener
result = Flattener(
predictiontime_frame=PredictionTimeFrame(
init_df=prediction_times_df, entity_id_col_name="id", timestamp_col_name="date"
)
).aggregate_timeseries(specs=[predictor_spec, outcome_spec])
result.df
Output:
| id | date | prediction_time_uuid | pred_test_feature_within_30_days_mean_fallback_nan | outc_test_outcome_within_31_days_maximum_fallback_0_dichotomous | |
|---|---|---|---|---|---|
| 0 | 1 | 2020-01-01 00:00:00 | 1-2020-01-01-00-00-00 | 2.5 | 0 |
| 1 | 1 | 2020-02-01 00:00:00 | 1-2020-02-01-00-00-00 | 1 | 1 |
| 2 | 2 | 2020-02-01 00:00:00 | 2-2020-02-01-00-00-00 | 4 | 0 |
📖 Tutorial
💬 Where to ask questions
| Type | |
|---|---|
| 🚨 Bug Reports | GitHub Issue Tracker |
| 🎁 Feature Requests & Ideas | GitHub Issue Tracker |
| 👩💻 Usage Questions | GitHub Discussions |
| 🗯 General Discussion | GitHub Discussions |
🎓 Projects
PSYCOP projects use timeseriesflattener, see more at the monorepo.
Metadata
Release files for timeseriesflattener 2.5.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| timeseriesflattener-2.5.2.tar.gz | 9.6 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| timeseriesflattener-2.5.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 18.2 MB
Release files / timeseriesflattener-2.5.2.tar.gz
| Download URL | timeseriesflattener-2.5.2.tar.gz |
|---|---|
| Size | 9.6 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3e2e0a72e34f40e968b3a809cb24907db2872098cf93d1b24bec01eac848fb61
|
|
BLAKE2b-256 checksum How to use checksums |
4b3e5f53871b1e0278cf6a1c9fc1422f8cfba43174297d7795574747ea449ccb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.12.1.2 readme-renderer/44.0 requests/2.32.4 requests-toolbelt/1.0.0 urllib3/2.4.0 tqdm/4.67.1 importlib-metadata/8.7.0 keyring/25.6.0 rfc3986/2.0.0 colorama/0.4.6 CPython/3.10.18
|
Release files / timeseriesflattener-2.5.2-py3-none-any.whl
| Download URL | timeseriesflattener-2.5.2-py3-none-any.whl |
|---|---|
| Size | 8.6 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4de3b77f8f7d2a456c223ffa5f260ee9ad972484ff90d976e5fa3692fdc4b0fe
|
|
BLAKE2b-256 checksum How to use checksums |
4f2f643fcd8a977494de6450d39043f6a732b4b3829eace0219d46980b781327
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.12.1.2 readme-renderer/44.0 requests/2.32.4 requests-toolbelt/1.0.0 urllib3/2.4.0 tqdm/4.67.1 importlib-metadata/8.7.0 keyring/25.6.0 rfc3986/2.0.0 colorama/0.4.6 CPython/3.10.18
|