Skip to main content

Detrending Periodic Signals (dips)

dips is an algorithm for detrending timeseries of strictly periodic signals. It does not assume any functional form for the signal or the background or the noise; it disentangles the strictly periodic component from everything else. We use it in astronomy for detrending Kepler, K2 and TESS timeseries of periodic variable stars, eclipsing binary stars, exoplanets etc. The algorithm is described in detail in Prsa et al. (2019), PASP, in review -- the reference will be updated shortly.

This repository contains a python 3 implementation of dips.

Pre-requisites

To run dips, you will need:

  • python 3
  • numpy
  • scipy

Installation

The dips program is available from pip. To install, run pip3 install dips for a local install, or sudo pip3 install dips for a global install.

If you prefer to install dips manually, grab the tarball from github, extract it and run python3 setup.py install in the top-level dips directory (local install), or sudo python3 setup.py install in the top-level dips directory (global install).

Running dips

The dips program is run from the command line. It takes a filename with the timeseries as input and computes the disentangled synchronous and asynchronous components of the signal as output. The disentangling process is iterative and might take an appreciable amount of time, depending on data length and pdf bin size.

Run dips with:

dips.py [-h] [-V] [-b BINS] [-t0 ORIGIN] [-P PERIOD] [-l LOGFILE] [-eta TOLERANCE] [-dxk DIFFERENCE] [-xi STEP_SIZE] [-af ATTENUATION] [--allow-upstep] [--cols COLS [COLS ...]] [--disable-mp] [--initial-pdf INITIAL_PDF] [--interim-prefix INTERIM_PREFIX] [--jitter JITTER] [--output-prefix OUTPUT_PREFIX] [--renormalize] [--save-interim SAVE_INTERIM] [--yonly] finput

The arguments are summarized in the table below.

Argument Usage Type Default value
-h, --help print out the help message and exit n/a n/a
-V, --version print dips version and exit n/a n/a
-b BINS, --bins BINS assign the number of synchronous pdf bins int 200
-t0 ORIGIN, --origin ORIGIN the zero-point of the timeseries float 0.0
-P PERIOD, --period PERIOD period of the synchronous signal float 1.0
-l LOGFILE, --logfile LOGFILE log file to send output to instead of screen str None
-eta TOLERANCE, --tolerance TOLERANCE tolerance for convergence float 1e-8
-dxk DIFFERENCE, --difference DIFFERENCE finite difference size for computing slopes float 2e-5
-xi STEP_SIZE, --step-size STEP_SIZE initial down-step multiplier float 1e-3
-af ATTENUATION, --attenuation ATTENUATION attenuation factor for xi float 0.9
--allow-upstep allow step size to increase during convergence bool False
--cols COL1 COL2 [COL3] a list of input columns to be parsed, starting from 0 list of ints 0 1
--disable-mp disable multiprocessing (force serial computation) bool False
--initial-pdf choice of pdf initialization ('flat', 'mean', 'median', 'random', or external filename) str 'median'
--interim-prefix filename prefix for interim results str finput
--output_prefix PREFIX filename prefix for saving results (PREFIX.signal, .trend, .ranges) str finput
--renormalize force pdf normalization to 1 after every iteration bool False
--save-interim STEP save intering solutions every STEP iterations int 0
--yonly use only y-distance instead of full euclidian distance bool False

Distributed with dips (in the tarball's examples directory) are three example input files, synthetic.data, kic3953981_sap.data and kic3547874_sap.data.

To run dips on synthetic data (see http://keplerEBs.villanova.edu/includes/DPS/dps_synthetic.html how the data were created) by using 33 bins, per-bin means as the initial pdf, and with serial calculation (disabling multiprocessing), issue:

dips synthetic.data -b 33 -P 0.91 --initial-pdf mean --disable-mp

To run dips on an eclipsing binary KIC 3953981, using 101 bins, allowing the step size to increase, using per-bin data median as the initial pdf, renormalizing the pdf after each iteration, using only y-direction length and saving every 10th iteration, issue:

dips kic3953981_sap.data -b 101 -t0 54953.82253243 -P 0.49201716 --allow-upstep --initial-pdf median --save-interim 10 --interim-prefix eb --renormalize --yonly

Finally, to run dips on a heartbeat star KIC 3547874, using 200 bins, starting with a flat pdf, computing total length in the y-direction only, renormalizing the synchronous pdf to 1.0 after each iteration, and allowing the step size to increase, issue:

dips kic3547874_sap.data --cols 0 2 -t0 54989.4209 -P 19.6921722 -b 200 --yonly --initial-pdf flat --renormalize --allow-upstep

These examples should provide a basic idea of how to invoke dips.

Metadata

Release files for dips 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dips 0.1.0
File Size Uploaded Provenance Actions
dips-0.1.0.tar.gz 1.8 MB Without provenance Details for dips-0.1.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for dips 0.1.0
File Interpreter ABI Platform Actions
dips-0.1.0-py3.6.egg Legacy Egg format - - Details for dips-0.1.0-py3.6.egg
dips-0.1.0-py3-none-any.whl Python 3 none any Details for dips-0.1.0-py3-none-any.whl

Total release size: 1.9 MB

Release files / dips-0.1.0.tar.gz

Download URL dips-0.1.0.tar.gz
Size 1.8 MB
Tags Source
SHA-256 checksum
How to use checksums
27feba9c5d63591254e5ba328616764a43750cb7521c41724470d7dbf25627f3
BLAKE2b-256 checksum
How to use checksums
8009510d829228a6775025c2b54884382a95d9499941b80bd6849c077cb85fb8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

Release files / dips-0.1.0-py3.6.egg

Download URL dips-0.1.0-py3.6.egg
Size 11.0 kB
Tags Egg
SHA-256 checksum
How to use checksums
43ad8627fc1e8a063810fe6fa8150d8f8965b8430ee6be4c13b36f56428bff7d
BLAKE2b-256 checksum
How to use checksums
1a52148fa197c27e6a1aac53a1736d0b5f4064974b7e4228047d2154aa6a4730
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

Release files / dips-0.1.0-py3-none-any.whl

Download URL dips-0.1.0-py3-none-any.whl
Size 19.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
713b0fb190de238d213a5c9afa2ce0cc439e73ab37c46233023f55a6488d8a11
BLAKE2b-256 checksum
How to use checksums
6ad968ccc1fcdc3bf3fb30dcb84085d9bc63b5b46cca1d8ebd1c18b5f424d74b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

Security for dips 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

3 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page