Skip to main content

Scikit Data Access Package for accessing scientific data sets.

Project description

Scikit Data Access

  • Import scientific data from various sources through one easy Python API.
  • Use iterator patterns for each data source (configurable data generators + functions to get next data chunk).
  • Skip parser programming and file format handling.
  • Enjoy a common namespace for all data and unleash the power of data fusion.
  • Handle data distribution in different modes: (1) local download, (2) caching of accessed data, or (3) online stream access
  • Easily pull data on cloud servers through Python scripts and facilitate large-scale parallel processing.
  • Build on an extensible plattform: Adding access to a new data source only requires addition of its "DataFetcher.py".
  • Open source (MIT License)

Scikit Data Access Overview

Supported data sets:

Namespace

Preview
(link)

Description & Data Source

Astronomy

astro.kepler Preview Light curves for stars imaged by the NASA Kepler Space Telescope
Source: https://keplerscience.arc.nasa.gov

astro.spectra Preview Spectra from the Sloan Digital Sky Survey
Source: https://www.sdss.org/dr14/spectro/

astro.tess.data Preview Light curves from TESS Data Alerts
Source: https://archive.stsci.edu/prepds/tess-data-alerts/

astro.tess.simulated Preview Simulated light curves from TESS End-to-End 6
Source: https://archive.stsci.edu/prepds/tess-data-alerts/

astro.voyager Preview Data from the Voyager mission.
Source: https://spdf.gsfc.nasa.gov/
Engineering

engineering.la.traffic_counts Preview Traffic Count data in Los Angeles.
Source: https://data.lacity.org/A-Livable-and-Sustainable-City/LADOT-Traffic-Counts-Summary/94wu-3ps3

engineering.webcam.mit_sailing Preview Images from webcams located at the MIT Sailing Pavilion
Source: http://sailing.mit.edu/webcam.php
Finance

finance.timeseries Preview Financial time series data retrieved using Alpha Vantage API.
Source: https://www.alphavantage.co/
Geoscience

geo.era_interim Preview Era-Interim data at different pressure values from
the European Centre for Medium-Range Weather Forecasts accessed through the University Corporation for Atmospheric Research.
Source: https://rda.ucar.edu/datasets/ds627.0/

geo.gldas Preview Land hydrology model produced by NASA. This version of the data is generated to match the GRACE temporal and spatial characteristics and is available as a complementary data product.
Source: https://grace.jpl.nasa.gov/data/get-data/land-water-content

geo.grace Preview NASA GRACE Tellus Monthly Mass Grids. 30-day measurements of changes in Earth’s gravity field to quantify equivalent water thickness.
Source: https://grace.jpl.nasa.gov/data/get-data/monthly-mass-grids-land

geo.grace.mascon Preview NASA GRACE Tellus Monthly Mass Grids - Global Mascons. 30-day measurements of changes in Earth’s gravity field to quantify equivalent water thickness. Source: https://grace.jpl.nasa.gov/data/get-data/jpl_global_mascons

geo.groundwater Preview United States groundwater monitoring wells measuring the depth to water level. Source: https://waterservices.usgs.gov

geo.magnetometer Preview Data collected at magnetic observatories operated by the U.S. Geological Survey. Source: https://geomag.usgs.gov

geo.mahali.rinex Preview Rinex files from the MIT led NSF project studying the Earth’s ionosphere with GPS.
Web: http://mahali.mit.edu

geo.mahali.tec Preview Total Electron Content from the MIT led NSF project studying the Earth’s ionosphere with GPS.
Web:http://mahali.mit.edu

geo.mahali.temperature Preview Temperature data from the MIT led NSF project studying the Earth’s ionosphere with GPS.
Web: http://mahali.mit.edu

geo.modis Preview Spectroradiometer aboard the NASA Terra and Aqua image satellites. Generates approximately daily images of the Earth’s surface.
Source:https://modis.gsfc.nasa.gov

geo.pbo Preview EarthScope - Plate Boundary Observatory (PBO): Daily GPS displacement time series measurements throughout the United States.
Source: http://www.unavco.org/projects/major-projects/pbo/pbo.html

geo.sentinel_1 Preview Sentinel-1 TOPSAR data from the European Space Agency retrieved from the Alaska Satellite Facility.
Source:https://www.asf.alaska.edu/

geo.srtm Preview Elevation data at a one arc second resolution from the Shuttle Radar Topography Mission (SRTMGL1).
Source: https://lpdaac.usgs.gov/dataset_discovery/measures/measures_products_table/srtmgl1_v003

geo.uavsar Preview UAVSAR SLC data from JPL.
Source: https://uavsar.jpl.nasa.gov/

geo.wyoming_sounding Preview Sounding data from the University of Wyoming.
Source: http://weather.uwyo.edu/upperair/sounding.html
Planetary Science

planetary.ode Preview Mars planetary data from PDS Geosciences Node's Orbital Data Explorer.
Source: http://pds-geosciences.wustl.edu/default.htm
Solar Science

solar.sdo Preview Images from the Solar Dynamics Observatory.
Source: https://sdo.gsfc.nasa.gov/

Install

pip install scikit-dataaccess

Documentation

Contributors

Project lead: Victor Pankratius (MIT)
Contributors: Cody M. Rude, Justin D. Li, David M. Blair, Michael G. Gowanlock, Guillaume Rongier, Victor Pankratius

New contributors welcome! Contact to contribute and add interface code for your own datasets :smile:

Acknowledgements

We acknowledge support from NASA AIST14-NNX15AG84G, NASA AIST16-80NSSC17K0125, NSF ACI-1442997, and NSF AGS-1343967.

Examples

Code examples (Jupyter notebooks) for all datasets listed above are available at: /skdaccess/examples

Scikit Data Access Overview

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for scikit-dataaccess, version 1.9.17.post2
Filename, size File type Python version Upload date Hashes
Filename, size scikit-dataaccess-1.9.17.post2.tar.gz (20.1 MB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page