Skip to main content

Scikit Data Access Package for accessing scientific data sets.

Project description

Scikit Data Access

  • Import scientific data from various sources through one easy Python API.
  • Use iterator patterns for each data source (configurable data generators + functions to get next data chunk).
  • Skip parser programming and file format handling.
  • Enjoy a common namespace for all data and unleash the power of data fusion.
  • Handle data distribution in different modes: (1) local download, (2) caching of accessed data, or (3) online stream access
  • Easily pull data on cloud servers through Python scripts and facilitate large-scale parallel processing.
  • Build on an extensible plattform: Adding access to a new data source only requires addition of its "DataFetcher.py".
  • Open source (MIT License)

Scikit Data Access Overview

Supported data sets:

Namespace

Preview
(link)

Description & Data Source

Astronomy

astro.kepler Preview Light curves for stars imaged by the NASA Kepler Space Telescope
Source: https://keplerscience.arc.nasa.gov

astro.spectra Preview Spectra from the Sloan Digital Sky Survey
Source: https://www.sdss.org/dr14/spectro/

astro.tess.data Preview Light curves from TESS Data Alerts
Source: https://archive.stsci.edu/prepds/tess-data-alerts/

astro.tess.simulated Preview Simulated light curves from TESS End-to-End 6
Source: https://archive.stsci.edu/prepds/tess-data-alerts/

astro.voyager Preview Data from the Voyager mission.
Source: https://spdf.gsfc.nasa.gov/
Engineering

engineering.la.traffic_counts Preview Traffic Count data in Los Angeles.
Source: https://data.lacity.org/A-Livable-and-Sustainable-City/LADOT-Traffic-Counts-Summary/94wu-3ps3

engineering.webcam.mit_sailing Preview Images from webcams located at the MIT Sailing Pavilion
Source: http://sailing.mit.edu/webcam.php
Finance

finance.timeseries Preview Financial time series data retrieved using Alpha Vantage API.
Source: https://www.alphavantage.co/
Geoscience

geo.era_interim Preview Era-Interim data at different pressure values from
the European Centre for Medium-Range Weather Forecasts accessed through the University Corporation for Atmospheric Research.
Source: https://rda.ucar.edu/datasets/ds627.0/

geo.gldas Preview Land hydrology model produced by NASA. This version of the data is generated to match the GRACE temporal and spatial characteristics and is available as a complementary data product.
Source: https://grace.jpl.nasa.gov/data/get-data/land-water-content

geo.grace Preview NASA GRACE Tellus Monthly Mass Grids. 30-day measurements of changes in Earth’s gravity field to quantify equivalent water thickness.
Source: https://grace.jpl.nasa.gov/data/get-data/monthly-mass-grids-land

geo.grace.mascon Preview NASA GRACE Tellus Monthly Mass Grids - Global Mascons. 30-day measurements of changes in Earth’s gravity field to quantify equivalent water thickness. Source: https://grace.jpl.nasa.gov/data/get-data/jpl_global_mascons

geo.groundwater Preview United States groundwater monitoring wells measuring the depth to water level. Source: https://waterservices.usgs.gov

geo.magnetometer Preview Data collected at magnetic observatories operated by the U.S. Geological Survey. Source: https://geomag.usgs.gov

geo.mahali.rinex Preview Rinex files from the MIT led NSF project studying the Earth’s ionosphere with GPS.
Web: http://mahali.mit.edu

geo.mahali.tec Preview Total Electron Content from the MIT led NSF project studying the Earth’s ionosphere with GPS.
Web:http://mahali.mit.edu

geo.mahali.temperature Preview Temperature data from the MIT led NSF project studying the Earth’s ionosphere with GPS.
Web: http://mahali.mit.edu

geo.modis Preview Spectroradiometer aboard the NASA Terra and Aqua image satellites. Generates approximately daily images of the Earth’s surface.
Source:https://modis.gsfc.nasa.gov

geo.pbo Preview EarthScope - Plate Boundary Observatory (PBO): Daily GPS displacement time series measurements throughout the United States.
Source: http://www.unavco.org/projects/major-projects/pbo/pbo.html

geo.sentinel_1 Preview Sentinel-1 TOPSAR data from the European Space Agency retrieved from the Alaska Satellite Facility.
Source:https://www.asf.alaska.edu/

geo.srtm Preview Elevation data at a one arc second resolution from the Shuttle Radar Topography Mission (SRTMGL1).
Source: https://lpdaac.usgs.gov/dataset_discovery/measures/measures_products_table/srtmgl1_v003

geo.uavsar Preview UAVSAR SLC data from JPL.
Source: https://uavsar.jpl.nasa.gov/

geo.wyoming_sounding Preview Sounding data from the University of Wyoming.
Source: http://weather.uwyo.edu/upperair/sounding.html
Planetary Science

planetary.ode Preview Mars planetary data from PDS Geosciences Node's Orbital Data Explorer.
Source: http://pds-geosciences.wustl.edu/default.htm
Solar Science

solar.sdo Preview Images from the Solar Dynamics Observatory.
Source: https://sdo.gsfc.nasa.gov/

Install

pip install scikit-dataaccess

Documentation

Contributors

Project lead: Victor Pankratius (MIT)
Contributors: Cody M. Rude, Justin D. Li, David M. Blair, Michael G. Gowanlock, Guillaume Rongier, Victor Pankratius

New contributors welcome! Contact to contribute and add interface code for your own datasets :smile:

Acknowledgements

We acknowledge support from NASA AIST14-NNX15AG84G, NASA AIST16-80NSSC17K0125, NSF ACI-1442997, and NSF AGS-1343967.

Examples

Code examples (Jupyter notebooks) for all datasets listed above are available at: /skdaccess/examples

Scikit Data Access Overview

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
scikit-dataaccess-1.9.17.post2.tar.gz (20.1 MB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page