Skip to main content

An efficient space physics data client for python

Project description

das2py

Das2 servers typically provide data relevant to space plasma and magnetospheric physics research. To retrieve data, an HTTP GET request is posted to a das2 server by a client program and a self-describing stream of data values covering the requested time range, at the requested time resolution, is provided in the response body. This package, das2py provides an efficient space physics data client for python. Streams are parsed and stored as NumPy arrays using a C extension, avoiding data copies and conversions.

Anaconda Package

Anaconda Package

Pre-build versions of das2py are available from Anaconda. If you're working in an Anoconda or Miniconda python 3 environment these are easier to install as no C compiler is required. To install the conda package run the command:

(base) $ conda install -c dasdevelopers das2py

The anaconda package automatically pulls in das2C, pycdf, and pthreads4w as needed.

If this works then test using:

(base) $ wget https://raw.githubusercontent.com/das-developers/das2py/master/examples/ex05_mex_marsis_query_by_angle.py
(base) $ python ex02_galileo_pws_spectra.py

If this command produces a plot similar to the following...

...then das2py is installed, and you can skip building the software and head straight the example program below.

Prerequisite

Compilation and installation of das2py has been tested on Linux, Windows, MacOS using both Python 2 and Python 3. The following packages are required to build das2py:

  • Das2C - Version 2.3 or above. Need not be installed, but must be built
  • NumPy - Version 1.10.1 or above
  • MatplotLib++ - For plotting data (optional, recommended)
  • CDF - For writing CDF files (optional)

Pre-requisite package install commands are give below.

$ sudo apt install python3-setuptools python3-dev python3-numpy # debian

New build instructions (wheel)

In this version, almost no environment variables are needed Below, das2C is built adjacent to das2py so that it can be included in das2py.

# (Works on Linux, more steps needed on Windows & MacOS. 
#  See buildfiles/pypi/ReadMe.md for details)

# First build and test das2C, installation is not necessary
git clone git@github.com:das-developers/das2C.git
cd das2C
env N_ARCH=/ make 
env N_ARCH=/ make test
cd ../

# Now build das2py using the PIP of your choice in an adjacent directory
git clone git@github.com:das-developers/das2py.git
cd das2py
env DAS2C_LIBDIR=$PWD/../das2C/build. DAS2C_INCDIR=$PWD/../das2C python3.9 -m build

That's it! Now you have a wheel file that can be installed where ever you like. The included setup.py instructs the python setuptools module to staticlly link das2C. So the wheel is self contained.

To install your new wheel into the user-local area:

pip3.9 install ./dist/das2py-2.3.0-cp310-cp310-linux_x86_64.whl
ls .local/bin/das_verify
ls .local/lib/python3.9/site-packages/das2
ls .local/lib/python3.9/site-packages/_das2*

To test it by validating one of the example files...

das_verify das2py/test/ex96_yscan_multispec.d2t

and to generate a plot of Cassini electron cyclotron frequencies.

python3.9 das2py/examples/ex09_cassini_fce_ephem_ticks.py 2017-09-14
okular cas_mag_fce_2017-09-14.png # Or whatever PNG viewer you like

Old Build and Install

The old makefile based build and install wrapper is still supported. It's based on distutils, and it works for Python 2.7 and up. For newer versions of python the pip-wheel build is recommended, for Python2 support read on...

Decide where you want to install das2py. In the example below I've selected /usr/local/lib/python3.9/site-packages but any location is fine so long as it is on the PYTHONPATH or you are willing to add it to your PYTHONPATH.

# Where to find the das2C static library
$ export DAS2C_INCDIR=$HOME/git/das2C
$ export DAS2C_LIBDIR=$HOME/git/das2C/build.  #last dot is not a typo

# Which python version to use
$ export PYVER=3.9

# Where you want to install the files
$ export INST_HOST_LIB=/usr/local/lib/python3.9
$ export INST_EXT_LIB=/usr/local/lib/python3.9

# Build and test
$ make           # <-- If only using system packages
$ make local     # <-- If using numpy or others from $HOME/.local
                 #     Do not use for RPM/DEB builds
$ make test

# Check install location, then install
$ make -n install
$ make install

First program

The following small program demonstrates how to query for data and generate a plot using das2py.

Query a URI for reduced resolution data

import das2
src = das2.get_source( 'tag:das2.org,2012:site:/uiowa/galileo/pws/survey_electric/das2' )
dataset = src.get( {'time' : ('1997-05-07T15:00', '1997-05-07T17:00', 4.0)} )[0] 
  • Servers come and go. The federated catalog provides stability for application code by maping URIs to data sources.
  • Browse for URIs using a das2 catalog browser, or using das2py to get the root node (see example 11).

Access quantities by physical dimensions

print(dataset)

vX = dataset['time']['center']
vY = dataset['frequency']['center']
vZ = dataset['electric']['center'] 
  • Datasets contain dimensions. Dimensions contain variables. Each variable in a dimension serves a purpose, the most common is to define the center point of a coordinate or measurement.
  • Array dimensions are not confused with physical dimensions. Dataset meaning is not tied to any particular array morphology.

Use Matplotlib to generate an image.

import matplotlib.pyplot as pyplot
import matplotlib.colors as colors

fig, ax = pyplot.subplots()
scaleZ = colors.LogNorm(vmin=vZ.array.min(), vmax=vZ.array.max())
ax.pcolormesh(vX.array, vY.array, vZ.array, norm=scaleZ, cmap='jet')
pyplot.show() 

Reporting bugs

Please use the github.com issue tracker report any problems with the library. If you've fixed a bug, 1) thanks!, 2) please send a pull request so that your updates can be merged into the main project.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

das2py-2.3.1-cp310-cp310-win_amd64.whl (2.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

das2py-2.3.1-cp310-cp310-macosx_14_0_arm64.whl (2.2 MB view details)

Uploaded CPython 3.10 macOS 14.0+ ARM64

das2py-2.3.1-cp310-cp310-macosx_12_0_x86_64.whl (3.2 MB view details)

Uploaded CPython 3.10 macOS 12.0+ x86-64

File details

Details for the file das2py-2.3.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: das2py-2.3.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 2.3 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.11

File hashes

Hashes for das2py-2.3.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 e1c406b5ad8ebde82fd85ef6b9633af51d32293b123cfd056dc812ac4d3d6420
MD5 9f54f92674db83f97b80755f06f12a06
BLAKE2b-256 a71795d1776734d93a94a33c5b6a29a8e07d76d4c7a9b5ac13fd76071d7d3039

See more details on using hashes here.

File details

Details for the file das2py-2.3.1-cp310-cp310-macosx_14_0_arm64.whl.

File metadata

File hashes

Hashes for das2py-2.3.1-cp310-cp310-macosx_14_0_arm64.whl
Algorithm Hash digest
SHA256 5054ad211e8873b3c686f6b6422ae7568b85e183f5282a30629523a360b1f7d0
MD5 b0a13dbaf80b53789e8836dd3de98faf
BLAKE2b-256 b995015b7e4d25e714be7e58b138f1908b676371384f69a6c36de8409a67a54e

See more details on using hashes here.

File details

Details for the file das2py-2.3.1-cp310-cp310-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for das2py-2.3.1-cp310-cp310-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 107fa0b5423e328fbf56373a8526186ad31bcc513fa17f66a2e5b2cba63d0688
MD5 1532d180a5554e9afe8ca243036bb15f
BLAKE2b-256 1fa1bea721c19528c2bc5015d31ecd95ef333a2cc86e537a4e6fe20b0787fe36

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page