Skip to main content

Tools and utilities for buoyancy glider data processing.

Project description

Buoyancy Glider Utils
=====================

This is a tool to read and process Seaglider data from basestation netCDF format to a level2 QC phase.
For more information see the documentation of the main variable processing functions with the `calc_` prefix.
Below is a short example of how to use the data to read in and process variables.


EXAMPLE USAGE
-------------
This package is meant to be used in an interactive environment - ideally Jupyter Notebook

```python
%pylab inline
import BuoyancyGliderUtils as bgu

sg543 = bgu.seaglider.load_basestation_netCDF_files('path_to_data/p5430*.nc')

# Load all variables needed for processing (excluding time, position and depth dimensions)
sg543.load_multiple_vars([
'temperature',
'salinity',
'pressure',
'eng_wlbb2flvmt_Chlsig',
'eng_wlbb2flvmt_wl470sig',
'eng_wlbb2flvmt_wl700sig',
'eng_qsp_PARuV',
])

# DataFrame of science variables with dimension `sg_data_point`
df = sg543.data['sg_data_point']

# Remove outliers and smooth temperature and salinity
temp = bgu.calc_physics(df.temperature, df.dives, df.depth)
salt = bgu.calc_physics(df.salinity, df.dives, df.depth)

# Backscattering processing and despiking
bb1, bb1_spikes, fig1 = bgu.calc_bb1(df.eng_wlbb2flvmt_wl470sig, temp, salt, df.dives, df.depth, 50, 1.29e-5)
bb2, bb2_spikes, fig2 = bgu.calc_bb2(df.eng_wlbb2flvmt_wl700sig, temp, salt, df.dives, df.depth, 50, 3.50e-6)

# Get theoretical PAR curve from exponential curve fit
par = bgu.calc_par(df.eng_qsp_PARuV, df.dives, df.depth, t, 6.678e-4, 10.6)

# Despike fluorescence and correct for quenching
flr_qnch, flr, qnch_layer, figs = bgu.calc_fluorescence(
df.eng_wlbb2flvmt_Chlsig, bb1, par,
df.dives, df.depth, df.time, df.latitude, df.longitude,
50, 0.0173, return_figure=True)

# See the demo file for more info
```


ABOUT
-----
This work was funded by the CSIR (where Luke was working at the time of writing the code).

- Version: 0.4.5
- Author: Luke Gregor, Tommy Ryan-Keogh
- Email: lukegre@gmail.com
- Date: 2018-12-18
- Institution: Council for Scientific and Industrial Research
- Research group: Southern Ocean Carbon - Climate Observatory (SOCCO)

Please use the guidlines given on https://integrity.mit.edu/handbook/writing-code to cite this code.

**Example citation:**
Source: buoyancy_glider_utils [https://gitlab.com/socco/BuoyancyGliderUtils](https://gitlab.com/socco/BuoyancyGliderUtils) retrieved on 15 July 2018.


CHANGE LOG
----------
**v0.4.5** (2018-12-18)

- printv function replaces if verbose: print (neater)
- figure text is now slightly bigger
- improved savitzky_golay - data not lost when there are nans
- quenching_correction no longer requires PAR / quenching mask

**v0.4.2** (2018-11-12)

- cleaning functions now seperated into seperate script
- plotting made more robust with catches if x is not psuedo-discrete
- blob outlier detection added

**v0.4.1** (2018-10-31)

- `seperate_spikes` has been moved to tools and has been renamed to `despike`
- quenching correction has been fixed
- `bin_depths` renamed to `grid_data` and input is now `x, y, var` instead of `var, x, y`
- `neighbourhood_iqr` has been deprecated! It wasn't doing anything
- fixed despiking and quenching reports (figures)

**v0.4** (2018-10-19)

- Added `calc_oxygen` module for both sbe and aanderaa oxygen optodes
- Added 3D plotting function `plot.section3D`
- `optics.seperate_spikes` added to calc_physics
- Updated `optics.find_bad_profiles` to either use deep mean or median
- User control on existing functions more explicit and robust


**v0.3.4** (2018-08-31)

- Photic depth function has been updated and is now more robust
- Package now available on pypi


**v0.3** (2018-07-23)

- Added calibration module that lets users calibrate gliders from bottle data
- A new density function that calculates density from corrected salinity and uses surface pressure as a reference pressure
- seaglider module now has a `load_basestation_netCDF_files` function that makes loading data more explicit.
- updated the demo file to include the new functions.

**v0.2** (2018-07-13)

- fixed bug when correcting for quenching (`index[0] does not exist` reported by Tommy)
- made an MLD function that works on ungridded data that returns output as a mask or depths
- `plot.bin_size` now accounts for nans that slocum glider data has
- quenching correction can now be calculated without PAR as input
- only one backscatter function - now the wavelength needs to be set
- section plots are rasterized
- deprication warning (`DataFrame.from_items`) in `optics.sunrise_sunset`
- Slocum output from GEOMAR MATLAB scripts can be imported with `slocum.load_geomar_slocum_matfile`
- `SeaGliderDataset.load_multiple_vars` returns the DataFrame if one dimension otherwise a dictionary of DataFrames
- Added a function `tools.mask_to_depth` to convert boolean layers to depths as a series indexed by dive number


PACKAGE STRUCTURE
-----------------
NOTE: This package structure is defined by the `__init__.py` file

- calc_physics - for temperature and salinity
- calc_bb - backscatter
- calc_fluorescence - fluorescence and quenching correction
- calc_par - get theoretical PAR
- calibration
- bottle_matchup
- model_figs
- robust_linear_fit
- plot
- pcolormesh
- contourf
- scatter
- bin_size
- save_figures_to_pdf
- section3D
- tools
- grid_data
- mask_to_depth
- time_average_per_dive
- merge_dimensions
- gaussian_kernel
- cleaning
- despike
- savitzky_golay
- despiking_report
- rolling_window
- mask_bad_dive_fraction
- horizontal_diff_outliers
- outlier_bounds_iqr
- outlier_bounds_std
- optics
- backscatter_dark_count
- find_bad_profiles
- fluorescence_dark_count
- par_dark_count
- par_fill_surface
- par_scaling
- photic_depth
- quenching_correction
- quenching_report
- sunset_sunrise
- physics
- mixed_layer_depth
- potential_density
- brunt_vaisala
- seaglider
- load_basestation_netCDF_files
- slocum
- load_geomar_slocum_matfile


ACKNOWLEDGEMENTS
----------------
- ion_functions.data.flo_functions: Christopher Wingard, Craig Risien, Russell Desiderio
- Tommy Ryan-Keogh and the other co-authors of Thomalla et al. (2017)
- The SOCCO team for feedback and bug reporting
- gibbs seawater package and seawater packages are used in BGU


TO DO
-----
- Integrate the EGO gliders NetCDF format (http://dx.doi.org/10.13155/34980)
- Add the concept of data provenence and processing tracking to the package.
This may require files to be returned as NetCDF from the processing functions


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for buoyancy-glider-utils, version 0.4.5
Filename, size & hash File type Python version Upload date
buoyancy_glider_utils-0.4.5-py3-none-any.whl (47.1 kB) View hashes Wheel py3
buoyancy_glider_utils-0.4.5.tar.gz (45.2 kB) View hashes Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page