Library for working with RedVox files.
Project description
### RedVox Python SDK
This repository contains code for reading and working with the RedVox API 900 and API 1000 (M) data formats.
[User Documentation](/docs/python_sdk)
[API Documentation](https://redvoxinc.github.io/redvox-sdk/v3.0.0b0/api_docs/redvox/index.html)
## Changelog
### 3.0.0b3 (2021-3-1) * Updated timing correction using offset model predictions * Added timing correction to file searching functionality
### 3.0.0b2 (2021-2-3)
Updated filter inputs to accept lists
Fixed bug when pickle-ing station objects
### 3.0.0b1 (2021-2-1)
Added functionality to automatically correct query times to gather all of user’s requested data
### 3.0.0b0 (2021-2-1)
First beta release of version 3
Combined API 900 and API 1000 file reading
Updated file reading interface
Timing correction
Cleanup cloud APIs
Access to new RedVox Cloud APIs (station status, timing stats)
Persistent RedVox cloud configuration files
Cleanup cyclic dependency issues
Fix bugs with nan handling
Add ability to sort unstructured RedVox files into structured RedVox files
### 3.0.0a23 (2021-1-7)
Update api900 file reader to truncate file timestamps instead of round
Update api900 to api1000 conversion to properly convert locations
Added functions to read api900 and api1000 data from common base directory
Use api900 to api1000 conversion method to put data from both formats into a DataWindow
### 3.0.0a22 (2021-1-5)
Update LZ4 compression to be compatible with other LZ4 libraries that work with API M data
### 3.0.0a21 (2020-11)
Add setters for all sub-messages
Extend start and end queries for reading data by slight amount
fixed a bug when creating stations using set locations
API M protobuf updated
fixed bug setting compressed audio data
### 3.0.0a20 (2020-11-16)
Add support for downloading API M data from cloud servers
Optimize data download using a custom work-stealing process pool
Standardize CLI
### 3.0.0a19 (2020-11-10)
Fixed bug with recalculating metadata of stations
### 3.0.0a18 (2020-11-09)
STATION_HEALTH as sensor type to consolidate battery health, power state, internal temperature, network health, available disk space and RAM, and cell service status into one sensor with the same sample rate and source (station_metrics)
Time sync data has been added to the station object
### 3.0.0a17 (2020-11-04)
Image sensors properly load into data windows
Updated LocationSensors to use best_locations in API M location sensors
### 3.0.0a16 (2020-10-27)
LocationData added; stores location data of a Station for later use
DataWindow will compute the most relevant locations into LocationData
Added sample interval standard deviation to SensorData
Resolved discrepancies between metadata values and data being read
### 3.0.0a15 (2020-10-20)
Add ability to post process derived movement events
### 3.0.0a14 (2020-10-06)
Add ability to query API M metadata from the Cloud HTTP API
Add methods and classes for working with derived movement results
### 3.0.0a13 (2020-10-01)
DataWindow will properly bound the data within the start and end times
DataWindow will fill gaps in data based on the existing data’s sample interval
DataWindow will keep previous locations if none exist between start and end times
Sensors will properly preserve the mean sample interval as new data is added
Added function to get time sync exchanges from API M packet
### 3.0.0a12 (2020-09-29)
Add methods for lazily streaming data from the file system.
### 3.0.0a11 (2020-09-29)
Add gallery command to CLI for viewing API M image data
### 3.0.0a10 (2020-09-24)
Data reading functions include location provider value
Data reading functions include the 2 nearest points if truncation removes all data
### 3.0.0a9 (2020-09-23)
Data reading functions properly set station’s best latency and offset
Updated DataWindow default values for gap duration to 0.5 seconds and padding to 120 seconds
Added getters and setters for num samples per window in app settings
### 3.0.0a8 (2020-09-22)
Improvements to DataWindow, SensorData, and Station classes
### 3.0.0a7 (2020-09-22)
Added DataWindow class; a station agnostic way of getting exactly the time window of data requested
Updated SensorData functions to return numpy arrays
Updated SensorData function names to better match what is being returned
Added functionality to return user-specified location values from api900 location sensors
### 3.0.0a6 (2020-09-18)
Improvements to raw_io. Include new summaries in ReadResult.
Fix bug where querying with just the short ID was not working correctly
Fix bug ensuring read packets are ordered
### 3.0.0a2 (2020-09-17)
First alpha release supporting API M
### 2.9.12 (2020-07-17)
Add support for 16 kHz audio streams
### 2.9.11 (2020-07-07)
Fix bug where reading data that spanned across several days would leave out data towards end of time window
### 2.9.10 (2020-06-25)
Authentication responses not return a copy of the claims for convenience
### 2.9.9 (2020-06-25)
Update dependencies
Encapsulate HTTP logic for HTTP Cloud Client
Add tests for Cloud Client
Add configurable timeout
Add custom errors and better error handling
Fix bug where connection would not be closed on authentication error
### 2.9.8 (2020-06-24)
Large metadata requests are now chunked by the client.
Change refresh token interval from 1 minute to 10 minutes
Re-use HTTP client with keep-alive for more efficient HTTP requests
Add chunked response for timing metadata request
### 2.9.5 (2020-06-23)
Make Cloud API refresh token interval configurable.
Allow Cloud API client to be used within “with” blocks for automatic closing of resources
### 2.9.2 (2020-06-11)
Update dependencies (now dataclasses will only be pulled in on Python 3.6)
### 2.9.1 (2020-06-05)
Fix bug where timing metadata was not converted into its associated data class.
### 2.9.0 (2020-06-05)
Add full fledged cloud based API client. This client seamlessly manages authentication tokens behind the scenes.
Update CLI data request methods to make use of new cloud based client.
### 2.8.7 (2020-06-03)
Integrate ability to access extracted metadata from RedVox packets utilizing the cloud data API.
### 2.8.6 (2020-05-12)
Add small HTTP interface to upcoming RedVox cloud API
Added new sub-command to CLI data_req_report which takes a report ID and will download the report data for authenticated users
### 2.8.5 (2020-05-11)
The auth_token CLI field for the data_req CLI command has been renamed to secret_auth to better reflect the fact that it is a shared secrete.
CLI for data req now makes the shared secret auth key optional dependent on the settings of the remote server.
### 2.8.4 (2020-05-07)
Add –protocol option to redvox-cli when making data request. This allows the data client to optionally connect over HTTP (mainly only useful for local testing)
### 2.8.3 (2020-05-06)
Add mach_time_zero to TimeSyncData class
Add best_tri_msg_indices to TimeSyncData class to identify which tri-message exchange indicated the best latency and offset
Add validation checks to ensure that there is no change in sample rate or mach_time_zero in the analyzed packets
Add check for change in mach_time_zero when identifying gaps
### 2.8.2 (2020-04-27)
Add workaround for accessing mach_time_zero in incorrectly constructed Android packets
### 2.8.1 (2020-04-02)
TriMessageStats will now append empty arrays in a tuple to tri_message_coeffs when the time sync sensor is empty or doesn’t exist.
Functions compute_barometric_height and compute _barometric_height_array take additional optional arguments: surface temperature in K, molar mass of air in kg/mol, acceleration of gravity in m/s2 and the universal gas constant in (kg * m2)/(K * mol * s2)
### 2.8.0 (2020-03-31)
Added a migration module that allows users to slowly begin migrating API 900 data towards API 1000. A flag can be set either through the API or by setting an environment variable ENABLE_MIGRATIONS=”1”. When enabled, all getters for numeric types will return floating point values (the only numeric type in API 1000).
### 2.7.9 (2020-03-25)
Added properties: server_acquisition_times, packet_duration and mach_time_zero to redvox/api900/timesync/api900_timesync.py to assist with analyzing time sync data
### 2.7.8 (2020-03-23)
Added a property: tri_message_coeffs to redvox/api900/timesync/api900_timesync.py to allow access to the tri-message coefficients
Allowed access to the function evaluate_latencies_and_offsets in redvox/api900/timesync/api900_timesync.py
### 2.7.7 (2020-03-19)
Added unit tests for redvox/common/stats_helper.py
The functions mean_of_means, variance_of_means and mean_of_variance in redvox/common/stats_helper.py will not fail when sum(counts) is 0.
### 2.7.6 (2020-03-19)
Added validation function that removes duplicated timestamps to redvox/api900/timesync/tri_message_stats.py
Added get_latency_mean, get_latency_std_dev, get_offset_mean and get_offset_std_dev functions to redvox/api900/timesync/api900_timesync.py
Updated unit tests
### 2.7.5 (2020-03-13)
Expose new package redvox/api900/qa
Expose new gap detection module redvox/api900/qa/gap_detection.py
Provide a public identify_time_gaps method
### 2.7.1 (2020-03-03)
Add and define useful constants to redvox/common/constants.py
Add cross-correlation functions to redvox/common/cross_stats.py
Add date time utilities to redvox/common/date_time_utils.py
Add rdvxz file statistics functions to redvox/common/file_statistics.py
Add statistical helper functions to redvox/common/stats_helper.py
Add new time synchronization package at redvox/api900/timesync
Add a location analyzer to redvox/api900/location_analyzer.py
Updated unit tests and documentation
Updated dependencies to latest versions
Added additional code QA (pylint, mypy, coverage)
### 2.6.1 (2020-01-28)
Update bulk data download client to utilize a RedVox provided authentication token
### 2.6.0 (2019-12-11)
- Update the redvox-cli
The CLI is now installed when this SDK is installed and available to the user as `redvox-cli`
Added improved logging and error handling
Added ability to download bulk RedVox data sets
### 2.5.1 (2019-10-23)
- Add top level setter for mach_time_zero
WrappedRedvoxPacket.set_mach_time_zero(self, mach_time_zero: int) -> ‘WrappedRedvoxPacket’
### 2.5.0 (2019-10-17)
- Add top level getters and setters for accessing time synchronization metrics stored in RedVox Packet metadata.
WrappedRedvoxPacket.best_latency(self) -> typing.Optional[float]
WrappedRedvoxPacket.set_best_latency(self, best_latency: float) -> ‘WrappedRedvoxPacket’
WrappedRedvoxPacket.best_offset(self) -> typing.Optional[float]
WrappedRedvoxPacket.set_best_offset(self, best_offset: float) -> ‘WrappedRedvoxPacket’
WrappedRedvoxPacket.is_synch_corrected(self) -> bool
WrappedRedvoxPacket.set_is_synch_corrected(self, is_synch_corrected: bool) -> ‘WrappedRedvoxPacket’
- Add shortcut for adding metadata to a RedVox Packet (previously the entire metadata needed to be set at a time)
WrappedRedvoxPacket.add_metadata(self, key: str, value: str) -> ‘WrappedRedvoxPacket’
### 2.4.0 (2019-10-8)
- Add mach_time_zero accessor to WrappedRedvoxPackets.
WrappedRedvoxPacket.mach_time_zero(self) -> typing.Optional[int]
### 2.3.0 (2019-9-25)
concat._identify_gaps now only checks for dropped data from sensors. It checks for timing continuity by ensuring that the gap between packets is no larger than a configurable amount for a given sample rate.
concat._identify_sensor_changes was added to identify sensor changes such as change in sample rate, change in sensor name, change in data type, or missing sensor data
### 2.2.1 (2019-5-14)
Added stat utils tests and updated function
Edited documentation typos; 2.2.0 documentation still valid for 2.2.1
### 2.2.0 (2019-4-26)
Add sensor timing correction
Update documentation
Fix more cyclic dependency issues
#### 2.1.1 (2019-4-24)
Start and end timestamps are now optional when reading .rdvxz files from a range. For timestamps that are not supplied, the timestamps are parsed from the data file names to find the earliest and latest timestamp.
Fixed a bug that created a cyclic dependency between the reader and concat modules.
### 2.0.0 (2019-4-12)
Add ability concatenate multiple .rdvxz files together [(docs)](https://bitbucket.org/redvoxhi/redvox-api900-python-reader/src/master/docs/v2.0.0/redvox-api900-docs.md#markdown-header-concatenating-wrappedredvoxpackets)
Add ability to identify gaps in continuous data when concatenating multiple files
Read a range of .rdvxz files from a directory with a given time window and optional redvox ids to filter against [(docs)](https://bitbucket.org/redvoxhi/redvox-api900-python-reader/src/master/docs/v2.0.0/redvox-api900-docs.md#markdown-header-loading-redvox-api-900-files-from-a-range)
Read a range of .rdvxz files from a structured directory with a given time window and optional redvox ids to filter against
Deprecated several public API methods for setting and accessing sensor fields
Add objects for summarizing RedVox data ranges [(docs)](https://bitbucket.org/redvoxhi/redvox-api900-python-reader/src/master/docs/v2.0.0/redvox-api900-docs.md#markdown-header-summarizing-wrappedredvoxpackets)
Add ability to plot summary RedVox data ranges
Refactor reader.py into several modules
Update dependency versions
### 1.5.0 (2019-3-20)
Add setters for all fields
Add the ability to easily create sensor channels and RedVox packets
Add CLI that * Converts .rdvxz files to .json files * Displays the contents of .rdvxz files
Add ability to compare files and sensor channels
Update documentation and API documentation
Add more examples
### 1.4.1 (2019-2-15)
Update required libraries
Add ability to get original compressed buffer from WrappedRedvoxPacket
Add utility functions for LZ4 compression
### v1.4.0 (2018-12-5)
Added support for serializing to/from JSON
Fixed bug where has_time_synchronization_channel() would return true even if the payload was empty
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.