Skip to main content

Parse observations from ictio.org's Citizen Observatory

Project description

IctioPy

Python3 data access library for ictio.org's Citizen Observatory for Amazon basin fish observation.

This python module provides helper routines for citizen scientists that want to load and analyse data from Ictio's citizen observatory for Amazon basin fish observation. The data from this Citizen Observatory is not freely available via public API nor public download, but it can be downloaded as a zip file after registration in web page.

Features

This module takes an Ictio_Basic zip file from ictio.org and performs the following tasks on it:

  1. unzips its contents to a secure temporary folder
  2. flattens decompressed file and folder structure to make file paths more predictable
  3. autodetects the name of the observations file, which has a changing name based on the db generation date
  4. verifies the existence of "taxa" file, containing a taxonomy of species in Amazon basin
  5. loads observations and taxa files into dataframes
  6. combines information from both dataframes into one, so observations are complemented with species data
  7. sanitizes and curates the structure and the values
  8. and returns a curated dataframe with all relevant data ready to be analysed

The structure of the dataframe is as follows:

Observation related data:

obs_id: Unique observation ID.

user_id: Anonymous, numeric ID of the user that made the observation.

protocol_name: Name of the observaton protocol used in the observation. Possible values: During Fishing, After Fishing, Market Survey and Port Survey

Chronology related data:

obs_year: Year of observation.

obs_month: Month of observation.

obs_day: Day of month of observation.

Location related data:

watershed_code: Unique identifier for watershed. For Ictio hotspots and personal locations, the watershed code and watershed name are inferred based on geographic position of Citizen Scientist at the time of observation.

watershed_name: Name of the watersed in which the osbervation was made.

port: This is the name of the port where data was collected and is only reported with the Port Protocol. This is not the location where fish were caught.

location_type: Ictio has three location types: Watersheds, Ictio Hotspots, and Personal Locations. This field will identify watersheds and Ictio Hotspots. This field will be null for personal locations. A personal location is any new location added with the upload tool or based on raw GPS coordinates.

country_name: Country in which the observation was made.

state_province_name: State/Province Code, automatically assigned by latitude and longitude. If there is a checklist assigned to a watershed, observation will be assigned to the State/Province where the centroid of the watershed is. If the watershed overlaps a boundary, it could be assigned to a different State/Province.

Species related data:

taxon_code: Species taxon code.

scientific_name: Scientific name of the species observed.

category: Category of the species observed*.

order1: First order of the species observed*.

family: Family of the species observed*.

*This data does not come from observations table. It is pulled from the separate taxa table that comes embedded in Ictio_basic zip file. The value of taxon_code (present in both tables) is used to perform value lookup.

Fishing related data:

complete_checklist: Indicates if the checklist was completed. A complete checklist is when all taxa caught during the fishing effort are reported. In a market survey it would be all taxa observed at the market. If observation was made via app, it is assumed that user reported a complete checklist.

number_of_fish: Number of individual fish reported for the given taxon.

weight: Total weight in Kg reported for the given taxon.

price_local_currency: Price per Kg in the local currency for the taxon.

fishing_duration: The duration of the fishing effort in hours.

num_of_fishers: Number of fishers participating in fishing effort.

NOTE: At the time of this writing (May 2022), the ictio.org's observation database contains +86.400 observations. Please bear in mind that processing the data can take more than 10 seconds. Your mileage may vary.

Installation

as usual:

pip install ictiopy

Usage

  • First, register and accout at ictio.org (don't forget to read terms of use).
  • Second, download data and store zip file locally in your hard disk.
  • Finally, use this code in order to obtain a ready to use Pandas dataframe with Ictio's observation data:
import ictiopy
pandas_dataframe = ictiopy.load_zipdb('path/to/Ictio_Basic_yyyymmdd.zip') 

About Ictio.org

drawing

Ictio is a database and a mobile phone app created to register observations of caught fish in the Amazon basin. It was developed as part of the Citizen Science for the Amazon project, which aims to connect citizens in the collection and sharing of information on the most important Amazonian fish species. Its wider objective is for that information to contribute in understanding Amazonian fish migration.

Ictio.org is a project of The Cornell Lab of Ornitology. Please, read Ictio's terms of use before using its data.

About MECODA and COS4CLOUD

This data parsing library has been specifically developed for MECODA, a toolkit built around Orange Data Mining for Citizen Observatory data analysis.

MECODA is part of the technological services for Citizen Observatories that are developed in the context of COS4CLOUD project, funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement #863463.

About Science for Change

If you want to tackle a societal problem that requires data to be gathered and analysed, you can count on Science for Change for designing, developing, creating a community around, or leverage data from a Citizen Observatory. Contact Science for Change for an evaluation of your case.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ictiopy-1.0.0.tar.gz (16.6 kB view details)

Uploaded Source

Built Distribution

ictiopy-1.0.0-py3-none-any.whl (14.4 kB view details)

Uploaded Python 3

File details

Details for the file ictiopy-1.0.0.tar.gz.

File metadata

  • Download URL: ictiopy-1.0.0.tar.gz
  • Upload date:
  • Size: 16.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.8.10

File hashes

Hashes for ictiopy-1.0.0.tar.gz
Algorithm Hash digest
SHA256 7c2e00418b987b5181734dd0d7af6c6d0d69e8cc6041f44f11391a47849f772d
MD5 3af84dbf527e86273d3347b2275abd94
BLAKE2b-256 e410f45ab5e079ff245c20a26d93f3b4f9b6f328b259d749584f147911436f18

See more details on using hashes here.

File details

Details for the file ictiopy-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: ictiopy-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 14.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.8.10

File hashes

Hashes for ictiopy-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f4498cc06d4cba9a710b4520bf880b8cd4dae2c690aa03c899ce712908b130a0
MD5 eb0a68cca9b8f1b6a47f9040c699352c
BLAKE2b-256 c36bd7f3de3c3768ffa77382b97099b425630838f234d063905489f91df46b8c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page