Skip to main content

Analysis of sensors and time series data

Project description

Smart Citizen Data

DOI Binder PyPI version Python application

Welcome to SmartCitizen Data. This is a data analysis framework for working with sensor data in different ways:

  • Interacting with several sensors APIs
  • Clean data, export and calculate metrics
  • Model sensor data and calibrate sensors
  • Generate data visualisations - matplotlib, plotly or uplot
  • Generate analysis reports in html or pdf and upload them to Zenodo

Installation

You can check it out in the Binder before installing if you want. Works with Python 3.* (tested between 3.9 and 3.11).

You can just run:

pip install scdata

Work on the source code

Simply clone the repository with:

git clone https://github.com/fablabbcn/smartcitizen-data.git
cd smartcitizen-data

Install scdata package with requirements:

python setup.py install

Or if you want to edit:

cd scdata
pip install --editable .

Tokens and config

A configuration file is available at ~/.config/scdata/config.yaml, which contains a set of configurable variables to allow or not the local storage of relevant data in the data folder, normally in ~/.cache/scdata:

data:
  cached_data_margin: 2
  load_cached_api: true
  reload_metadata: true
  store_cached_api: true
paths:
  config: /Users/username/.config/scdata
  data: /Users/username/.cache/scdata
  export: /Users/username/.cache/scdata/export
  interim: /Users/username/.cache/scdata/interim
  inventory: ''
  models: /Users/username/.cache/scdata/models
  processed: /Users/username/.cache/scdata/processed
  raw: /Users/username/.cache/scdata/raw
  reports: /Users/username/.cache/scdata/reports
  uploads: /Users/username/.cache/scdata/uploads
zenodo_real_base_url: https://zenodo.org
zenodo_sandbox_base_url: http://sandbox.zenodo.org

Also, .env files will be picked from ~/.cache/scdata. An env.example is provided. If you want to upload data to Zenodo, you will need to fill set an environment variable called ZENODO_TOKEN in your environment. The .env file is not mandatory for scdata to work. It only serves for authentication if you want to POST or GET data without throttling. You can still do GET requests but they will be throttled if not authenticated/authorized by the platform. See user roles for more info.

Using with jupyter lab (optional)

It can also be used with jupyter lab or jupyter. For this install juypterlab.

Contribute

Issues and PR more than welcome!

Funding

This work has received funding from the European Union's Horizon 2020 research and innovation program under the grant agreement No. 689954

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scdata-1.2.6.tar.gz (309.4 kB view details)

Uploaded Source

Built Distribution

scdata-1.2.6-py3-none-any.whl (333.1 kB view details)

Uploaded Python 3

File details

Details for the file scdata-1.2.6.tar.gz.

File metadata

  • Download URL: scdata-1.2.6.tar.gz
  • Upload date:
  • Size: 309.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.9.21

File hashes

Hashes for scdata-1.2.6.tar.gz
Algorithm Hash digest
SHA256 2c3a01e0bc989e9f7f9d48b3491856215330416b7a605688c64b4c610a37c6fb
MD5 2f8890bdb9b19a83dfce00c97524a044
BLAKE2b-256 05bc138cd2f597252ec030dbf081ee133cac8b3499b764baeda3dc068f477249

See more details on using hashes here.

File details

Details for the file scdata-1.2.6-py3-none-any.whl.

File metadata

  • Download URL: scdata-1.2.6-py3-none-any.whl
  • Upload date:
  • Size: 333.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.9.21

File hashes

Hashes for scdata-1.2.6-py3-none-any.whl
Algorithm Hash digest
SHA256 badd0b915f3d1cb49f4947725bed1f7be6d1300282c1253d82e07bd8da2273fc
MD5 1fb518543a9cae3782fed64f637554ba
BLAKE2b-256 8ea13a8c5704e0ede73d5837683fddf322765c6ad8b5bac96aa144eca716de8d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page