Skip to main content

Convert some data into Panda DataFrames

Project description

Convert some data into Panda DataFrames

British Petroleum (BP)

It parse sheet like Primary Energy Consumption (not like Primary Energy - Cons by fuel).

Open: http://www.bp.com/statisticalreview or https://www.bp.com/en/global/corporate/energy-economics/statistical-review-of-world-energy.html

Download Statistical Review of World Energy – all data.

Use:

from shifter_pandas.bp import UNITS_ENERGY, BPDatasource

shifter_ds = BPDatasource("bp-stats-review-2021-all-data.xlsx")

df = shifter_ds.datasource(units_filter=UNITS_ENERGY, regions_filter=["Switzerland"])
df

Swiss Office Federal of Statistics (OFS)

From https://www.bfs.admin.ch/bfs/fr/home/services/recherche/stat-tab-donnees-interactives.html create a stat table.

Click on À propos du tableau

Click on Rendez ce tableau disponible dans votre application

Use:

from shifter_pandas.ofs import OFSDatasource

shifter_ds = OFSDatasource("<URL>")

df = shifter_ds.datasource(<Requête Json>)
df

And replace <URL> and <Requête Json> with the content of the fields of the OFS web page.

Interesting sources

Our World in Data

Select a publication.

Click Download.

Click Full data (CSV).

Use:

import pandas as pd
from shifter_pandas.wikidata_ import WikidataDatasource

df_owid = pd.read_csv("<file name>")
wdds = WikidataDatasource()
df_wd = wdds.datasource_code(wikidata_id=True, wikidata_name=True, wikidata_type=True)
df = pd.merge(df_owid, df_wd, how="inner", left_on='iso_code', right_on='Code')
df

Interesting sources

World Bank

Open https://data.worldbank.org/

Find a chart

In Download click CSV

Use:

from shifter_pandas.worldbank import wbDatasource

df = wbDatasource("<file name>")
df

Interesting sources

Wikidata

By providing the wikidata_* parameters, you can ass some data from WikiData.

Careful, the WikiData is relatively slow then the first time you run it il will be slow. We use a cache to make it fast the next times.

You can also get the country list with population and ISO 2 code with:

from shifter_pandas.wikidata_ import (
    ELEMENT_COUNTRY,
    PROPERTY_ISO_3166_1_ALPHA_2,
    PROPERTY_POPULATION,
    WikidataDatasource,
)

shifter_ds = WikidataDatasource()
df = shifter_ds.datasource(
    instance_of=ELEMENT_COUNTRY,
    with_id=True,
    with_name=True,
    properties=[PROPERTY_ISO_3166_1_ALPHA_2, PROPERTY_POPULATION],
    limit=1000,
)
df

Contributing

Install the pre-commit hooks:

pip install pre-commit
pre-commit install --allow-missing-config

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shifter_pandas-0.4.0.tar.gz (14.4 kB view hashes)

Uploaded Source

Built Distribution

shifter_pandas-0.4.0-py3-none-any.whl (15.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page