A package to export bulk data from the CoinGecko API.
Project description
CoinGecko API Wrapper
This project provides a Python wrapper for the CoinGecko API, enabling users to fetch cryptocurrency data, process it into pandas DataFrames, and upload it to AWS S3 as Parquet files. The wrapper abstracts away the complexity of asynchronous API calls and provides an easy-to-use interface for interacting with the CoinGecko API.
Table of Contents
Installation
-
Clone the repository:
```bash git clone https://github.com/yourusername/coingecko-api-wrapper.git cd coingecko-api-wrapper ```
-
Set up a virtual environment:
```bash python3 -m venv venv source venv/bin/activate ```
-
Install the required dependencies:
```bash pip install -r requirements.txt ```
Class Structure
CoinGeckoAPI
The `CoinGeckoAPI` class is responsible for interacting directly with the CoinGecko API. It handles:
- Making asynchronous GET requests to the API.
- Managing rate limits using `AsyncLimiter`.
- Fetching data from paginated endpoints.
Methods:
- `fetch(endpoint: str, params: dict = None, retries: int = 3) -> dict`: Performs a GET request to the specified endpoint.
- `fetch_all_pages(endpoint: str, params: dict, max_pages: int) -> list`: Fetches all pages of data asynchronously.
CoinGeckoDataProcessor
The `CoinGeckoDataProcessor` class processes raw data fetched from the CoinGecko API. It handles:
- Merging timeseries data into a pandas DataFrame.
- Saving DataFrames to Parquet files.
Methods:
- `merge_timeseries_data(data: dict, coingecko_id: str) -> pd.DataFrame`: Cleans and merges timeseries data.
- `save_to_parquet(df: pd.DataFrame, file_path: str)`: Saves a DataFrame to a Parquet file.
CoinGecko
The `CoinGecko` class is the main interface for users. It abstracts away the complexities of asynchronous calls and provides simple, synchronous methods to fetch data, process it, and upload it to S3.
Methods:
- `get_categories() -> pd.DataFrame`: Fetches all categories from the CoinGecko API.
- `get_coins(coins: int, category: str = None) -> pd.DataFrame`: Fetches a list of assets based on market cap.
- `get_timeseries(coingecko_ids: list) -> pd.DataFrame`: Fetches and processes timeseries data.
- `export_data(coins: Union[int, List[str]], export_format: str = 'df') -> pd.DataFrame`: Fetches and exports data as a DataFrame or saves it as a Parquet file.
- `upload_to_s3(df: pd.DataFrame, aws_access_key_id: str, aws_secret_access_key: str, bucket_name: str, folder_name: str = None, file_name: str = None)`: Uploads a DataFrame to AWS S3 as a Parquet file.
Usage
Fetching Categories
```python from coingecko import CoinGecko
cg = CoinGecko(api_key="your-api-key")
Fetch and print the categories DataFrame
categories_df = cg.get_categories() print(categories_df) ```
Fetching Coins
```python from coingecko import CoinGecko
cg = CoinGecko(api_key="your-api-key")
Fetch top 100 coins by market cap
coins_df = cg.get_coins(100) print(coins_df) ```
Fetching Timeseries Data
```python from coingecko import CoinGecko
cg = CoinGecko(api_key="your-api-key")
Fetch timeseries data for a list of coin IDs
timeseries_df = cg.get_timeseries(['bitcoin', 'ethereum']) print(timeseries_df) ```
Exporting Data to Parquet
```python from coingecko import CoinGecko
cg = CoinGecko(api_key="your-api-key")
Export data to a Parquet file
cg.export_data(coins=100, export_format='parquet') ```
Uploading Data to S3
```python from coingecko import CoinGecko
cg = CoinGecko(api_key="your-api-key")
Fetch and upload coins data to S3
coins_df = cg.get_coins(100) cg.upload_to_s3( df=coins_df, aws_access_key_id="your-aws-access-key-id", aws_secret_access_key="your-aws-secret-access-key", bucket_name="your-s3-bucket-name", folder_name="your-folder-name", file_name="your-file-name.parquet" ) ```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file coingecko_exporter-0.3.3.tar.gz
.
File metadata
- Download URL: coingecko_exporter-0.3.3.tar.gz
- Upload date:
- Size: 5.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e58b0356bb682f7dbe53c590338604d5518dcc4e714df4c8e46435f71e62e32a |
|
MD5 | bf5589659f89c2e68d53568e08765e53 |
|
BLAKE2b-256 | 37bbbba055e57b41b6f2f484ccfe906ed6b37d1b7a42841ef17a1f01bd212a99 |
File details
Details for the file coingecko_exporter-0.3.3-py3-none-any.whl
.
File metadata
- Download URL: coingecko_exporter-0.3.3-py3-none-any.whl
- Upload date:
- Size: 6.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 11fe638efc5aa17740785422c033251fa20216dca521d86be70b944dada03092 |
|
MD5 | e1b8c9795983fb53c9212850a00d652d |
|
BLAKE2b-256 | 208e37ee1a8c4d7e4bdb86329837338aaa0e0a170701e066184914f0e0b3581a |