Skip to main content

ETL para extraer, transformar y cargar datos del clima en S3

Project description

Facu Weather Flow

Facu Weather Flow es una librería de Python modular que implementa un pipeline ETL para procesar datos del clima de cualquier ciudad que necesites.

Facu Weather Flow is a modular Python library that implements an ETL pipeline to process weather data for any city you need.


🚀 Características / Features

  • Extracción de datos desde la API de OpenWeatherMap
    Data extraction from the OpenWeatherMap API
  • Transformación a DTOs estructurados
    Transformation into structured DTOs
  • Almacenamiento como archivos Parquet
    Storage as Parquet files
  • Carga directa en un bucket S3 en AWS
    Direct upload to an AWS S3 bucket
  • Configuración flexible mediante variables de entorno
    Flexible configuration through environment variables
  • Uso como librería o desde la línea de comandos (weather-flow)
    Usable as a library or from the command line (weather-flow)

📦 Instalación / Installation

pip install facu-weather-flow

🧪 Uso desde CLI / CLI Usage

weather-flow

O como módulo / Or as a module:

python -m facu_weather_flow

⚙️ Variables de entorno / Environment Variables

La librería requiere las siguientes variables, definidas en un archivo .env o como variables del entorno:
The library requires the following variables, defined in a .env file or as environment variables:

OPENWEATHERMAP_API_KEY=tu_api_key_openweathermap / your_openweathermap_api_key
CITY_DATA=nombre_ciudad_o_archivo / city_name_or_file
AWS_ACCESS_KEY_ID=tu_access_key / your_access_key
AWS_SECRET_ACCESS_KEY=tu_secret_key / your_secret_key
AWS_DEFAULT_REGION=us-east-1
AWS_WEATHER_BUCKET_NAME=nombre_del_bucket_s3 / your_s3_bucket_name
WEATHER_OUTPUT_DIR=/ruta/de/salida/local / local_output_path

📚 Ejemplo de uso como módulo / Example module usage

from facu_weather_flow.main import run_pipeline

run_pipeline()

🧊 Licencia / License

MIT © Facu Vega

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

facu_weather_flow-0.1.2.tar.gz (3.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

facu_weather_flow-0.1.2-py3-none-any.whl (3.4 kB view details)

Uploaded Python 3

File details

Details for the file facu_weather_flow-0.1.2.tar.gz.

File metadata

  • Download URL: facu_weather_flow-0.1.2.tar.gz
  • Upload date:
  • Size: 3.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.3

File hashes

Hashes for facu_weather_flow-0.1.2.tar.gz
Algorithm Hash digest
SHA256 9f22b68679122a25c9b2dd23a10e523ab77d6c7e55abe206056469d8ca97a342
MD5 b623d06d08b27b9bd4f06b0e62543313
BLAKE2b-256 44f7feb6507f7a33852668d6c68b5c36e6f399c344176c6962aa1976319fbd4e

See more details on using hashes here.

File details

Details for the file facu_weather_flow-0.1.2-py3-none-any.whl.

File metadata

File hashes

Hashes for facu_weather_flow-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 96a73865ec5661f9b1cf4b100e9c6e05c3f46a0b7cdf9cf351d03d9f013a3ef4
MD5 9162cfd2c659ade8451f614bd3d83658
BLAKE2b-256 b2fcd4f804ea3521a527f62edd5a72dc1f354b3425329c9aea79add441dc0361

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page