Skip to main content

An abstraction of data source for extraction applications usage

Project description

sour-cereal

About

An abstraction of connections to real data sources for extraction applications usage.

It implements simple methods for extraction preparation, monitoring, execution and cleaning into a class, meant to be inherited.

The goal of this project consists in creating a standardized API for communicating with data sources in applications written in Python.

Installing

You can simply install it using pip as follows:

$ pip install sour-cereal

Usage

from sour_cereal import SourceConnection

class FooDataSource(SourceConnection):
    def get_status_of_extraction(self: 'FooDataSource', *args, **kwargs):
        return datetime.now()

    def check_availability_of_extraction(self: 'FooDataSource', status: datetime):
        return status.hour >= 7		# data is ready only after 7pm

    def execute_extraction(self: 'FooDataSource', *args, **kwargs):
		# Extract some data
        return ['file1', 'file2']

For more examples, please check the files inside the sour_cereal/examples folder.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sour-cereal-1.0.1.tar.gz (4.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page