pyspark-config 0.0.2.4

Configurable data pipeline with Pyspark

Pyspark-config

Pyspark-Config is a Python module for pyspark use with the help of a configuration file, granting access to build distributed data piplines with configurable input and output.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Installation

Dependencies

• Python (>= 3.6)
• Pyspark (>= 2.4.5)
• PyYaml (>= 5.3.1)
• Dataclass (>= 0.0.0)

User installation

A step by step series of examples that tell you how to get a development env running

Say what the step will be

Give the example

And repeat

until finished

End with an example of getting some data out of the system or using it for a little demo

Changelog

See the changelog for a history of notable changes to scikit-learn.

Running the tests

Explain how to run the automated tests for this system

Break down into end to end tests

Explain what these tests test and why

Give an example

And coding style tests

Explain what these tests test and why

Give an example

Deployment

Add additional notes about how to deploy this on a live system

Built With

• Dropwizard - The web framework used
• Maven - Dependency Management
• ROME - Used to generate RSS Feeds

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

Authors

• Patrizio Guagliardo - Patrizio1301

See also the list of contributors who participated in this project.

License

This project is distributed under the 3-Clause BSD license. - see the LICENSE.md file for details.