No project description provided
Project description
Datashack
What is this?
Create AWS big data streaming pipeline with Datashack sdk. For example:
Users = StreamingTable("users")
Users['id'] = Column('string')
Users['age'] = Column('int')
Users['name'] = Column('string')
run datashack plan/apply
to see the actual changes or actually applying them to your AWS account.
creates this pipeline:
- Provision Kinesis+Spark+Glue+Iam
- Automate schema evolution
- Tests
Pre-requisites
To work with this project, you will need to have the following software installed on your machine:
-
Node.js: You can download and install Node.js from the official website: https://nodejs.org/en/download/
-
Python: You can download and install Python from the official website: https://www.python.org/downloads/
-
Terraform and CDKTF: These tools are used for defining infrastructure as code. To install them, follow the instructions on the HashiCorp website: https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli. Additionally, you can install the CDKTF CLI by running the following command:
npm install --global cdktf-cli@latest
Make sure to run this command in your terminal or command prompt.
Getting Started
run in your terminal
pip install datashack-sdk
git clone https://github.com/datashack-dev/datashack-sdk-examples
datashack plan ./datashack-sdk-examples/my_app/models
Stay tuned
We are working on a fully funcional beta with many more features. Join here so we can ping you
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for datashack_sdk-0.0.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 061ae4af399a8876ceffba42772352a9f4ac63984c933402f5605842dec0b61d |
|
MD5 | 74df95c2981aecea3c2e229a148bf056 |
|
BLAKE2b-256 | 1499d98132565c36d56fa096a30013d58aa8587f8656dde203c33db0899badda |