PySpark Project Buiding Tool
Project description
PySpark CLI
This will implement a PySpark Project boiler plate code based on user input.
Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.
PySpark is the Python API for Spark.
Installation Steps:
git clone https://github.com/qburst/PySparkCLI.git
cd PySparkCLI
pip3 install -e . --user
Create a PySpark Project
pysparkcli create [PROJECT_NAME] --master [MASTER_URL] --cores [NUMBER]
master - The URL of the cluster it connects to. You can also use -m instead of --master.
cores - You can also use -c instead of --cores.
Run a PySpark Project
pysparkcli run [PROJECT_NAME]
Contribution Guidelines
Check out here for our contribution guidelines.
Sponsors
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pysparkcli-0.0.1-py2.7.egg.
File metadata
- Download URL: pysparkcli-0.0.1-py2.7.egg
- Upload date:
- Size: 11.9 kB
- Tags: Egg
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.40.0 CPython/3.7.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f2f48d0de6965de8f1350a7a90d09d0df06caa1f472fc5a033388c85325185f8
|
|
| MD5 |
819cef186b8b14544d64c133c6ec9bec
|
|
| BLAKE2b-256 |
8743a68cb3cf9f1aaa2d70992ca419de5e4c0beddda6db881c16a12e4ba5e66b
|
