spark_plotting_tools
Project description
spark_plottingtools
spark_plotting_tools is a Python library that implements for dummy table
Installation
The code is packaged for PyPI, so that the installation consists in running:
pip install spark-plotting-tools --user
Usage
wrapper take Dummy
from spark_plotting_tools import generated_dummy_table_artifactory
from spark_plotting_tools import generated_dummy_table_datum
import spark_dataframe_tools
Generated Dummy Table Datum
============================================================
path = "fields_pe_datum2.csv"
table_name = "t_kctk_collateralization_atrb"
storage_zone = "master"
sample_parquet = 10
columns_integer_default={}
columns_date_default={"gf_cutoff_date":"2026-01-01"}
columns_string_default={}
columns_decimal_default={"other_concepts_amount":"500.00"}
generated_dummy_table_datum(spark=spark,
path=path,
table_name=table_name,
storage_zone=storage_zone,
sample_parquet=sample_parquet,
partition_colum=["gf_cutoff_date"],
columns_integer_default=columns_integer_default,
columns_date_default=columns_date_default,
columns_string_default=columns_string_default,
columns_decimal_default=columns_decimal_default
)
Generated Dummy Table Artifactory
============================================================
path = "lclsupplierspurchases.output.schema"
sample_parquet = 10
columns_integer_default={}
columns_date_default={"gf_cutoff_date":"2026-01-01"}
columns_string_default={}
columns_decimal_default={"other_concepts_amount":"500.00"}
generated_dummy_table_artifactory(spark=spark,
path=path,
sample_parquet=sample_parquet,
columns_integer_default=columns_integer_default,
columns_date_default=columns_date_default,
columns_string_default=columns_string_default,
columns_decimal_default=columns_decimal_default
)
import os, sys
is_windows = sys.platform.startswith('win')
path_directory = os.path.join("DIRECTORY_DUMMY", table_name)
if is_windows:
path_directory = path_directory.replace("\\", "/")
df = spark.read.parquet(path_directory)
df.show2(10)
License
New features v1.0
BugFix
- choco install visualcpp-build-tools
Reference
- Jonathan Quiza github.
- Jonathan Quiza RumiMLSpark.
- Jonathan Quiza linkedin.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for spark_plotting_tools-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9278d615a6df35d61242574236845e3f2ef700e3a7b38d34641a609f0d862e3e |
|
MD5 | aa5312a1c3b4221d932559195e1cd42d |
|
BLAKE2b-256 | 1749aa312dbf449bb3437470857fa8332f58fb7e2784cf2360c74c2e4b67b8b2 |
Close
Hashes for spark_plotting_tools-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 904791af0051a897b3968c2e6e76c72a41852d1f6533f3a04a7df79ab0c4b1e9 |
|
MD5 | 7e95c90f7505dc04970e8549d4faaaec |
|
BLAKE2b-256 | 8cce08e705c48e7d516009c62c1fba72cc61709331c08f8e6efe386f26e78ee7 |