Useful towel-related stuff.
Project description
pySpark-connector-kairosdb provides you an easy way to get data from KairosDB and make it available on Spark as a DataFrame. It’s simple as that:
#!/usr/bin/env python # sconnk means "py(S)park-(CONN)ector-(K)airosdb" from sconnk import Connection, Dataframe query_data = { "start_relative": { "value": "5", "unit": "years" }, "metrics": [{ "name": "test_", "limit": 5 }, { "name": "DP_058424", "limit": 10 }, { "name": "teste_gzip", "limit": 5 }, { "name": "DP_063321", "limit": 10 }] } # Creating a connection with KairosDB database (in the given address). conn = Connection("http://localhost:8080/") # Performing our query on KairosDB. json_data = conn.query(query_data) # Creating a new Dataframe object passing the JSON returned by KairosDB API. df = Dataframe(json_data).df # Print the dataframe. df.show()
Remember this’s a ALPHA module without good documentation, examples and even well implemented features. We’ve a long highway to cross.
Future
This module is in development and we’ve the following plannings for the future of this module:
Write good documentation
Write tests - a lot of them.
Add support to RDD
Don’t write to JSON file in order to parse it in Spark.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file pySpark-connector-kairosdb-0.1.0.tar.gz
.
File metadata
- Download URL: pySpark-connector-kairosdb-0.1.0.tar.gz
- Upload date:
- Size: 2.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
062a97f39546265aa4c9160b37b785e53b91d6deff7bd21adcd0bb7c0b05b9d5
|
|
MD5 |
ad38d0e8f3fbe1be6e05bddcfea5b09f
|
|
BLAKE2b-256 |
d5d81c0b46f8bf206e31a225788e6ac1601f565d3278f5623da052267048a522
|