Useful towel-related stuff.
Project description
pySpark-connector-kairosdb provides you an easy way to get data from KairosDB and make it available on Spark as a DataFrame. It’s simple as that:
#!/usr/bin/env python # sconnk means "py(S)park-(CONN)ector-(K)airosdb" from sconnk import Connection, Dataframe query_data = { "start_relative": { "value": "5", "unit": "years" }, "metrics": [{ "name": "test_", "limit": 5 }, { "name": "DP_058424", "limit": 10 }, { "name": "teste_gzip", "limit": 5 }, { "name": "DP_063321", "limit": 10 }] } # Creating a connection with KairosDB database (in the given address). conn = Connection("http://localhost:8080/") # Performing our query on KairosDB. json_data = conn.query(query_data) # Creating a new Dataframe object passing the JSON returned by KairosDB API. df = Dataframe(json_data).df # Print the dataframe. df.show()
Remember this’s a ALPHA module without good documentation, examples and even well implemented features. We’ve a long highway to cross.
Future
This module is in development and we’ve the following plannings for the future of this module:
Write good documentation
Write tests - a lot of them.
Add support to RDD
Don’t write to JSON file in order to parse it in Spark.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for pySpark-connector-kairosdb-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 062a97f39546265aa4c9160b37b785e53b91d6deff7bd21adcd0bb7c0b05b9d5 |
|
MD5 | ad38d0e8f3fbe1be6e05bddcfea5b09f |
|
BLAKE2b-256 | d5d81c0b46f8bf206e31a225788e6ac1601f565d3278f5623da052267048a522 |