Motion Lake Client, a client for the Motion Lake API (a Mobility Data Lake)
Project description
MotionLake Client
MotionLake Client is a Python client library for interacting with a storage server designed for a new mobility data lake solution. It provides functionalities to create collections, store data, query data, and retrieve collections.
Installation
You can install the library via pip:
pip install motion-lake-client
Usage
Here's a brief overview of how to use the library:
from motion_lake_client import BaseClient
# Initialize the client with the base URL of the storage server
client = BaseClient(lake_url="http://localhost:8000")
# Create a new collection
client.create_collection("my_collection")
# Store data in a collection
data = b"example_data"
timestamp = int(datetime.now().timestamp())
client.store("my_collection", data, timestamp)
# Query data from a collection
results = client.query(
"my_collection", min_timestamp=0, max_timestamp=timestamp, ascending=True
)
# Retrieve last item from a collection
last_item = client.get_last_item("my_collection")
# Retrieve first item from a collection
first_item = client.get_first_item("my_collection")
# Get items between two timestamps
items_between = client.get_items_between(
"my_collection", min_timestamp=0, max_timestamp=timestamp
)
# Get items before a timestamp
items_before = client.get_items_before("my_collection", timestamp, limit=10)
# Get items after a timestamp
items_after = client.get_items_after("my_collection", timestamp, limit=10)
# Get all collections
collections = client.get_collections()
Documentation
The library provides a series of classes and methods for storing, querying, and managing collections of data items. Each item is timestamped and can be stored in various formats. Below is a detailed usage guide for each component provided by the API.
Prerequisites
Before using the API, make sure you have the requests
library installed:
pip install requests
Initializing the Client
To start interacting with the data storage server, instantiate the BaseClient
with the URL of the storage server:
from datetime import datetime
from my_module import BaseClient, ContentType
# Initialize the client; replace 'http://localhost:8000' with your server's URL
client = BaseClient('http://localhost:8000')
Creating a Data Collection
Create a new data collection by specifying its name:
response = client.create_collection("weather_data")
print(response)
Storing Data
Store data in a specified collection:
data = b"Example data"
timestamp = datetime.now()
# Store data as raw bytes
response = client.store("weather_data", data, timestamp, ContentType.RAW)
print(response)
You can also specify whether to create the collection if it doesn't exist:
response = client.store("weather_data", data, timestamp, ContentType.JSON, create_collection=True)
print(response)
Querying Data
Retrieve items from a collection based on various criteria:
-
Query by Timestamp Range:
from datetime import datetime, timedelta start_date = datetime.now() - timedelta(days=1) end_date = datetime.now() items = client.get_items_between("weather_data", start_date, end_date) for item in items: print("Timestamp:", item.timestamp, "Data:", item.data)
-
Get Last N Items:
last_items = client.get_last_items("weather_data", 5) for item in last_items: print("Timestamp:", item.timestamp, "Data:", item.data)
-
Get First N Items:
first_items = client.get_first_items("weather_data", 5) for item in first_items: print("Timestamp:", item.timestamp, "Data:", item.data)
-
Get Items but skip data (only load timestamps):
first_items = client.get_first_items("weather_data", 5, skip_data=True) for item in first_items: print("Timestamp:", item.timestamp) assert item.data is None, "Data should be None, otherwise developer made a mistake (aka me)"
Advanced Queries
Perform an advanced SQL-like query (make sure your query string contains the placeholder [table]
):
query = "SELECT * FROM [table] WHERE data LIKE '%sample%'"
min_timestamp = datetime(2023, 1, 1)
max_timestamp = datetime(2023, 1, 31)
response = client.advanced_query("weather_data", query, min_timestamp, max_timestamp)
print(response)
Managing Collections
-
List All Collections:
collections = client.get_collections() for collection in collections: print(f"Collection: {collection.name}, Items: {collection.count}")
-
Delete a Collection:
response = client.delete_collection("weather_data") print(response)
Contributing
Contributions are welcome! Please feel free to submit a pull request or open an issue for any bugs or feature requests.
License
All rights reserved.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for motion_lake_client-0.0.15.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c11601934bed203cc4ace2600c22fabd3942b60bc05473f34e47f16711c450b |
|
MD5 | 40de80e2704789ad01ad01282ed97837 |
|
BLAKE2b-256 | eec36cd674596cdcd65b4ab394e1d4e807dca33807c1605f9747ce7c85506014 |
Hashes for motion_lake_client-0.0.15-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c4a4e40849216709e0fd14bacddb05d3dae1bbe8c5b62238ad8dfaaabb60f4b9 |
|
MD5 | 26456f56c3d56faa104b20caa52d212f |
|
BLAKE2b-256 | e044744ca6f3850e2cd7df42966a272e1776271121768cc36c65ca2481a04b36 |