Package to interact with the Highlighter Perception System
Project description
Highlighter SDK
The Highlighter SDK Python library provides convenient access to the Highlighter API from applications written in Python. There is also a CLI to access the Highlighter API in your shell.
The library also provides other features. For example:
- Easy configuration for fast setup and use across multiple Highlighter accounts.
- Functions to help get datasets in and out of Highlighter
- Helpers for pagination.
Installation
pip install highlighter-sdk
API Tokens, Environment Variables and Profiles
If you have just a single set of Highlighter credentials you can simply set the appropriate environment variables.
export HL_WEB_GRAPHQL_ENDPOINT="https://<client-account>.highlighter.ai/graphql"
export HL_WEB_GRAPHQL_API_TOKEN="###"
# Only required if you have datasets stored outside Highlighter's managed
# aws s3 storage
export AWS_ACCESS_KEY_ID=###
export AWS_SECRET_ACCESS_KEY=###
export AWS_DEFAULT_REGION=###
If you have several Highlighter credentials we suggest you use the
profiles option. You can create a ~/.highlighter-profiles.yaml
via the
cli.
hl config create --name my-profile --api-token ### --endpoint https://...
Other commands for managing your profiles can be seen with the --help
flag,
If you're a Maverick Renegade you can manage the ~/.highlighter-profiles.yaml
manually. Below is an example,
# Example ~/.highlighter-profiles.yaml
my_profile:
endpoint_url: https://<client-account>.highlighter.ai/graphql
api_token: ###
# Only required if you have datasets stored outside Highlighter's managed
# aws s3 storage
cloud:
- type: aws-s3
aws_access_key_id: ###
aws_secret_access_key: ###
aws_default_region: ###
To use as a profile in the cli simply use,
hl --profile <profile-name> <command>
In a script you can use,
# in a script
from highlighter import HLClient
client = HLClient.from_profile("profile-name")
Additionally HLClient
can be initialized using environment variables or
by passing credentials direcly
from highlighter import HLClient
client = HLClient.from_env()
# or
api_token = "###"
endpoint_url = "https://...highligher.ai/graphql
client = HLCient.from_credential(api_token, endpoint_url)
Finally, if you are in the position where you want to write a specified profile's
credentials to an evnironment file such as .env
or .envrc
you can use
the write
command. This will create or append to the specified file.
hl --profile my-profile write .envrc
Python API
Once you have a HLClient
object you can use it perform queries or mutations. Here is a simple example.
from highlighter import HLClient
from pydantic import BaseModel
client = HLClient.from_env()
# You don't always need to define your own BaseModels
# many common BaseModels are defined in highlighter.base_models
# this is simply for completeness
class ObjectClassType(BaseModel):
id: int
name: str
id = 1234 # add an object class id from you account
result = client.ObjectClass(
return_type=ObjectClassType,
id=id,
)
print(result)
Some queries may return arbitrarily many results. These queries are
paginated and are called Connections
and the queries are named accordingly.
We have provided a paginate
function to help with these Connections
from highlighter import HLClient, paginate
from pydantic import BaseModel
client = HLClient.from_env()
uuids = [
"abc123-abc123-abc123-abc123",
"xyz456-xyz456-xyz456-xyz456",
]
# The following BaseModels are all defined in
# highlighter.base_models. They are simply here
# for completeness
class PageInfo(BaseModel):
hasNextPage: bool
endCursor: Optional[str]
class ObjectClass(BaseModel):
id: str
uuid: str
name: str
class ObjectClassTypeConnection(BaseModel):
pageInfo: PageInfo
nodes: List[ObjectClass]
generator = paginate(
client..objectClassConnection,
ObjectClassTypeConnection,
uuid=uuids,
)
for object_class in generator:
print(object_class)
Datasets
Highlighter SDK provides a dataset representation that can populated
from several sources {HighlighterWeb.Assessments | Local files {.hdf, records.json, coco.json} | S3Bucket}
.
Once populated the Highlighter.Datasets
object contains 2 Pandas.DataFrames
(data_files_df
and annotations_df
) that you can manipulate as required. When you're
ready you can write to disk or upload to Highligher using one of the Writer
classes
to dump your data to disk in a format that can be consumed by your downstream code.
If you need you can also roll-your-own Writer
by implementing the highlighter.datasets.interfaces.IWriter
interface.
Documentation
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for highlighter_sdk-0.5.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4789b323f4fe0f02e1ca69cb5c93990df5688b3c4c053f00f564897311ca89db |
|
MD5 | 4239c26ba2db6d6888a74e79447b5277 |
|
BLAKE2b-256 | 644927bab6809d97c8a4b5edc5327443bd2f4303edc35d6ab1b760f84954ab2a |