A python helper library to report metrics and communicate with Polyaxon.
Project description
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Build Status](https://travis-ci.org/polyaxon/polyaxon-helper.svg?branch=master)](https://travis-ci.org/polyaxon/polyaxon-helper)
[![PyPI version](https://badge.fury.io/py/polyaxon-helper.svg)](https://badge.fury.io/py/polyaxon-helper)
[![Slack](https://img.shields.io/badge/chat-on%20slack-aadada.svg?logo=slack&longCache=true)](https://join.slack.com/t/polyaxon/shared_invite/enQtMzQ0ODc2MDg1ODc0LWY2ZTdkMTNmZjBlZmRmNjQxYmYwMTBiMDZiMWJhODI2ZTk0MDU4Mjg5YzA5M2NhYzc5ZjhiMjczMDllYmQ2MDg)
# Polyaxon-helper
Polyaxon helper is a lightweight python library to report metrics and communicate information with Polyaxon.
## Installation
```bash
$ pip install -U polyaxon-helper
```
for python3
```bash
$ pip3 install -U polyaxon-helper
```
## Install polyaxon
Please check [polyaxon installation guide](https://docs.polyaxon.com/installation/introduction)
## Install in polyaxonfile
If you want to delegate the installation to polyaxon during the build process,
add a new step to the `run` section in your polyaxonfile:
```yaml
...
build:
image: ...
build_steps:
- ...
- pip install -U polyaxon-helper
- ...
run:
cmd: ...
```
## Usage
### Getting env variables defined by Polyaxon
```python
from polyaxon_helper import (
get_cluster_def,
get_declarations,
get_experiment_info,
get_task_info,
get_tf_config,
get_job_info,
get_outputs_path,
get_outputs_refs_paths,
get_data_paths,
get_log_level
)
cluster_def = get_cluster_def()
declarations = get_declarations()
experiment_info = get_experiment_info()
task_info = get_task_info()
job_info = get_job_info()
outputs_path = get_outputs_path()
outputs_refs_paths = get_outputs_refs_paths()
data_paths = get_data_paths()
tf_config = get_tf_config()
log_level = get_log_level()
```
* `get_cluster_def`: Returns cluster definition created by polyaxon.
```json
{
"master": ["plxjob-master0-8eefb7a1146f476ca66e3bee9b88c1de:2000"],
"worker": ["plxjob-worker1-8eefb7a1146f476ca66e3bee9b88c1de:2000",
"plxjob-worker2-8eefb7a1146f476ca66e3bee9b88c1de:2000"],
"ps": ["plxjob-ps3-8eefb7a1146f476ca66e3bee9b88c1de:2000"],
}
```
* `get_declarations`: Returns all the experiment declarations based on both,
* declarations section
* matrix section
* `get_tf_config`: Returns the TF_CONFIG defining the cluster and the current task.
if `envvar` is not null, it will set and env variable with `envvar`.
* `get_experiment_info`: Returns information about the experiment.
* project_name
* experiment_group_name
* experiment_name
* project_uuid
* experiment_group_uuid
* experiment_uuid
* `get_job_info`: Returns information about the job.
* project_name
* job_name
* project_uuid
* job_uuid
* `get_task_info`: Returns the task info: `{"type": str, "index": int}`.
* `outputs_path`: The outputs path generated by polyaxon based on the hierarchy of the experiment.
`user/project/group/experiment/files`
* `data_paths`: The data paths mounted for the job/experiment:
{
'data1': '/data/1',
'data-foo': '/data/foo',
}
* `outputs_path`: The references outputs paths requested by the user,
the order follows the order specified by the user:
{
'experiments': [
`user/project/job12/files`,
], 'experiments': [
`user/project/group/experiment1/files`,
`user/project/experiment100/files`
]
}
* `get_log_level`: If set on the polyaxonfile it will return the log level.
### Reporting metrics to Polyaxon
In order to report metrics for an experiment, just add these lines in you program.
```python
from polyaxon_helper import send_metrics
send_metrics(accuracy=0.9, precision=0.95)
```
## Quick start
Please check our [quick start guide](https://docs.polyaxon.com/quick_start) to start training your first experiment.
## License
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fpolyaxon%2Fpolyaxon-helper.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fpolyaxon%2Fpolyaxon-helper?ref=badge_large)
[![Build Status](https://travis-ci.org/polyaxon/polyaxon-helper.svg?branch=master)](https://travis-ci.org/polyaxon/polyaxon-helper)
[![PyPI version](https://badge.fury.io/py/polyaxon-helper.svg)](https://badge.fury.io/py/polyaxon-helper)
[![Slack](https://img.shields.io/badge/chat-on%20slack-aadada.svg?logo=slack&longCache=true)](https://join.slack.com/t/polyaxon/shared_invite/enQtMzQ0ODc2MDg1ODc0LWY2ZTdkMTNmZjBlZmRmNjQxYmYwMTBiMDZiMWJhODI2ZTk0MDU4Mjg5YzA5M2NhYzc5ZjhiMjczMDllYmQ2MDg)
# Polyaxon-helper
Polyaxon helper is a lightweight python library to report metrics and communicate information with Polyaxon.
## Installation
```bash
$ pip install -U polyaxon-helper
```
for python3
```bash
$ pip3 install -U polyaxon-helper
```
## Install polyaxon
Please check [polyaxon installation guide](https://docs.polyaxon.com/installation/introduction)
## Install in polyaxonfile
If you want to delegate the installation to polyaxon during the build process,
add a new step to the `run` section in your polyaxonfile:
```yaml
...
build:
image: ...
build_steps:
- ...
- pip install -U polyaxon-helper
- ...
run:
cmd: ...
```
## Usage
### Getting env variables defined by Polyaxon
```python
from polyaxon_helper import (
get_cluster_def,
get_declarations,
get_experiment_info,
get_task_info,
get_tf_config,
get_job_info,
get_outputs_path,
get_outputs_refs_paths,
get_data_paths,
get_log_level
)
cluster_def = get_cluster_def()
declarations = get_declarations()
experiment_info = get_experiment_info()
task_info = get_task_info()
job_info = get_job_info()
outputs_path = get_outputs_path()
outputs_refs_paths = get_outputs_refs_paths()
data_paths = get_data_paths()
tf_config = get_tf_config()
log_level = get_log_level()
```
* `get_cluster_def`: Returns cluster definition created by polyaxon.
```json
{
"master": ["plxjob-master0-8eefb7a1146f476ca66e3bee9b88c1de:2000"],
"worker": ["plxjob-worker1-8eefb7a1146f476ca66e3bee9b88c1de:2000",
"plxjob-worker2-8eefb7a1146f476ca66e3bee9b88c1de:2000"],
"ps": ["plxjob-ps3-8eefb7a1146f476ca66e3bee9b88c1de:2000"],
}
```
* `get_declarations`: Returns all the experiment declarations based on both,
* declarations section
* matrix section
* `get_tf_config`: Returns the TF_CONFIG defining the cluster and the current task.
if `envvar` is not null, it will set and env variable with `envvar`.
* `get_experiment_info`: Returns information about the experiment.
* project_name
* experiment_group_name
* experiment_name
* project_uuid
* experiment_group_uuid
* experiment_uuid
* `get_job_info`: Returns information about the job.
* project_name
* job_name
* project_uuid
* job_uuid
* `get_task_info`: Returns the task info: `{"type": str, "index": int}`.
* `outputs_path`: The outputs path generated by polyaxon based on the hierarchy of the experiment.
`user/project/group/experiment/files`
* `data_paths`: The data paths mounted for the job/experiment:
{
'data1': '/data/1',
'data-foo': '/data/foo',
}
* `outputs_path`: The references outputs paths requested by the user,
the order follows the order specified by the user:
{
'experiments': [
`user/project/job12/files`,
], 'experiments': [
`user/project/group/experiment1/files`,
`user/project/experiment100/files`
]
}
* `get_log_level`: If set on the polyaxonfile it will return the log level.
### Reporting metrics to Polyaxon
In order to report metrics for an experiment, just add these lines in you program.
```python
from polyaxon_helper import send_metrics
send_metrics(accuracy=0.9, precision=0.95)
```
## Quick start
Please check our [quick start guide](https://docs.polyaxon.com/quick_start) to start training your first experiment.
## License
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fpolyaxon%2Fpolyaxon-helper.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fpolyaxon%2Fpolyaxon-helper?ref=badge_large)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
polyaxon-helper-0.4.3.tar.gz
(7.1 kB
view hashes)
Built Distribution
Close
Hashes for polyaxon_helper-0.4.3-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cafcd855d015752e6d08ae7adb7bddfc29c0d9f9cfbecd8df6109dd8fccc3121 |
|
MD5 | a966b63742065754088da232af884fb1 |
|
BLAKE2b-256 | 518bfbb2c25b50bccd2702c8cac0fddcf9d83b76cb04a8ac6218353ab4d03868 |