Neptune.ai integration with Kedro
Project description
Neptune + Kedro integration
Kedro plugin for experiment tracking and metadata management. It lets you browse, filter and sort runs in a nice UI, visualize node/pipeline metadata, and compare pipelines.
What will you get with this integration?
- browse, filter, and sort your model training runs
- compare nodes and pipelines on metrics, visual node outputs, and more
- display all pipeline metadata including learning curves for metrics, plots, and images, rich media like video and audio or interactive visualizations from Plotly, Altair, or Bokeh
- and do whatever else you would expect from a modern ML metadata store
Kedro pipeline metadata in custom dashboard in the Neptune web app
Note: The Kedro-Neptune plugin supports distributed pipeline execution and works in Kedro setups that use orchestrators like Airflow or Kubeflow.
Resources
- Documentation
- Code example on GitHub
- Example run logged in the Neptune app
- How to Compare Kedro pipelines
- How to Compare results between Kedro nodes
- How to Display Kedro node metadata and outputs
Example
On the command line:
pip install kedro neptune[kedro]
kedro new --starter=pandas-iris
In your Kedro project directory:
kedro neptune init
In a pipeline node, in nodes.py
:
import neptune
# Add a Neptune run handler to the report_accuracy() function
def report_accuracy(
y_pred: pd.Series,
y_test: pd.Series,
neptune_run: neptune.handler.Handler,
) -> None:
accuracy = (y_pred == y_test).sum() / len(y_test)
logger = logging.getLogger(__name__)
logger.info("Model has accuracy of %.3f on test data.", accuracy)
# Log metrics to the Neptune run
neptune_run["nodes/report/accuracy"] = accuracy * 100
# Add the Neptune run handler to the Kedro pipeline
node(
func=report_accuracy,
inputs=["y_pred", "y_test", "neptune_run"],
outputs=None,
name="report_accuracy",
)
On the command line, run the Kedro pipeline:
kedro run
Support
If you got stuck or simply want to talk to us, here are your options:
- Check our FAQ page
- You can submit bug reports, feature requests, or contributions directly to the repository.
- Chat! When in the Neptune application click on the blue message icon in the bottom-right corner and send a message. A real person will talk to you ASAP (typically very ASAP),
- You can just shoot us an email at support@neptune.ai
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kedro_neptune-0.3.0.tar.gz
(14.4 kB
view hashes)
Built Distribution
Close
Hashes for kedro_neptune-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd1c292edee23460b899c3174376bba5fdf8f328cdbb366c78b624dc44f7ccaa |
|
MD5 | a605607bb417b7e71ed34c627292ef07 |
|
BLAKE2b-256 | 1854024c9892e3247df3b8d5fc2a2e7b0e2c61bf03cbbd9d16b8200c2e8da5f5 |