Emulator for Custom Handlers for Inference Endpoints
Project description
Emulator for Custom Handlers for Inference Endpoints
🤗 Inference Endpoints offers a secure production solution to easily deploy any 🤗 Transformers and Sentence-Transformers models from the Hub on dedicated and autoscaling infrastructure managed by Hugging Face.
🤗 Inference Endpoints support all of the 🤗 Transformers and Sentence-Transformers tasks as well as custom tasks not supported by 🤗 Transformers yet like speaker diarization and diffusion.
The hf-endpoints-emulator
package provides a simple way to test your custom handlers locally before deploying them to Inference Endpoints. It is also useful for debugging your custom handlers.
The package provides a hf-endpoints-emulator
command line tool that can be used to run your custom handlers locally. It also provides a hf_endpoint_emulator
Python package that can be used to run your custom handlers locally from Python.
Installation
pip install hf-endpoints-emulator
Usage
You can check the examples/
directory for examples on how to use the hf-endpoints-emulator
package.
Command Line
hf-endpoints-emulator --handler <handler>
This will start a web server that will run your custom handler. The web server will be accessible at http://localhost:5000
. You can then send requests to the web server to test your custom handler.
curl
curl --request POST \
--url http://localhost/:5000 \
--header 'Content-Type: application/json' \
--data '{
"inputs": "I like you."
}'
python
import requests
url = "http://localhost:5000/"
payload = {"inputs": "test"}
headers = {"Content-Type": "application/json"}
response = requests.request("POST", url, json=payload, headers=headers)
print(response.json())
Python pacakge
from hf_endpoints_emulator.emulator import emulate
emulate(handler_path="examples/my_handler.py", port=5000)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hf-endpoints-emulator-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 75dc5545b340fc1d15b27deddfb584e2c8c4779876ceb9864e90ed95bc267743 |
|
MD5 | e5b038f7c3519e69655a0cbbb4875abc |
|
BLAKE2b-256 | 73b5b58ea49d1a58519b60718abc15b7482d35ccc1a97628303b854907f7e70d |
Hashes for hf_endpoints_emulator-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 09a5f5bbed79a09288f6de8498906eab8a40bd15f46fd6a17d7d07565a2d339f |
|
MD5 | f3f05d9c9f825fbe441fcf18fe5e7216 |
|
BLAKE2b-256 | 3d7d26e40f0b7dabc45bba04f88756ae80ffa60812768e9e76abad5c8f77898b |