No project description provided
Project description
FlagEval Serving
Serving Framework of AI Models for Evaluating on FlagEval Platform.
Installation
pip install --upgrade flageval-serving
Usage
-
Model: of course we have a model that is ready be evaluated, let's assume it lives in the path:
/path/to/model
; -
Then we can write our service code, let's put the service code in
service.py
or './tests/service.py' and take a NLP model as the example:from flageval.serving.service import NLPModelService, NLPEvalRequest, NLPEvalResponse, NLPCompletion class DemoService(NLPModelService): def global_init(self, model_path: str): print("Initial model with path", model_path) def infer(self, req: NLPEvalRequest) -> NLPEvalResponse: return NLPEvalResponse( completions=[ NLPCompletion( text='Hello, world!', tokens='Hello, world!', ), ] )
-
Finally, we use the
flageval-serving
command to serve:flageval-serving --service service:DemoService dev /path/to/model # start a development server flageval-serving --service service:DemoService run /path/to/model # start a production server
Dockerfile
FlagEval evaluation platform construction image
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
flageval_serving-0.1.6.tar.gz
(10.5 kB
view hashes)