FlagEval Serving

Serving Framework of AI Models for Evaluating on FlagEval Platform.

Installation

pip install --upgrade flageval-serving

Usage

Model: of course we have a model that is ready be evaluated, let's assume it lives in the path: /path/to/model;

Then we can write our service code, let's put the service code in service.py or './tests/service.py' and take a NLP model as the example:

from flageval.serving.service import NLPModelService, NLPEvalRequest, NLPEvalResponse, NLPCompletion


class DemoService(NLPModelService):
    def global_init(self, model_path: str):
        print("Initial model with path", model_path)

    def infer(self, req: NLPEvalRequest) -> NLPEvalResponse:
        return NLPEvalResponse(
            completions=[
                NLPCompletion(
                    text='Hello, world!',
                    tokens='Hello, world!',
                ),
            ]
        )

Finally, we use the flageval-serving command to serve:

flageval-serving --service service:DemoService dev /path/to/model  # start a development server
flageval-serving --service service:DemoService run /path/to/model  # start a production server

Dockerfile

FlagEval evaluation platform construction image

Metadata

Release files for flageval-serving 0.2.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for flageval-serving 0.2.5
File	Size	Uploaded
flageval_serving-0.2.5.tar.gz	12.2 kB	Aug 11, 2025	Details

Release files / flageval_serving-0.2.5.tar.gz

Download URL	flageval_serving-0.2.5.tar.gz
Size	12.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`c8869fa3db954fd93d1499d3be90ba0c541b025f196f55b0283049f269d694c8`
BLAKE2b-256 checksum How to use checksums	`77f6347ad3d04567708e44d158d9536df1bfa86dc03c8b1d4a55739a3da69fd0`
Upload date	Aug 11, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.1.0 CPython/3.12.7`