FlagEval Serving

Serving Framework of AI Models for Evaluating on FlagEval Platform.

Installation

pip install --upgrade flageval-serving

Usage

Model: of course we have a model that is ready be evaluated, let's assume it lives in the path: /path/to/model;

Then we can write our service code, let's put the service code in service.py or './tests/service.py' and take a NLP model as the example:

from flageval.serving.service import NLPModelService, NLPEvalRequest, NLPEvalResponse, NLPCompletion


class DemoService(NLPModelService):
    def global_init(self, model_path: str):
        print("Initial model with path", model_path)

    def infer(self, req: NLPEvalRequest) -> NLPEvalResponse:
        return NLPEvalResponse(
            completions=[
                NLPCompletion(
                    text='Hello, world!',
                    tokens='Hello, world!',
                ),
            ]
        )

Finally, we use the flageval-serving command to serve:

flageval-serving --service service:DemoService dev /path/to/model  # start a development server
flageval-serving --service service:DemoService run /path/to/model  # start a production server

Dockerfile

FlagEval evaluation platform construction image

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flageval_serving-0.2.5.tar.gz (12.2 kB view details)

Uploaded Aug 11, 2025 Source

File details

Details for the file flageval_serving-0.2.5.tar.gz.

File metadata

Download URL: flageval_serving-0.2.5.tar.gz
Upload date: Aug 11, 2025
Size: 12.2 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.1.0 CPython/3.12.7

File hashes

Hashes for flageval_serving-0.2.5.tar.gz
Algorithm	Hash digest
SHA256	`c8869fa3db954fd93d1499d3be90ba0c541b025f196f55b0283049f269d694c8`
MD5	`6d84a685ffff9703b23abb27029c5b07`
BLAKE2b-256	`77f6347ad3d04567708e44d158d9536df1bfa86dc03c8b1d4a55739a3da69fd0`