Skip to main content

Xinference Restful Client

Xinference provides a restful client for managing and accessing models programmatically.

Install

pip install xinference-client

Usage

from xinference_client import RESTfulClient as Client

client = Client("http://localhost:9997")
model_uid = client.launch_model(model_name="chatglm2")
model = client.get_model(model_uid)

chat_history = []
prompt = "What is the largest animal?"
model.chat(
    prompt,
    chat_history=chat_history,
    generate_config={"max_tokens": 1024}
)

Result:

{
  "id": "chatcmpl-8d76b65a-bad0-42ef-912d-4a0533d90d61",
  "model": "56f69622-1e73-11ee-a3bd-9af9f16816c6",
  "object": "chat.completion",
  "created": 1688919187,
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The largest animal that has been scientifically measured is the blue whale, which has a maximum length of around 23 meters (75 feet) for adult animals and can weigh up to 150,000 pounds (68,000 kg). However, it is important to note that this is just an estimate and that the largest animal known to science may be larger still. Some scientists believe that the largest animals may not have a clear \"size\" in the same way that humans do, as their size can vary depending on the environment and the stage of their life."
      },
      "finish_reason": "None"
    }
  ],
  "usage": {
    "prompt_tokens": -1,
    "completion_tokens": -1,
    "total_tokens": -1
  }
}

Metadata

Release files for xinference-client 4.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xinference-client 4.2.0
File Size Uploaded Provenance Actions
xinference_client-4.2.0.tar.gz 67.6 kB Without provenance Details for xinference_client-4.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for xinference-client 4.2.0
File Interpreter ABI Platform Actions
xinference_client-4.2.0-py3-none-any.whl Python 3 none any Details for xinference_client-4.2.0-py3-none-any.whl

Total release size: 115.7 kB

Release files / xinference_client-4.2.0.tar.gz

Download URL xinference_client-4.2.0.tar.gz
Size 67.6 kB
Tags Source
SHA-256 checksum
How to use checksums
a041b9c78a8405ad049341dda792a6e2d85ef19ced59ac9f3f462f294be78263
BLAKE2b-256 checksum
How to use checksums
9b7dc3fb03f539607026fa230cd7fb3fc44cb275337d954e378ca1b12ca9c436
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Release files / xinference_client-4.2.0-py3-none-any.whl

Download URL xinference_client-4.2.0-py3-none-any.whl
Size 48.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ae277bddf09c50d7ce98ebae7921253737118aa2fcba506d4d534ca7637b3a37
BLAKE2b-256 checksum
How to use checksums
6490c54a85609498eae820f8711f3fc844f974b89d320eba4b0c9e75a73678d4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Security for xinference-client 4.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.2.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page