Skip to main content

Xinference Restful Client

Xinference provides a restful client for managing and accessing models programmatically.

Install

pip install xinference-client

Usage

from xinference_client import RESTfulClient as Client

client = Client("http://localhost:9997")
model_uid = client.launch_model(model_name="chatglm2")
model = client.get_model(model_uid)

chat_history = []
prompt = "What is the largest animal?"
model.chat(
    prompt,
    chat_history=chat_history,
    generate_config={"max_tokens": 1024}
)

Result:

{
  "id": "chatcmpl-8d76b65a-bad0-42ef-912d-4a0533d90d61",
  "model": "56f69622-1e73-11ee-a3bd-9af9f16816c6",
  "object": "chat.completion",
  "created": 1688919187,
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The largest animal that has been scientifically measured is the blue whale, which has a maximum length of around 23 meters (75 feet) for adult animals and can weigh up to 150,000 pounds (68,000 kg). However, it is important to note that this is just an estimate and that the largest animal known to science may be larger still. Some scientists believe that the largest animals may not have a clear \"size\" in the same way that humans do, as their size can vary depending on the environment and the stage of their life."
      },
      "finish_reason": "None"
    }
  ],
  "usage": {
    "prompt_tokens": -1,
    "completion_tokens": -1,
    "total_tokens": -1
  }
}

Metadata

Release files for xinference-client 4.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xinference-client 4.1.0
File Size Uploaded Provenance Actions
xinference_client-4.1.0.tar.gz 65.6 kB Without provenance Details for xinference_client-4.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for xinference-client 4.1.0
File Interpreter ABI Platform Actions
xinference_client-4.1.0-py3-none-any.whl Python 3 none any Details for xinference_client-4.1.0-py3-none-any.whl

Total release size: 111.4 kB

Release files / xinference_client-4.1.0.tar.gz

Download URL xinference_client-4.1.0.tar.gz
Size 65.6 kB
Tags Source
SHA-256 checksum
How to use checksums
96e5c86d8f025e9cb31ef812320f30ac367332eb897ff3f8b70fb3a58611e420
BLAKE2b-256 checksum
How to use checksums
7a0c9448d110e882975d807572d8fbf541c39bf0c2c986d07ab2e8fe12e57f55
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Release files / xinference_client-4.1.0-py3-none-any.whl

Download URL xinference_client-4.1.0-py3-none-any.whl
Size 45.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
87931c3d819d24cad1c0df015b24b3afdcc20038bdd333731e854b20d67e7a75
BLAKE2b-256 checksum
How to use checksums
d6145e7a76fb6a409bdf2176a80f54d70e0ecb5e2f9b723897af3f470ffe4ecb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Security for xinference-client 4.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

4.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page