Fastspeed

Fastspeed is a toolkit for serving LLM.

Requirements

Python >= 3.6

Installation

pip install fastspeed

Example

server

fastspeed --model model_name_or_path  # default model_hub=modelscope
fastspeed --model model_name_or_path --model_hub hf

client

curl -X 'POST' \
  'http://127.0.0.1:8000/chat/completions' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ],
  "model": ""
}'

Change Log

0.1.1 (2024-09-15)

Suport half, cuda and trust_remote_code=True

0.1.0 (2024-08-29)

Fastapi with single GPU, model from huggingface or modelscope

Release files for fastspeed 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fastspeed 0.1.1
File	Size	Uploaded
fastspeed-0.1.1.tar.gz	4.5 kB	Sep 14, 2024	Details

Built distribution (wheel)

Table of built distributions (wheels) for fastspeed 0.1.1
File	Interpreter	ABI	Platform	Reset
fastspeed-0.1.1-py2.py3-none-any.whl 4.4 kB Sep 14, 2024	Python 2, Python 3	none	any	Details

Total release size: 8.9 kB

Release files / fastspeed-0.1.1.tar.gz

Download URL	fastspeed-0.1.1.tar.gz
Size	4.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`5d9d82b8d6159f0e5bd6d424c422b6156ce8c630298fecc6515bd8c920d06516`
BLAKE2b-256 checksum How to use checksums	`dc0e25e591b93593288add097758fbf478bae7a79eefcbb28f216f1b7998c925`
Upload date	Sep 14, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/5.1.1 CPython/3.11.10`

Release files / fastspeed-0.1.1-py2.py3-none-any.whl

Download URL	fastspeed-0.1.1-py2.py3-none-any.whl
Size	4.4 kB
Tags	Python 2 Python 3
SHA-256 checksum How to use checksums	`0be6cac2258b54a85a9595a1688279543707834095f5fdef0466e54c006a4464`
BLAKE2b-256 checksum How to use checksums	`68512b9cd08f5fdcdf0320d018e98466c40edb0e303e5ac838694123e0654e47`
Upload date	Sep 14, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/5.1.1 CPython/3.11.10`