xiaothink 1.4.7

An AI toolkit that helps users quickly call interfaces related to the Xiaothink framework.

Xiaothink Python Module Usage Documentation

Xiaothink is an AI research organization focused on Natural Language Processing (NLP), dedicated to training advanced on-device models with limited data and computing resources. The Xiaothink Python module is our core toolkit, covering various functions such as text-based Q&A, multimodal Q&A, image compression, sentiment classification, and more. Below is the detailed usage guide and code examples.

Table of Contents

1. Installation
2. License
3. Local Dialogue Models
4. PaddlePaddle-based Models
5. PyTorch-based Models (T17)
   • Translation
   • Novel Role-Playing (v1.4.4 New)
6. Image Feature Extraction and Multimodal Dialogue
7. Image Compression to Feature Technology (img_zip)
8. Sentiment Classification Tool
9. AI Rate Detection Tool
10. TinySkill Toolbox
11. ImgTok Vision Support
12. Image Compression 2nd Generation (img_zip2)
13. OpenAI-Compatible Server & Web Chat (v1.4.7 New)
14. Model Name Reference Table
15. Changelog

---

Installation

First, you need to install the Xiaothink module via pip:

pip install xiaothink          # Basic install (no deep learning backend)
pip install xiaothink[torch]   # With PyTorch (recommended for T17)
pip install xiaothink[all]     # With all backends

Backend Control via Environment Variable (v1.4.2 New)

Set the XIAOTHINK_BACKEND environment variable to control which deep learning backend the library imports. This prevents compatibility issues from unused backends (e.g., numpy 2.0 breaking TensorFlow).

Usage (set before running your Python script):

Command	Effect
set XIAOTHINK_BACKEND=torch (Windows)	Import PyTorch only (recommended for T17 users)
set XIAOTHINK_BACKEND=paddle (Windows)	Import PaddlePaddle only (recommended for T7.5 users)
set XIAOTHINK_BACKEND=tensorflow (Windows)	Import TensorFlow only (recommended for T7 and below)
Not set or auto	Try all backends (default)

Example (Windows PowerShell):

# Use PyTorch only, completely skip TensorFlow and PaddlePaddle
$env:XIAOTHINK_BACKEND='torch'
python your_script.py

Example (Linux/Mac):

# Use PyTorch only, completely skip TensorFlow and PaddlePaddle
export XIAOTHINK_BACKEND=torch
python your_script.py

Tip: If you have multiple deep learning frameworks installed but only use one, set this environment variable to speed up imports and avoid compatibility issues.

---

License

This project is licensed under the Apache License, Version 2.0 - see the LICENSE file for details.

The NOTICE file contains additional attribution information for the proprietary technologies included in this module.

---

Local Text-only Dialogue Models

For locally loaded dialogue models, you should call the corresponding function according to the model type.

Single-turn Dialogue (to be removed in future versions)

Suitable for single-turn dialogue scenarios.

Example Code

import xiaothink.llm.inference.test_formal as tf

model = tf.QianyanModel(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'# vocab file is provided in the model repository
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat_SingleTurn(inp, temp=0.32)  # Use chat_SingleTurn for single-turn dialogue
    print('\n[A]:', re, '\n')

Multi-turn Dialogue

Suitable for multi-turn dialogue scenarios.

Example Code

import xiaothink.llm.inference.test_formal as tf

model = tf.QianyanModel(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'# vocab file is provided in the model repository
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temp=0.32)  # Use chat for multi-turn dialogue
    print('\n[A]:', re, '\n')

Manually Adding History

Use the add_his method to manually add historical dialogue records, suitable for presetting dialogue context.

import xiaothink.llm.inference.test_formal as tf

model = tf.QianyanModel(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'
)

# Manually add history
model.add_his('What is your name?', 'My name is "Xiaosi", nice to meet you!')
model.add_his('Hello, Xiaosi', 'Hello to you too!')

# Subsequent dialogue will be based on the added history context
while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temp=0.32)
    print('\n[A]:', re, '\n')

Note:

• add_his(q, a, form) method accepts three parameters: question q, answer a, and format form
• T6 series models use form=1 (instruction-tuned models) by default, pre-trained models use form='pretrain'
• T7.5 series models use form=2
• T17 series models (PyTorch) do not require the form parameter

Text Continuation

Suitable for more flexible text continuation scenarios.

Example Code

import xiaothink.llm.inference.test as test

MT = 't6_beta_dense'
m, d = test.load(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'# vocab file is provided in the model repository
)

inp='Hello!'
belle_chat = '{"conversations": [{"role": "user", "content": {inp}}, {"role": "assistant", "content": "'.replace('{inp}', inp)    # Instruct format supported by instruction-tuned models in the T6 series
inp_m = belle_chat

ret = test.generate_texts_loop(m, d, inp_m,    
                               num_generate=100,
                               every=lambda a: print(a, end='', flush=True),
                               temperature=0.32,
                               pass_char=['▩'])    # ▩ is the <unk> token for T6 series models

Important Note: For local models, it is recommended to use the model.chat function for multi-turn dialogue. For pre-trained models without instruction tuning, it is recommended to use the test.generate_texts_loop function. The single-turn dialogue function model.chat_SingleTurn will be removed in future versions.

---

PaddlePaddle-based Models (New in v1.4.0)

Xiaothink now supports PaddlePaddle-based models with RWKV architecture. These models offer efficient inference and are suitable for resource-constrained environments.

Installation

pip install xiaothink
pip install paddlepaddle  # or paddlepaddle-gpu for GPU support

Multi-turn Dialogue (PaddlePaddle)

from xiaothink.llm.inference_paddle import QianyanModel

model = QianyanModel(
    ckpt_dir=r'path/to/your/t7.5_model',
    MT='t7.5_paddle_small_instruct_pro'
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temp=0.34, form=2)  # form=2 for simplified format
    print('\n[A]:', re, '\n')

Text Generation (PaddlePaddle)

from xiaothink.llm.inference_paddle import TextGenerator

generator = TextGenerator(
    checkpoint_path=r'path/to/your/t7.5_model',
    MT='t7.5_paddle_small_instruct_pro'
)
generator.load_model()

generated_text = generator.generate_text(
    prompt='<|U|>Hello, how are you?<|A|>',
    max_length=100,
    temperature=0.8,
    top_p=0.9,
    repetition_penalty=1.2
)
print(generated_text)

Supported Model Architectures (PaddlePaddle)

Model Name	MT Parameter	Description	form Parameter
Xiaothink-T7.5-0.1B	't7.5_paddle_small_instruct'	Base instruction model	form=2
Xiaothink-T7.5-0.1B-Pro	't7.5_paddle_small_instruct_pro'	Enhanced instruction model	form=2
Xiaothink-T7.5-0.1B-Thinking	't7.5_paddle_small_instruct_thinking'	Chain-of-thought model	form=2
Xiaothink-T7.5-0.1B-Poem	't7.5_paddle_small_instruct_poem'	Poetry generation model	form=2
Xiaothink-T7.5-0.1B-Translator	't7.5_paddle_small_instruct_translator'	Translation model	form=2

Automatic Device Selection

The PaddlePaddle module automatically selects the optimal device based on GPU memory usage:

# Automatic device selection (default)
# If GPU memory usage > 80%, switches to CPU
AUTO_DEVICE = True
GPU_MEMORY_THRESHOLD = 80.0

# Or manually set device
import paddle
paddle.set_device('cpu')  # or 'gpu:0'

Train-On-Time (TOT) Dynamic Learning

Xiaothink provides an innovative Train-On-Time (TOT) feature that enables dynamic learning during inference. Unlike traditional models that only use pre-trained knowledge, TOT models continuously learn from similar examples in your training data repository. This technology can significantly enhance the model's basic capabilities. However, it's important to note that TOT models' inference speed will be somewhat affected, as it needs to perform similarity matching and fine-tuning before each conversation. Additionally, this technology cannot significantly improve the model's mathematical and reasoning abilities, and can only be used to enhance basic model capabilities.

How TOT Works:

1. Similarity Matching: Automatically finds similar instructions from your training data
2. Dynamic Fine-tuning: Fine-tunes the model in memory using these similar examples
3. Enhanced Response: Generates more accurate answers based on the newly learned knowledge
4. Memory Management: Optimizes GPU memory usage and cleans up resources

Key Features:

• Real-time Learning: Every conversation triggers learning from relevant examples
• Similarity-based Matching: Uses difflib to find semantically similar instructions
• Parallel Processing: Multi-core similarity calculation for faster matching
• Multi-format Support: Loads models from various checkpoint formats
• GPU Optimization: Automatic GPU memory management

from xiaothink.llm.inference_paddle import TOTModel, TOT_AVAILABLE

if TOT_AVAILABLE:
    # 自定义训练数据路径
    custom_data_paths = [
        r'path/to/your/belle_train.jsonl',
        r'path/to/your/coig_minimind.jsonl',
        r'path/to/your/firefly_data.jsonl'
    ]
    
    model = TOTModel(
        ckpt_dir=r'path/to/your/t7.5_model',
        MT='t7.5_paddle_small_instruct_pro',
        data_paths=custom_data_paths  # 自定义训练数据路径
    )
    
    # The model will automatically learn from similar examples before answering
    while True:
        inp = input('[Q]: ')
        if inp == '[CLEAN]':
            model.clean_his()
            continue
        re = model.chat(inp, temp=0.68)
        print('\n[A]:', re, '\n')
else:
    print("TOT feature requires PaddlePaddle support")

Custom Data Paths: You can now specify your own training data paths when initializing TOTModel:

# Default behavior (uses built-in paths)
model = TOTModel(ckpt_dir='path/to/model')

# Custom data paths
model = TOTModel(
    ckpt_dir='path/to/model',
    data_paths=[
        'path/to/data1.jsonl',
        'path/to/data2.jsonl',
        'path/to/data3.txt'
    ]
)

Training Data Requirements: The TOT system allows users to customize training datasets by passing them through the data_paths parameter. The system will load and process training data based on the paths provided by the user.

Supported File Formats

The TOT system supports the following file formats:

• .jsonl files (JSON Lines format)
• .txt files (text files)

Supported Data Structures

The system can identify and process two main data structures:

1. Conversations Format

{
  "conversations": [
    {"content": "User question"},
    {"content": "Assistant answer"}
  ]
}

2. Instruction-Output Format

{
  "instruction": "Instruction content",
  "output": "Output content"
}

Memory Optimization:

• Automatic GPU memory cleanup
• In-memory fine-tuning without disk writes
• Batch processing for efficient training

---

PyTorch-based Models (T17, v1.4.2 New)

Xiaothink now supports PyTorch-based models with GRU/LSTM architecture and history retrieval mechanism. These models are designed for efficient on-device inference with innovative history-aware capabilities.

Installation

pip install xiaothink
pip install torch  # PyTorch is required for T17 models

Multi-turn Dialogue (PyTorch)

from xiaothink.llm.inference_torch.torch_formal import TorchModel

model = TorchModel(
    ckpt_dir=r'path/to/your/t17_model',
    MT='t17_tiny'
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temperature=0.7, top_p=0.9)
    print('\n[A]:', re, '\n')

Text Generation (PyTorch)

from xiaothink.llm.inference_torch.torch_test import TextGenerator

generator = TextGenerator(
    checkpoint_path=r'path/to/your/t17_model',
    MT='t17_tiny'
)
generator.load_model()

generated_text = generator.generate_text(
    prompt='<s>你好，请介绍一下自己。',
    max_length=128,
    temperature=0.7,
    top_p=0.9
)
print(generated_text)

Supported Model Architectures (PyTorch)

Model Name	MT Parameter	Description
Xiaothink-T17-Tiny	't17_tiny'	GRU3-based history retrieval model (d_model=450, num_layers=14)
Xiaothink-T17-Tiny-Novel	't17_tiny_novel'	Long-text history retrieval model (d_model=768, max_seq_len=10000)

Key Features

• History Retrieval: Innovative mechanism that retrieves relevant historical context during generation
• Efficient Inference: Optimized for on-device deployment with low memory footprint
• Gate Fusion: Learnable gate mechanism for blending RNN output with retrieved history
• Causal Masking: Ensures no future information leakage during training and inference
• ImgTok Vision Integration (vision models): Automatic <imgtoken> tag processing and ImgTok output reconstruction

Translation (PyTorch)

TorchModel provides a built-in translate() method supporting English-Chinese translation:

from xiaothink.llm.inference_torch.torch_formal import TorchModel

model = TorchModel(ckpt_dir=r'path/to/your/t17_model', MT='t17_tiny')

# English to Chinese
result = model.translate("Hello, how are you?", mode="en2zh")
print(result)

# Chinese to English
result = model.translate("今天天气真好", mode="zh2en")
print(result)

# Free translation mode
result = model.translate("帮我翻译这段话", mode="free")
print(result)

Novel Role-Playing (TorchNovelModel, New in v1.4.4)

TorchNovelModel extends TorchModel with novel role-playing features, supporting multi-character auto-completion and story background settings.

from xiaothink.llm.inference_torch.torch_novel import TorchNovelModel

# Initialize novel model (vision model recommended for ImgTok support)
model = TorchNovelModel(
    ckpt_dir=r'path/to/your/t17_model',
    MT='t17_tiny_vision'
)

# Set characters
model.set_model_name('顾言;林婉')     # Model role(s), multiple separated by ;
model.set_user_name('林婉')           # User role
model.set_setting('夜黑风高的晚上，林婉正准备睡觉。')  # Story setting

# chat_auto: auto-format dialogue with character names
response = model.chat_auto('你好！（招了招手）')
# Auto-formatted as: 林婉说道："你好！（招了招手）"顾言说道："

# chat: manual-format dialogue
response = model.chat('林婉说："你还好吗？"')

# Multi-character auto-completion: if output starts with "顾", auto-completes to "顾言"

Character First-Character Auto-Completion:

• When model_name='顾言;林婉', the first output token is constrained to character first characters only
• If the first character matches (e.g., "顾"), the rest of the name ("言") is force-appended
• Stops at newlines or four spaces (not preserved), or " (right quote, preserved)

---

Image Feature Extraction and Multimodal Dialogue

Dual-vision Solution

In version 1.2.0, we introduced an innovative dual-vision solution:

1. Image Compression to Feature (img_zip): Convert images to text tokens that can be inserted anywhere in the dialogue.
2. Native Vision Encoder: Pass the latest image to the native vision model's vision encoder (standard approach).

This solution achieves:

• Detailed analysis of the latest single image based on the native vision encoder
• Understanding of multiple images in the context based on img_zip technology
• Significant reduction in computing resource requirements

Vision Model Usage Guidelines

For vision-enabled models, regardless of whether there is image input, you should use the following code:

from xiaothink.llm.inference.test_formal import QianyanModel

if __name__ == '__main__':
    model = QianyanModel(
        ckpt_dir=r'path/to/your/vision_model',
        MT='t6_standard_vision',  # Note: model type is vision model
        vocab=r'path/to/your/vocab.txt',
        imgzip_model_path='path/to/img_zip/model.keras'  # Specify img_zip model path
    )

    temp = 0.28  # Temperature parameter
    
    while True:
        inp = input('[Q]: ')
        if inp == '[CLEAN]':
            print('[Context Cleared]\n\n')
            model.clean_his()
            continue
        # Use chat_vision for dialogue
        ret = model.chat_vision(inp, temp=temp, pre_text='', pass_start_char=[])
        print('\n[A]:', ret, '\n')

Important Notes:

• Vision models must use the chat_vision method; do not use chat (which is only for text-only models)
• You must prepare an img_zip image compression encoder model that matches the vision model
• Mismatched models will cause the model to fail to understand the meaning of encoded tokens

Image Processing Interfaces

Two new image processing interfaces have been added:

1. img2ms (for non-native vision models):

   description = model.img2ms('path/to/image.jpg', temp=0.28)
   print(description)
   

2. img2ms_vision (for native vision models):

   description = model.img2ms_vision('path/to/image.jpg', temp=0.28, max_shape=224)
   print(description)
   

Image Reference Syntax

In dialogue, use the following syntax to reference images:

<img>image path or URL</img>Please describe this image

The model will automatically parse the image path, extract features, and answer based on the image content.

Notes:

1. Image paths should use absolute paths to ensure correct parsing
2. Native vision models only support analyzing the most recent image
3. img_zip technology supports referencing multiple images in the context

---

Image Compression to Feature Technology (img_zip)

The img_zip module provides advanced image and video compression/decompression functions based on deep learning feature extraction technology. Below are the detailed usage methods:

1. Command-line Interactive Mode

python -m xiaothink.llm.img_zip.img_zip

After running, you will enter an interactive command-line interface:

===== img_zip Image Video Compression Tool =====
Please enter .keras model path: path/to/your/imgzip_model.keras
Model loaded successfully!

Please select a function:
1. Compress image
2. Decompress image
3. Compress video
4. Decompress video
0. Exit

Please select (0-4):

2. Python Code Invocation

from xiaothink.llm.img_zip.img_zip import ImgZip

# Initialize instance
img_zip = ImgZip(model_path='path/to/your/imgzip_model.keras')

# Compress image
compressed_path = img_zip.compress_image(
    img_path='input.jpg',
    patch=True,  # Whether to use patch processing
    save_path='compressed_img',  # Save path prefix
    ability=0.02,# New feature in 1.2.5: Set custom compression rate to 0.02 (when ability is 0, it means not using custom compression rate). The algorithm calculates and compresses to a close size (there may be errors between theoretical calculation and actual size)
)

# Generates two files: compressed_img.npy and compressed_img.shape

# Decompress image
img_zip.decompress_image(
    compressed_input='compressed_img',  # Compressed file prefix
    patch=True,  # Whether to use patch processing
    save_path='decompressed.jpg'  # Output path
)

# Compress video
compressed_paths, metadata_path = img_zip.compress_video(
    video_path='input.mp4',
    output_dir='compressed_video',  # Output directory
    patch=True  # Whether to use patch processing
)

# Decompress video
img_zip.decompress_video(
    compressed_dir='compressed_video',  # Compressed file directory
    output_path='decompressed.mp4'  # Output path
)

# Convert image to array and save
img_array = img_zip.image_to_array('input.jpg')
img_zip.save_image_array(img_array, 'image_array.npy')

# Load image from array
loaded_array = img_zip.load_image_array('image_array.npy')
img = img_zip.array_to_image(loaded_array)
img.save('restored.jpg')

3. Key Function Descriptions

1. Compress Image (compress_image)

   • patch=True: Split large images into 80x80 patches for separate processing
   • Outputs two files: .npy (feature vectors) and .shape (original size information)

2. Decompress Image (decompress_image)

   • Requires both .npy and .shape files
   • Automatically restores original dimensions

3. Video Processing (compress_video/decompress_video)

   • Automatically extracts video frames and processes them in batches
   • Preserves original video frame rate and resolution information
   • Uses temporary directories for intermediate file processing

4. Parameter Descriptions

Parameter	Type	Description
model_path	str	Path to img_zip model (.keras file)
patch	bool	Whether to use patch processing (default: True)
save_path	str	Output file path prefix
img_path	str	Input image path
video_path	str	Input video path
output_dir	str	Output directory path
output_path	str	Output file path

5. Processing Flow Features

1. Patch Processing:

   • Automatically splits large images into 80x80 patches
   • Each patch is independently encoded into feature vectors
   • Preserves original size information

2. Video Processing:

   • Automatically extracts frames and processes them in batches
   • Preserves original video parameters (fps, resolution)
   • Uses temporary directories for intermediate file processing

3. Progress Display:

   • All operations come with detailed progress bars
   • Displays current processing step and remaining time

4. Error Handling:

   • Comprehensive exception catching mechanism
   • Detailed error information prompts

6. Usage Recommendations

1. For images larger than 80x80, it is recommended to use patch processing (patch=True)
2. Video processing requires sufficient disk space for temporary frame files
3. Ensure the input model matches the processing task
4. Use absolute paths to avoid file location issues

This module is the core component of Xiaothink vision models (especially non-native ones). Based on efficient image feature representation and compression, it can enable any text-only AI model to have basic vision capabilities through fine-tuning.

---

Sentiment Classification Tool

The sentiment classification tool is based on loaded dialogue models and provides text sentiment tendency analysis functionality, which can quickly determine the sentiment category of input text (e.g., positive, negative, neutral, etc.).

Feature Description

• This tool is a customized interface based on Xiaothink framework (Xiaothink T6 series, etc.) models
• Implements sentiment classification based on Xiaothink framework language models without the need to load additional classification models
• Supports input of ultra-long text and returns sentiment analysis results
• It is recommended to use single-turn dialogue enhanced models, such as: Xiaothink-T6-0.15B-ST

Usage Example

from xiaothink.llm.inference.test_formal import *
from xiaothink.llm.tools.classify import *

if __name__ == '__main__':
    # Initialize basic dialogue model
    model = QianyanModel(
        ckpt_dir=r'path/to/your/t6_model',  # Model weight directory  It is recommended to use _ST version models
        MT='t6_standard',  # Model type (must match weights)
        vocab=r'path/to/your/vocab.txt',  # Vocabulary path
        use_patch=0  # Do not use patch processing (text-only model)
    )
    
    # Initialize sentiment classification model (depends on basic dialogue model)
    cmodel = ClassifyModel(model)
    
    # Loop input text for sentiment classification
    while True:
        inp = input('Enter text: ')
        res = cmodel.emotion(inp)  # Call sentiment classification interface
        print(res)  # Output sentiment analysis results

Notes

1. The sentiment classification model depends on an initialized QianyanModel; ensure the base model is loaded successfully
2. It is recommended to use instruction-tuned models (e.g., t6_standard); non-tuned models may affect classification accuracy
3. The output result format is: {'Positive': 0.6667, 'Negative': 0.1667, 'Neutral': 0.1667}

---

AI Rate Detection Tool

The AI rate detection tool is based on loaded detection models and provides text AI generation probability analysis functionality. It can accurately determine the AI generation probability of each character in the text, output the overall AI rate average, and return detailed character-level detection information, achieving comprehensive traceability analysis of text AI generation traces.

Feature Description

• This tool is a customized interface based on Xiaothink framework (Xiaothink T series, etc.) models
• Implements text AI rate analysis based on Xiaothink framework detection models without the need to load independent detection models
• Supports ultra-long text detection and batch text detection, returning multi-dimensional complete detection results
• Can output four levels of results: overall AI rate average, detection conclusion, probability statistics information, and character-level detailed information

Usage Example

if __name__ == "__main__" and 1:
    # 1. Initialize detector
    detector = AIDetector(
        ckpt_dir=r'E:\Xiaothink Framework\Paper\ganskchat\ckpt_test_t7',
        model_type='t7',
        print_load_info=True
    )

    # 2. Detect text (Some models may not support Chinese text detection)
    test_texts = [
        "This is a sentence that a car repair blogger active on mobile internet used to start many of his videos before being sued by BYD. Finally, this 'most miserable repairman in history' has received the first-instance judgment of being sued by BYD.",
        "\"Isn't it,\" Grandma looked up at the osmanthus tree, her eyes filled with gentle memories, \"This was planted by your grandfather back then, almost thirty years ago. At that time, he said, planting an osmanthus tree, it will bloom in autumn, fragrant and beautiful, and when we have children, we can make osmanthus cake to eat.\"",
        "These days, my heart has been quite unsettled. Sitting in the yard enjoying the cool air tonight, I suddenly thought of the lotus pond I pass by every day. In this moonlight of the full moon, it should have a different appearance. The moon gradually rose higher, and the laughter of children on the road outside the wall could no longer be heard; my wife was patting Run'er inside the house, humming a lullaby drowsily. I quietly put on my large shirt and went out the door."
    ]

    # 3. Execute detection
    for text in test_texts:
        print(f"\n{'='*60}")
        print(f"Detected Text: {text}")
        result = detector.detect_ai_rate(text)
        
        print(f"AI Rate (Probability Average): {result['AI Rate (Probability Average)']}")
        print(f"Detection Conclusion: {result['Detection Conclusion']}")
        print(f"Probability Statistics: Min={result['Probability Statistics']['Minimum Probability']} | Max={result['Probability Statistics']['Maximum Probability']}")
        
        # Optional: Print character-level details
        print("\nCharacter-level Details:")
        for detail in result['Character-level Details']:
            print(f"  Position {detail['Character Position']}: Previous Text 「{detail['Complete Previous Text']}」→ Character 「{detail['Target Character']}」→ Probability {detail['Prediction Probability']}")

    # 4. Release resources
    detector.close()

Notes

1. When initializing the AI rate detector, ensure ckpt_dir points to the correct model weight directory; otherwise, model loading will fail
2. Automatic Backend Selection: Based on model_type, the backend is automatically selected:
   • t17 series models → PyTorch backend
   • t7.5 / paddle series models → PaddlePaddle backend
   • Other models → TensorFlow backend
3. Core Accuracy Note: This tool has relatively accurate AI rate detection results for small model-generated text, which can meet the traceability needs of small model-generated content; however, it has poor AI rate detection effect for large model-generated text, and the detection results have low reference value. It is strictly prohibited to use this tool for AI determination scenarios of large model-generated content
4. After detection is completed, you must call the detector.close() method to release resources such as video memory and hardware handles to avoid memory leaks and excessive video memory usage caused by long-term operation
5. Character-level details are optional output items. For ultra-long text of ten thousand characters, printing these details will significantly increase output time and can be selectively printed according to actual needs
6. When detecting a large number of texts in batches, it is recommended to process them in batches according to text length to avoid detection lag caused by passing too many ultra-long texts in a single batch
7. Enabling print_load_info=True when loading the model allows you to view loading progress and hardware adaptation information, which is convenient for troubleshooting model loading exceptions

PyTorch Backend Example (T17 Models)

from xiaothink.llm.tools.ai_possibility import AIDetector

if __name__ == "__main__":
    # Initialize detector (automatically selects PyTorch backend)
    detector = AIDetector(
        ckpt_dir=r'path/to/your/t17_model',
        model_type='t17_tiny',
        print_load_info=True
    )

    # Detect text
    test_text = "This is a text to be detected."
    result = detector.detect_ai_rate(test_text)
    
    print(f"AI Rate: {result['AI Rate (Probability Average)']}")
    print(f"Detection Conclusion: {result['Detection Conclusion']}")

    # Release resources
    detector.close()

Threshold Calibration (v1.4.2 New)

The AI rate detection tool provides flexible threshold configuration and calibration to adapt to different detection scenarios.

1. Default Thresholds

Threshold	Default	Description
threshold_high	0.01	AI rate >= this value → "High probability AI generated"
threshold_low	0.005	AI rate >= this value → "Suspected AI generated"

2. Manual Threshold Adjustment

Developers can set custom thresholds during initialization or dynamically:

# Method 1: Set during initialization
detector = AIDetector(
    ckpt_dir=r'path/to/your/t17_model',
    model_type='t17_tiny',
    threshold_high=0.02,   # Custom high threshold
    threshold_low=0.008    # Custom low threshold
)

# Method 2: Dynamic adjustment
detector.set_thresholds(high=0.015, low=0.006)  # Modify high threshold only
detector.set_thresholds(low=0.004)               # Modify low threshold only

3. User Calibration

Automatically calculate optimal thresholds by providing AI-generated and human text samples. It is recommended to provide at least 3 samples per category; longer texts yield better calibration.

Interactive Calibration (suitable for terminal use):

detector = AIDetector(
    ckpt_dir=r'path/to/your/t17_model',
    model_type='t17_tiny'
)
# Enter interactive calibration: follow prompts to input 3 AI + 3 human texts
calib_result = detector.auto_calibrate()
print(f"Calibrated high threshold: {calib_result['Calibrated Thresholds']['High Threshold']:.6f}")
print(f"Calibrated low threshold: {calib_result['Calibrated Thresholds']['Low Threshold']:.6f}")

Programmatic Calibration (suitable for code integration):

detector = AIDetector(ckpt_dir=r'path/to/your/t17_model', model_type='t17_tiny')

ai_samples = [
    "This is the first AI-generated text for calibration.",
    "This is the second AI-generated text with different content.",
    "This is the third AI-generated text for better accuracy."
]
human_samples = [
    "This is a human-written text for distinguishing AI from human.",
    "This is the second human-written text in natural style.",
    "This is the third human sample for improved reliability."
]

calib_result = detector.calibrate(ai_samples, human_samples)
print(f"Before: high={calib_result['Pre-calibration Thresholds']['High Threshold']}, "
      f"low={calib_result['Pre-calibration Thresholds']['Low Threshold']}")
print(f"After: high={calib_result['Calibrated Thresholds']['High Threshold']}, "
      f"low={calib_result['Calibrated Thresholds']['Low Threshold']}")

4. View Current Threshold Configuration

info = detector.get_threshold_info()
print(f"High Threshold: {info['High AI Threshold']}")
print(f"Low Threshold: {info['Low AI Threshold']}")
print(f"Calibrated: {info['Calibrated']}")

Each detection result also includes the current threshold configuration:

result = detector.detect_ai_rate("Test text")
print(result['Threshold Configuration'])
# Output: {'High Threshold': 0.01, 'Low Threshold': 0.005, 'Calibrated': False}

---

TinySkill Toolbox (v1.4.2 New)

TinySkill is a lightweight skill registration and execution mechanism provided by the Xiaothink library. By registering prompt templates and execution types, users can quickly perform text generation or text classification tasks with the T17 model.

Each TinySkill can set its own temperature, max_length, repetition_penalty defaults, which can be overridden at runtime via **kwargs.

Note: As of the Xiaothink-T17 series release, all models only support Chinese text processing. TinySkill prompts and keywords must be in Chinese.

1. Core Concepts

Concept	Description
TinySkill	A task definition containing an id, name, prompt template, and execution type
generate type	Text generation task — the model generates text based on the prompt
classify type	Text classification task — calculates probabilities based on keyword hit rates in the model output
TinyToolbox	Manages registration, querying, and execution of TinySkills

2. Quick Start

from xiaothink.llm.inference_torch.torch_formal import TorchModel
from xiaothink.llm.tools.tiny_skill import TinyToolbox, register_default_skills

# 1. Load the model
model = TorchModel(ckpt_dir="model_weight_directory", MT="t17_tiny")
model.set_form("ua_chat")

# 2. Create toolbox and register built-in skills
toolbox = TinyToolbox(model)
register_default_skills(toolbox)

# 3. Execute by skill name
result = toolbox.run("题目生成", "春天来了，万物复苏。")
print(result["text"])  # Generated title

# 4. Execute by skill id
result = toolbox.run("generate_title", "人工智能正在改变世界")
print(result["text"])

# 5. List all built-in skills
for sk in toolbox.list_skills():
    print(f"[{sk['id']}] {sk['name']} - {sk['type']} - {sk['description']}")

3. Registering Custom TinySkills

3.1 Text Generation (generate)

Default params for generate type: temperature=0.5, max_length=64, repetition_penalty=1.1

from xiaothink.llm.tools.tiny_skill import TinySkill

skill = TinySkill(
    id_="summarize_cn",
    name="内容概括",
    prompt="{input}请用一句话概括以下内容",
    type_="generate",
    description="用一句话概括输入文本",
    temperature=0.5,
    max_length=128,
    repetition_penalty=1.1,
)
toolbox.register(skill)

result = toolbox.run("内容概括", "今天天气真好，阳光明媚。")
print(result["text"])

3.2 Text Classification (classify)

Default params for classify type: temperature=0.001, max_length=128, repetition_penalty=1.0

skill = TinySkill(
    id_="spam_detect",
    name="垃圾邮件检测",
    prompt="{input}判断以下内容是否为垃圾邮件，给出分析过程",  # Chinese prompt only
    type_="classify",
    keywords=[
        {"name": "垃圾邮件", "id": "0", "key": ["垃圾", "诈骗", "中奖", "点击链接"]},
    ],
    other={"id": "1", "name": "正常邮件"},
    description="判断内容是否为垃圾邮件",
    temperature=0.001,
    repetition_penalty=1.0,
)
toolbox.register(skill)

result = toolbox.run("spam_detect", "恭喜您中奖了！点击领取奖品")
print(f"Probabilities: {result['probabilities']}")  # {'0': 0.XX, '1': 0.XX}
print(f"Prediction: {result['prediction']['name']}")

4. Classification Logic

Classify-type skills use keyword hit probability:

1. Call the model with an extremely low temperature (0.001) and repetition_penalty=1.0 for deterministic output
2. Count keyword occurrences for each category: text.count(keyword)
3. Probability = category hit count / total hit count
4. If total hits = 0, falls back to the other category (probability = 1.0)

Example: Classifier definition:

• Category "0" (广告推销): keywords ["广告", "营销", "推销"]
• Category "1" (生活通知): other category

Model output: "这条短信涉嫌营销推广，属于广告宣传。" Hit stats: "广告"×1 + "营销"×1 + "推销"×0 = 2 hits Probability: {"0": 1.0, "1": 0.0} → Classified as 广告推销.

5. Built-in TinySkills

ID	Name	Type	Prompt	Default Params	Description
summarize_word	词语概括	generate	{input}请你用一个词语概括内容	temp=0.5, maxlen=64, rp=1.1	Summarize with one word
summarize_short	短词概括	generate	{input}请你用简短的词语概括内容	temp=0.5, maxlen=64, rp=1.1	Summarize with short words
generate_title	题目生成	generate	{input}为这段文本生成题目	temp=0.5, maxlen=64, rp=1.1	Generate a title
generate_poem	古诗生成	generate	{input}为以上内容生成一段优美古诗	temp=0.5, maxlen=64, rp=1.1	Generate a classical poem
generate_philosophy	哲理文本	generate	{input}为以上内容生成一段富有哲理的文本	temp=0.5, maxlen=64, rp=1.1	Generate philosophical text
generate_jueju	绝句生成	generate	关于"{input}"请生成一首绝句	temp=0.5, maxlen=64, rp=1.1	Generate a jueju poem
sms_classify	短信分类	classify	{input}请你分析这段短信是广告推销还是生活通知？给出推理过程	temp=0.001, rp=1.0	Classify ads vs. notifications. Keywords: 广告、营销、推销、销、宣传; other: 生活通知
emotion_analysis	情感分析	classify	{input}分析以上文本的情感倾向	temp=0.001, rp=1.0	Analyze positive/negative/neutral. Positive keys: 正、积极、好、愉快、开心、喜欢; Negative keys: 反、消极、坏、差、烂、不、讨厌; other: 中性

6. Custom Registration Example

# Custom classification: Feedback type analysis
feedback_skill = TinySkill(
    id_="feedback_classify",
    name="反馈分类",
    prompt="{input}分析以下用户反馈的类型",  # Chinese only
    type_="classify",
    keywords=[
        {"name": "投诉", "id": "0", "key": ["投诉", "不满", "差评", "失望"]},
        {"name": "建议", "id": "1", "key": ["建议", "希望", "改进", "优化"]},
        {"name": "咨询", "id": "2", "key": ["请问", "怎么", "如何"]},
    ],
    other={"id": "-1", "name": "其他"},
    description="分析用户反馈类型（投诉/建议/咨询/其他）",
)
toolbox.register(feedback_skill)

7. Parameter Priority

Priority: run(**kwargs) > TinySkill defaults > Toolbox internal defaults

Toolbox internal defaults:

Type	temperature	max_length	repetition_penalty
generate	0.5	64	1.1
classify	0.001	128	1.0

result = toolbox.run(
    "古诗生成",
    "秋天景色",
    temperature=0.9,           # Overrides built-in 0.5
    max_length=256,            # Overrides built-in 64
    repetition_penalty=1.05,   # Overrides built-in 1.1
    top_p=0.9,
    stop_conditions=["<", "\n", "。"]  # Custom stop conditions
)

---

ImgTok Vision Support (v1.4.3 NEW)

The xiaothink.llm.xiaothink_img module provides image-text bridging for vision-language models:

from xiaothink.llm.xiaothink_img import ImgProcessor

# Initialize processor (model is lazily loaded)
img_proc = ImgProcessor(
    model_path='path/to/phase2_best.pth',  # imgzip checkpoint
    device='cuda',
    n_char=512  # tokens per image
)

# Input: Replace <imgtoken> tags with ImgTok tokens
text_with_images = "<|U|><imgtoken>cat.jpg</imgtoken><|A|>a cat sitting<|E|>"
processed = img_proc.preprocess_input(text_with_images)
# → "<|U|>ImgTok_107ImgTok_373ImgTok_630...<|A|>a cat sitting<|E|>"

# Output: Reconstruct images from ImgTok tokens
model_output = "ImgTok_107ImgTok_373...a cat sitting"
processed = img_proc.postprocess_output(model_output)
# → "<imgtoken>/tmp/xiaothink_img/img_xxx.png</imgtoken>a cat sitting"

# Clean up
img_proc.unload()

How It Works:

• Encoding: Image (80×80) → imgzip compressor → 512 int8 values → ImgTok_{id} tokens
• Decoding: ImgTok_{id} tokens → 512 int8 values → imgzip decompressor → 80×80 RGB image
• Token Mapping: Token ID = position × 255 + (value + 127) + 1
  • Position 0-511, Value -127~127 → 255 possible values per position
  • Total: 512 × 255 = 130,560 unique image tokens

Supported Models for Vision:

Model	MT Parameter	Architecture	Vocab Size
T17-Tiny-Vision	t17_tiny_vision	GRU3 + History Retrieval	192,894 (52,894 text + 140,000 ImgTok)

Auto-Integration: When using TorchModel with t17_tiny_vision, <imgtoken> tags in input prompts are automatically processed, and consecutive ImgTok tokens in output are automatically reconstructed to images.

---

Image Compression 2nd Generation (img_zip2, v1.4.5 NEW)

xiaothink.llm.img_zip2 is the second generation PyTorch-based image compression module (successor to the TensorFlow-based img_zip). It compresses 80×80 images into 512 int8 integers.

from xiaothink.llm.img_zip2 import (
    ImageCompressor, load_checkpoint, MODEL_REGISTRY
)

# Load model (must specify mt and checkpoint path)
model, ckpt = load_checkpoint(
    'path/to/phase2_best.pth',
    device='cuda',
    mt='test_vmof_2'  # model type
)
model.eval()

# Compress: (1,3,80,80) tensor → 512 int8 values
ints = model.compress(image_tensor)

# Decompress: 512 int8 values → (1,3,80,80) tensor
recon = model.decompress(ints, device='cuda')

Command-line Interface:

# Full pipeline: image → .jx → reconstructed image
python -m xiaothink.llm.img_zip2.infer input.jpg --ckpt model.pth

# Compress only: image → .jx file (no reconstruction)
python -m xiaothink.llm.img_zip2.infer input.jpg --ckpt model.pth --no_recon

# Decompress only: .jx file → reconstructed image
python -m xiaothink.llm.img_zip2.infer input.jx --ckpt model.pth

Parameters:

• input: Input image path (.jpg/.png/etc.) or .jx file path
• --ckpt: Model checkpoint path (.pth, required)
• --mt: Model type (auto-detected from checkpoint if not specified)
• --save_dir: Output directory
• --device: Device (cuda/cpu, default: cuda)
• --no_recon: Compress only, skip reconstruction

Interactive TUI:

python -m xiaothink.llm.img_zip2.tui

Launches an interactive menu with the following options:

1. Compress image → .jx file
2. Decompress .jx → reconstructed image
3. Full pipeline (compress + reconstruct + PSNR)
4. Image → ImgTok token sequence
5. ImgTok token → reconstructed image
6. View model info
7. Exit

Adjustable BPP Compression (v1.4.6 NEW):

Control bits-per-pixel by scaling the input image — no model modification needed:

# Specify scale factor (larger scale = higher bpp = better quality)
python -m xiaothink.llm.img_zip2.infer input.jpg --ckpt model.pth --scale 0.5

# TUI option 1 "Compress image" provides interactive scale selection
python -m xiaothink.llm.img_zip2.tui

from xiaothink.llm.img_zip2 import SCALES, calc_bpp, infer_one

# 7 scales: [0.25, 0.35, 0.5, 0.7, 1.0, 1.4, 2.0]
for scale in SCALES:
    jx_path, recon = infer_one('photo.jpg', 'model.pth', scale=scale)

# bpp = jx_bytes × 8 / original_pixels
bpp = calc_bpp(os.path.getsize('photo.jx'), W_orig, H_orig)

Principle: input scaling → patch count changes → total bytes change → bpp changes (each 80×80 patch compresses to 512 ints → LZMA, fixed internally)

scale	Input (e.g. 2048×1365)	Patches	Est. bpp
0.25x	512×341	~35	~0.08
0.50x	1024×682	~117	~0.15
1.00x	2048×1365	~442	~0.27
2.00x	4096×2730	~1820	~0.55

Note: img2llm (image-to-token) is not affected — it always converts to fixed 80×80 → 512 ints.

Key Differences from 1st Generation:

Feature	img_zip (1st Gen)	img_zip2 (2nd Gen)
Framework	TensorFlow/Keras	PyTorch
Model Format	.keras	.pth
Architecture	Single Autoencoder	Triple-Expert + PostProcessor (>100M params)
MT Registry	None	MODEL_REGISTRY dynamic routing
Loading	ImgZip(model_path)	load_checkpoint(path, device, mt)

For LLM Integration: Use ImgProcessor with mt parameter:

from xiaothink.llm.xiaothink_img import ImgProcessor

img_proc = ImgProcessor(
    model_path='path/to/phase2_best.pth',
    device='cuda',
    mt='test_vmof_2'  # explicitly specify model type
)

---

OpenAI-Compatible Server & Web Chat (v1.4.7 New)

The xiaothink.llm.inference_torch.openai_server module provides an OpenAI-format compatible API server with streaming and non-streaming chat completion.

Starting the Server

from xiaothink.llm.inference_torch.torch_formal import TorchModel
from xiaothink.llm.inference_torch.openai_server import OpenAIServer

# Load model
model = TorchModel(ckpt_dir='path/to/model', MT='t17_tiny')

# Start server (without web UI)
server = OpenAIServer(model, host='127.0.0.1', port=8000)
server.run()

Enable Web Chat UI

# Add use_gui=True to enable the built-in chat page
server = OpenAIServer(model, host='127.0.0.1', port=8000, use_gui=True)
server.run()
# Visit http://127.0.0.1:8000/ to start chatting

The web UI supports: parameter tuning (temperature/top_p/max_tokens/repetition_penalty), chat/continuation mode switching, and clear history.

API Key Authentication

server = OpenAIServer(model, host='0.0.0.0', port=8000, api_key='sk-your-key')
server.run()

Calling via OpenAI Client

from openai import OpenAI

client = OpenAI(
    base_url='http://127.0.0.1:8000/v1',
    api_key='sk-your-key'  # Any value works if api_key is not configured
)

# Streaming chat
stream = client.chat.completions.create(
    model='xiaothink',
    messages=[{'role': 'user', 'content': 'Hello'}],
    temperature=0.85,
    top_p=0.9,
    max_tokens=512,
    stream=True
)
for chunk in stream:
    print(chunk.choices[0].delta.content or '', end='')

# Non-streaming chat
response = client.chat.completions.create(
    model='xiaothink',
    messages=[{'role': 'user', 'content': 'Hello'}],
    stream=False
)
print(response.choices[0].message.content)

---

Xiaothink framework series model names, their corresponding MT (model architecture version), and form (model prompt input format) list:

Model Name (by release time)	mt parameter	form parameter
Xiaothink-T17-Tiny	mt='t17_tiny'	No form parameter needed
Xiaothink-T17-Tiny-Vision	mt='t17_tiny_vision'	No form parameter needed
Xiaothink-T17-MoE-2B	mt='t17_moe_2b_a0.15b'	No form parameter needed
Xiaothink-T7.5-0.1B	mt='t7.5_paddle_small_instruct'	form=2
Xiaothink-T7-ART(0.07B)	mt='t7_cpu_standard'	form=1
Xiaothink-T6-0.08B	mt='t6_beta_dense'	form=1
Xiaothink-T6-0.15B	mt='t6_standard'	form=1
Xiaothink-T6-0.02B	mt='t6_fast'	form=1
Xiaothink-T6-0.5B	mt='t6_large'	form=1
Xiaothink-T6-0.5B-pretrain	mt='t6_large'	form='pretrain'

---

Changelog

Version 1.4.7 (2026-07-16)

• New Feature - OpenAI-Compatible Server & Web Chat:
  • Added xiaothink.llm.inference_torch.openai_server module
  • OpenAIServer(model, host, port) — launches an OpenAI-compatible API server
  • POST /v1/chat/completions — supports temperature, top_p, max_tokens, stream parameters
  • GET /v1/models — lists available models
  • use_gui=True enables a web chat interface with parameter tuning, chat/continuation mode switching, and clear history
  • SSE streaming output with 5-token buffer mechanism to prevent stop marker leakage
  • Messages → prompt conversion based on Xiaothink history format (<|U|>/<|A|>)
  • Supports Bearer Token API key authentication (api_key parameter)

Version 1.4.6 (2026-06-29)

• New Feature - Adjustable BPP Compression:
  • Control bpp via input image scaling: infer_one / CLI now support --scale parameter
  • 7 preset scales SCALES = [0.25, 0.35, 0.5, 0.7, 1.0, 1.4, 2.0]
  • New calc_bpp(jx_bytes, w, h) helper for bits-per-pixel calculation
  • bpp = jx_bytes × 8 / original pixels; smaller scale = lower bpp
  • TUI compress option provides interactive scale selection and custom scale factor
  • Exported SCALES, calc_bpp to xiaothink.llm.img_zip2
  • Note: img2llm (image-to-token) not affected — always 80×80 → 512 ints

Version 1.4.5 (2026-06-29)

• New Module - img_zip2 (2nd Generation Image Compression):
  • Added xiaothink.llm.img_zip2 module: PyTorch-based extreme image compression (80×80 ↔ 512 int8)
  • Triple-Expert Spatial Autoencoder: DN (Deep-Narrow) + SW (Shallow-Wide) + ViT encoder with soft-routing gate
  • Iterative PostProcessor with diffusion-style step embedding for quality enhancement (>100M params total)
  • MODEL_REGISTRY MT mechanism: dynamically routes to encoder/decoder architectures based on model type
  • Sub-modules: img_compressor (core model + training), img2llm (image ↔ LLM token bridge), infer (arbitrary-size image patching + .jx format)
  • load_checkpoint(path, device, mt) auto-detects MT from checkpoint, supports explicit mt override
• Refactored xiaothink_img.py:
  • Now imports from xiaothink.llm.img_zip2 instead of external v16/imgzip2
  • Added mt parameter to ImgProcessor and get_img_model() for explicit model type specification
  • Dynamic LATENT_DIM/INT_SCALE/NUM_VALUES read from loaded model instead of hardcoded constants
  • model_path is now required (no longer falls back to CONFIG['pretrained'])
• Compatibility: Existing TorchModel/TorchNovelModel APIs unchanged — they accept imgzip_checkpoint and internally create ImgProcessor

Version 1.4.4 (2026-06-28)

• New Class - TorchNovelModel:
  • Added xiaothink.llm.inference_torch.torch_novel.TorchNovelModel for novel role-playing
  • Inherits from TorchModel with novel-specific dialogue features
  • set_model_name(): set model role name(s), supports multiple roles separated by ; (e.g., '顾言;林婉')
  • set_user_name(): set user role name
  • set_setting(): set story background
  • chat_auto(): auto-format dialogue — automatically prepends {user_name}："{text}"\n to prompt
  • chat(): manual-format dialogue — appends user input as-is
  • Multi-role auto-completion: if model outputs the first character of a role name (e.g., '顾'), automatically completes the full name ('言')
  • Stops at newlines or four spaces (not preserved), or " (preserved)
  • Full ImgTok vision support with explicit imgzip_checkpoint parameter
• API Enhancement - TorchModel:
  • Added imgzip_checkpoint parameter to TorchModel.__init__() for explicit imgzip model path
  • Image token processing handled at TorchModel/TorchNovelModel level, not in low-level torch_test.py
• Bug Fixes:
  • Fixed tensor size mismatch error when running t17_tiny_vision inference: t17_tiny_vision was missing from RNN_FAMILY_NAMES, causing full-sequence mode with overflowed history_tensor
  • Fixed max_history_len too small for inference sequences (was 512 // save_every = 2, now minimum 64 with dynamic expansion)

Version 1.4.3 (2026-06-19)

• New Module - ImgTok Vision Support:
  • Added xiaothink.llm.xiaothink_img module for image-text bridging
  • ImgProcessor class: unified image ↔ ImgTok token conversion
  • Input preprocessing: <imgtoken>image_path</imgtoken> tags auto-replaced with ImgTok_{id} sequences via imgzip compressor (80×80 → 512 int8 → ImgTok tokens)
  • Output postprocessing: consecutive ImgTok_{id} sequences (≥512) auto-reconstructed to images via imgzip decoder, replaced with <imgtoken>temp_path</imgtoken> tags
  • Lazy-loading singleton model for efficient reuse
  • Text processing functions: replace_imgtoken_tags(), replace_imgtok_with_tags(), extract_imgtok_groups()
• New Model:
  • Added t17_tiny_vision model: GRU3 + history retrieval architecture for vision-language tasks. Config: d_model=768, num_heads=8, num_layers=12, history_top_k=8. Checkpoint dir: ckpt_test_t17_tiny_vision
• New Vocabulary:
  • Added vocab_img.json: original 52,894 tokens + 140,000 ImgTok tokens (ImgTok_1 ~ ImgTok_140000), total 192,894 tokens
  • Token mapping: Token ID = position × 255 + (value + 127) + 1, covering 512 positions × 255 values = 130,560 unique image tokens
  • WordTokenizer.tokenize() now handles ImgTok_ prefix with specialized fast path
• Training Improvements:
  • Added CPU Offload mechanism for optimizer states (USE_CPU_OFFLOAD) to reduce GPU memory
  • Added Inexact Autoregression training mode (USE_INEXACT_AR, INEXACT_AR_CUT_EVERY): predict only every N-th position, reducing llm_head compute and logits memory
  • Optimizer state offload: move Adam exp_avg/exp_avg_sq to CPU between steps
• Model Loading:
  • Embedding expansion: when loading checkpoints with smaller vocabulary, preserve pre-trained weights and randomly initialize new tokens
  • Support for STRICT_LOAD=False mode with smart vocabulary expansion
• Files Updated:
  • torch_test.py: integrated ImgTok input preprocessing and output postprocessing
  • torch_train.py: added <imgtoken> tag processing in _create_encoded_cache()
  • torch_models.py: added t17_tiny_vision config and model creation branch
  • All training files updated to use vocab_img.json by default
  • setup.py requires Pillow>=9.0.0 for image processing

Version 1.4.2 (2026-06-10)

• New Module:
  • Added xiaothink.llm.inference_torch module for PyTorch-based inference
  • Supports Xiaothink-T17 series models (GRU/LSTM architecture + history retrieval mechanism)
  • Provides TorchModel class for PyTorch inference
  • New TinySkill Toolbox: xiaothink.llm.tools.tiny_skill with skill registration and execution (generate/classify types), 8 built-in skills (word summary, short summary, title generation, poem generation, philosophical text, jueju poem, SMS classification, emotion analysis)
• Model Features:
  • Innovative history retrieval mechanism: retrieves relevant historical context during generation
  • Gate fusion: learnable gate mechanism for blending RNN output with retrieved history
  • Causal masking: ensures no future information leakage during training and inference
  • Efficient inference: optimized for on-device deployment with low memory footprint
• Supported MT Architectures:
  • 't17_tiny': GRU-based model with history retrieval
• AI Rate Detection:
  • Added PyTorch backend support for AI rate detection
  • Automatic backend selection based on model_type
  • New Calibration Feature: calibrate() and auto_calibrate() methods for automatic threshold optimization using AI/human text samples
  • Manual Threshold Adjustment: set_thresholds(high, low) method for custom thresholds
  • New threshold_high and threshold_low parameters in __init__
  • Detection results now include Threshold Configuration field with current thresholds and calibration status
  • Import Order Fix: Reordered backend imports (PyTorch > PaddlePaddle > TensorFlow) to prevent numpy 2.0 TensorFlow crash from contaminating other backends
  • New XIAOTHINK_BACKEND Env Var: Set to torch/paddle/tensorflow/auto to import only the specified backend, completely skipping others to avoid compatibility issues

Version 1.4.1 (2026-02-16)

• New Module:
  • Added xiaothink.llm.inference_paddle module for PaddlePaddle-based inference
  • Supports Xiaothink-T7.5 series models with RWKV architecture
  • Provides TextGenerator and QianyanModel classes for PaddlePaddle
• Updated Dependencies:
  • Removed TensorFlow as a required dependency (now optional)
  • Added PaddlePaddle as the primary deep learning framework
  • Added jieba for Chinese word segmentation
• Model Support:
  • Added support for MT architectures: 't7.5_paddle_small_instruct', 't7.5_paddle_small_instruct_pro', etc.
  • Supports automatic device selection (CPU/GPU) based on memory usage

Version 1.4.0 (2026-02-16)[Yanked]

• Note: This version has been yanked due to issues with the README.md file content. Please use the updated version instead.

Version 1.3.2 (2025-12-27)

• Updated Interfaces:
  • Added "AI Rate Detection" interface based on Xiaothink-T series models.
• New Models:
  • Added support for MT architectures "t7" and "t7_cpu_standard" in the Xiaothink-T7 series models.

Version 1.3.1 (2025-10-31)

• Updated Interfaces:
  • Added custom input shape (must be supported by the corresponding model) for vision-related interfaces instead of the fixed 80803 in previous versions
  • The ImgZIP command-line interface also added custom input shape (must be supported by the corresponding model) instead of the fixed 80803 in previous versions, and added comprehensive quality scores based on SNR, PSNR, and SSIM.

Version 1.3.0 (2025-10-17)[Yanked]

• New Models:
  • Added support for the Xiaothink-T7 series model architecture.

Version 1.2.5 (2025-09-02)

• Updated Interfaces:
  • Added "custom compression rate" function to the ImgZIP command-line interface, supporting other compression rates beyond the model's native compression rate (implemented based on calculating and scaling the original image).

Version 1.2.4 (2025-08-30)

• Updated Interfaces:
  • Updated the import method of ImgZIP-related interfaces in the documentation to: from xiaothink.llm.img_zip.img_zip import ImgZip

Version 1.2.3 (2025-08-30)

• New Features:
  • Added Xiaothink-T6-0.02B series models (MT='t6_fast')
  • Added Xiaothink-T6-0.5B series models (MT='t6_large')
  • Added support for form='pretrain' in the model.chat method. For instruction-tuned models in the T6 series, form=1 should be used; for pre-trained models, form='pretrain' should be used

Version 1.2.2 (2025-08-18)

• New Features:
  • Added sentiment classification tool to implement text sentiment tendency analysis through ClassifyModel
  • Added xiaothink.llm.tools.classify module to support sentiment classification based on basic dialogue models
  • Provided cmodel.emotion(inp) interface to return real-time text sentiment results

Version 1.2.1 (2025-08-16)

• New Models:
  • Added Xiaothink-T6-0.15B series models (MT='t6_standard')

Version 1.2.0 (2025-08-08)

• Breakthrough Innovation:

  • Added support for native vision models using an innovative dual-vision solution
  • Dual-path processing of image compression to feature tokens (img_zip) + native vision encoder
  • Retains multi-image context understanding capability while achieving single-image detail analysis

• New Interfaces:

  • model.chat_vision: Specialized dialogue interface for vision models
  • model.img2ms: Image description interface for non-native vision models
  • model.img2ms_vision: Image description interface for native vision models (supports max_shape parameter)

• Module Expansion:

  • Added xiaothink.llm.img_zip.img_zip command-line tool
  • Supports compression and decompression of images and videos
  • Provides rich parameters to adjust compression quality

• Usage Guidelines:

  • Vision models must use the chat_vision method
  • Must use a matching img_zip encoder model
  • Image paths should use absolute paths

Version 1.1.0 (2025-08-02)

• New Features:

  • Added img2ms and ms2img interfaces to achieve high compression ratio lossy compression of images
  • Supports converting images into AI-readable feature tokens
  • Extended dialogue models to support multimodal input (image + text)
  • In test_formal, it supports converting feature tokens generated by multimodal AI into images and saving them to the system temporary folder by default.

• Technical Upgrades:

  • Based on Xiaothink framework's self-developed img_zip technology
  • Supports intelligent compression of 80x80x3 image patches
  • When outputting 96 feature values, combined with .7z algorithm, it can achieve an ultra-high compression ratio of 10%

• Usage Method:

  • Insert images using the <img>{image_path}</img> tag in dialogue
  • Need to specify the img_zip model path when initializing the model
  • Supports multimodal dialogue (image description, image Q&A, and other scenarios)

---

The above covers the main functions and usage methods of the Xiaothink Python module.

If you have any questions or suggestions, please feel free to contact us: [e-mail].