Metadata-Version: 2.4
Name: xiaothink
Version: 1.4.2
Summary: An AI toolkit that helps users quickly call interfaces related to the Xiaothink framework.
Author: Shi Jingqi
Author-email: xiaothink@foxmail.com
License: Apache License 2.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: numpy>=1.21.0
Requires-Dist: requests>=2.25.0
Provides-Extra: tensorflow
Requires-Dist: tensorflow>=2.10.0; extra == "tensorflow"
Provides-Extra: paddle
Requires-Dist: paddlepaddle>=2.5.0; extra == "paddle"
Requires-Dist: jieba>=0.42.1; extra == "paddle"
Provides-Extra: paddle-gpu
Requires-Dist: paddlepaddle-gpu>=2.5.0; extra == "paddle-gpu"
Requires-Dist: jieba>=0.42.1; extra == "paddle-gpu"
Provides-Extra: torch
Requires-Dist: torch>=2.0.0; extra == "torch"
Provides-Extra: vision
Requires-Dist: Pillow>=9.0.0; extra == "vision"
Provides-Extra: all
Requires-Dist: tensorflow>=2.10.0; extra == "all"
Requires-Dist: paddlepaddle>=2.5.0; extra == "all"
Requires-Dist: jieba>=0.42.1; extra == "all"
Requires-Dist: torch>=2.0.0; extra == "all"
Requires-Dist: Pillow>=9.0.0; extra == "all"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Xiaothink Python Module Usage Documentation

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
Xiaothink is an AI research organization focused on Natural Language Processing (NLP), dedicated to training advanced on-device models with limited data and computing resources. The Xiaothink Python module is our core toolkit, covering various functions such as text-based Q&A, multimodal Q&A, image compression, sentiment classification, and more. Below is the detailed usage guide and code examples.

## Table of Contents
1. [Installation](#installation)
2. [License](#license)
3. [Local Dialogue Models](#local-dialogue-models)
4. [PaddlePaddle-based Models](#paddlepaddle-based-models-v140-new)
5. [PyTorch-based Models (T17)](#pytorch-based-models-t17-v142-new)
6. [Image Feature Extraction and Multimodal Dialogue](#image-feature-extraction-and-multimodal-dialogue)
7. [Image Compression to Feature Technology (img_zip)](#image-compression-to-feature-technology-img_zip)
8. [Sentiment Classification Tool](#sentiment-classification-tool)
9. [AI Rate Detection Tool](#ai-rate-detection-tool)
10. [TinySkill Toolbox](#tinyskill-toolbox-v142-new)
11. [Model Name Reference Table](#-xiaothink-model-names)
12. [Changelog](#changelog)

---

## Installation

First, you need to install the Xiaothink module via pip:

```bash
pip install xiaothink          # Basic install (no deep learning backend)
pip install xiaothink[torch]   # With PyTorch (recommended for T17)
pip install xiaothink[all]     # With all backends
```

### Backend Control via Environment Variable (v1.4.2 New)

Set the `XIAOTHINK_BACKEND` environment variable to control which deep learning backend the library imports. This prevents compatibility issues from unused backends (e.g., numpy 2.0 breaking TensorFlow).

**Usage** (set before running your Python script):

| Command | Effect |
|---------|--------|
| `set XIAOTHINK_BACKEND=torch` (Windows) | Import PyTorch only (recommended for T17 users) |
| `set XIAOTHINK_BACKEND=paddle` (Windows) | Import PaddlePaddle only (recommended for T7.5 users) |
| `set XIAOTHINK_BACKEND=tensorflow` (Windows) | Import TensorFlow only (recommended for T7 and below) |
| Not set or `auto` | Try all backends (default) |

**Example (Windows PowerShell)**:
```powershell
# Use PyTorch only, completely skip TensorFlow and PaddlePaddle
$env:XIAOTHINK_BACKEND='torch'
python your_script.py
```

**Example (Linux/Mac)**:
```bash
# Use PyTorch only, completely skip TensorFlow and PaddlePaddle
export XIAOTHINK_BACKEND=torch
python your_script.py
```

> **Tip**: If you have multiple deep learning frameworks installed but only use one, set this environment variable to speed up imports and avoid compatibility issues.

---

## License

This project is licensed under the Apache License, Version 2.0 - see the [LICENSE](LICENSE) file for details.

The [NOTICE](NOTICE) file contains additional attribution information for the proprietary technologies included in this module.

---

## Local Text-only Dialogue Models

For locally loaded dialogue models, you should call the corresponding function according to the model type.

### Single-turn Dialogue (to be removed in future versions)

Suitable for single-turn dialogue scenarios.

### Example Code

```python
import xiaothink.llm.inference.test_formal as tf

model = tf.QianyanModel(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'# vocab file is provided in the model repository
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat_SingleTurn(inp, temp=0.32)  # Use chat_SingleTurn for single-turn dialogue
    print('\n[A]:', re, '\n')
```

### Multi-turn Dialogue

Suitable for multi-turn dialogue scenarios.

### Example Code

```python
import xiaothink.llm.inference.test_formal as tf

model = tf.QianyanModel(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'# vocab file is provided in the model repository
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temp=0.32)  # Use chat for multi-turn dialogue
    print('\n[A]:', re, '\n')
```

### Manually Adding History

Use the `add_his` method to manually add historical dialogue records, suitable for presetting dialogue context.

```python
import xiaothink.llm.inference.test_formal as tf

model = tf.QianyanModel(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'
)

# Manually add history
model.add_his('What is your name?', 'My name is "Xiaosi", nice to meet you!')
model.add_his('Hello, Xiaosi', 'Hello to you too!')

# Subsequent dialogue will be based on the added history context
while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temp=0.32)
    print('\n[A]:', re, '\n')
```

**Note**:
- `add_his(q, a, form)` method accepts three parameters: question `q`, answer `a`, and format `form`
- T6 series models use `form=1` (instruction-tuned models) by default, pre-trained models use `form='pretrain'`
- T7.5 series models use `form=2`
- T17 series models (PyTorch) do not require the `form` parameter

### Text Continuation

Suitable for more flexible text continuation scenarios.

### Example Code

```python
import xiaothink.llm.inference.test as test

MT = 't6_beta_dense'
m, d = test.load(
    ckpt_dir=r'path/to/your/t6_model',
    MT='t6_beta_dense',
    vocab=r'path/to/your/vocab'# vocab file is provided in the model repository
)

inp='Hello!'
belle_chat = '{"conversations": [{"role": "user", "content": {inp}}, {"role": "assistant", "content": "'.replace('{inp}', inp)    # Instruct format supported by instruction-tuned models in the T6 series
inp_m = belle_chat

ret = test.generate_texts_loop(m, d, inp_m,    
                               num_generate=100,
                               every=lambda a: print(a, end='', flush=True),
                               temperature=0.32,
                               pass_char=['▩'])    # ▩ is the <unk> token for T6 series models
```

**Important Note**: For local models, it is recommended to use the `model.chat` function for multi-turn dialogue. For pre-trained models without instruction tuning, it is recommended to use the `test.generate_texts_loop` function. **The single-turn dialogue function `model.chat_SingleTurn` will be removed in future versions.**

---

## PaddlePaddle-based Models (New in v1.4.0)

Xiaothink now supports PaddlePaddle-based models with RWKV architecture. These models offer efficient inference and are suitable for resource-constrained environments.

### Installation

```bash
pip install xiaothink
pip install paddlepaddle  # or paddlepaddle-gpu for GPU support
```

### Multi-turn Dialogue (PaddlePaddle)

```python
from xiaothink.llm.inference_paddle import QianyanModel

model = QianyanModel(
    ckpt_dir=r'path/to/your/t7.5_model',
    MT='t7.5_paddle_small_instruct_pro'
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temp=0.34, form=2)  # form=2 for simplified format
    print('\n[A]:', re, '\n')
```

### Text Generation (PaddlePaddle)

```python
from xiaothink.llm.inference_paddle import TextGenerator

generator = TextGenerator(
    checkpoint_path=r'path/to/your/t7.5_model',
    MT='t7.5_paddle_small_instruct_pro'
)
generator.load_model()

generated_text = generator.generate_text(
    prompt='<|U|>Hello, how are you?<|A|>',
    max_length=100,
    temperature=0.8,
    top_p=0.9,
    repetition_penalty=1.2
)
print(generated_text)
```

### Supported Model Architectures (PaddlePaddle)

| Model Name | MT Parameter | Description | form Parameter |
|------------|--------------|-------------|----------------|
| Xiaothink-T7.5-0.1B | 't7.5_paddle_small_instruct' | Base instruction model | form=2 |
| Xiaothink-T7.5-0.1B-Pro | 't7.5_paddle_small_instruct_pro' | Enhanced instruction model | form=2 |
| Xiaothink-T7.5-0.1B-Thinking | 't7.5_paddle_small_instruct_thinking' | Chain-of-thought model | form=2 |
| Xiaothink-T7.5-0.1B-Poem | 't7.5_paddle_small_instruct_poem' | Poetry generation model | form=2 |
| Xiaothink-T7.5-0.1B-Translator | 't7.5_paddle_small_instruct_translator' | Translation model | form=2 |

### Automatic Device Selection

The PaddlePaddle module automatically selects the optimal device based on GPU memory usage:

```python
# Automatic device selection (default)
# If GPU memory usage > 80%, switches to CPU
AUTO_DEVICE = True
GPU_MEMORY_THRESHOLD = 80.0

# Or manually set device
import paddle
paddle.set_device('cpu')  # or 'gpu:0'
```

### Train-On-Time (TOT) Dynamic Learning

Xiaothink provides an innovative Train-On-Time (TOT) feature that enables dynamic learning during inference. Unlike traditional models that only use pre-trained knowledge, TOT models continuously learn from similar examples in your training data repository. This technology can significantly enhance the model's basic capabilities. However, it's important to note that TOT models' inference speed will be somewhat affected, as it needs to perform similarity matching and fine-tuning before each conversation. Additionally, this technology cannot significantly improve the model's mathematical and reasoning abilities, and can only be used to enhance basic model capabilities.

**How TOT Works:**
1. **Similarity Matching**: Automatically finds similar instructions from your training data
2. **Dynamic Fine-tuning**: Fine-tunes the model in memory using these similar examples
3. **Enhanced Response**: Generates more accurate answers based on the newly learned knowledge
4. **Memory Management**: Optimizes GPU memory usage and cleans up resources

**Key Features:**
- **Real-time Learning**: Every conversation triggers learning from relevant examples
- **Similarity-based Matching**: Uses difflib to find semantically similar instructions
- **Parallel Processing**: Multi-core similarity calculation for faster matching
- **Multi-format Support**: Loads models from various checkpoint formats
- **GPU Optimization**: Automatic GPU memory management

```python
from xiaothink.llm.inference_paddle import TOTModel, TOT_AVAILABLE

if TOT_AVAILABLE:
    # 自定义训练数据路径
    custom_data_paths = [
        r'path/to/your/belle_train.jsonl',
        r'path/to/your/coig_minimind.jsonl',
        r'path/to/your/firefly_data.jsonl'
    ]
    
    model = TOTModel(
        ckpt_dir=r'path/to/your/t7.5_model',
        MT='t7.5_paddle_small_instruct_pro',
        data_paths=custom_data_paths  # 自定义训练数据路径
    )
    
    # The model will automatically learn from similar examples before answering
    while True:
        inp = input('[Q]: ')
        if inp == '[CLEAN]':
            model.clean_his()
            continue
        re = model.chat(inp, temp=0.68)
        print('\n[A]:', re, '\n')
else:
    print("TOT feature requires PaddlePaddle support")
```

**Custom Data Paths:**
You can now specify your own training data paths when initializing TOTModel:

```python
# Default behavior (uses built-in paths)
model = TOTModel(ckpt_dir='path/to/model')

# Custom data paths
model = TOTModel(
    ckpt_dir='path/to/model',
    data_paths=[
        'path/to/data1.jsonl',
        'path/to/data2.jsonl',
        'path/to/data3.txt'
    ]
)
```

**Training Data Requirements:**
The TOT system allows users to customize training datasets by passing them through the `data_paths` parameter. The system will load and process training data based on the paths provided by the user.

## Supported File Formats

The TOT system supports the following file formats:

- .jsonl files (JSON Lines format)
- .txt files (text files)

## Supported Data Structures
The system can identify and process two main data structures:

### 1. Conversations Format
```json
{
  "conversations": [
    {"content": "User question"},
    {"content": "Assistant answer"}
  ]
}
```

### 2. Instruction-Output Format
```json
{
  "instruction": "Instruction content",
  "output": "Output content"
}
```

**Memory Optimization:**
- Automatic GPU memory cleanup
- In-memory fine-tuning without disk writes
- Batch processing for efficient training

---

## PyTorch-based Models (T17, v1.4.2 New)

Xiaothink now supports PyTorch-based models with GRU/LSTM architecture and history retrieval mechanism. These models are designed for efficient on-device inference with innovative history-aware capabilities.

### Installation

```bash
pip install xiaothink
pip install torch  # PyTorch is required for T17 models
```

### Multi-turn Dialogue (PyTorch)

```python
from xiaothink.llm.inference_torch.torch_formal import TorchModel

model = TorchModel(
    ckpt_dir=r'path/to/your/t17_model',
    MT='t17_tiny'
)

while True:
    inp = input('[Q]: ')
    if inp == '[CLEAN]':
        print('[Context Cleared]\n\n')
        model.clean_his()
        continue
    re = model.chat(inp, temperature=0.7, top_p=0.9)
    print('\n[A]:', re, '\n')
```

### Text Generation (PyTorch)

```python
from xiaothink.llm.inference_torch.torch_test import TextGenerator

generator = TextGenerator(
    checkpoint_path=r'path/to/your/t17_model',
    MT='t17_tiny'
)
generator.load_model()

generated_text = generator.generate_text(
    prompt='<s>你好，请介绍一下自己。',
    max_length=128,
    temperature=0.7,
    top_p=0.9
)
print(generated_text)
```

### Supported Model Architectures (PyTorch)

| Model Name | MT Parameter | Description |
|------------|--------------|-------------|
| Xiaothink-T17-Tiny | 't17_tiny' | GRU-based model with history retrieval |

### Key Features

- **History Retrieval**: Innovative mechanism that retrieves relevant historical context during generation
- **Efficient Inference**: Optimized for on-device deployment with low memory footprint
- **Gate Fusion**: Learnable gate mechanism for blending RNN output with retrieved history
- **Causal Masking**: Ensures no future information leakage during training and inference

---

## Image Feature Extraction and Multimodal Dialogue

### Dual-vision Solution

In version 1.2.0, we introduced an innovative dual-vision solution:
1. **Image Compression to Feature (img_zip)**: Convert images to text tokens that can be inserted anywhere in the dialogue.
2. **Native Vision Encoder**: Pass the latest image to the native vision model's vision encoder (standard approach).

This solution achieves:
- Detailed analysis of the latest single image based on the native vision encoder
- Understanding of multiple images in the context based on img_zip technology
- Significant reduction in computing resource requirements

### Vision Model Usage Guidelines

For vision-enabled models, regardless of whether there is image input, you should use the following code:

```python
from xiaothink.llm.inference.test_formal import QianyanModel

if __name__ == '__main__':
    model = QianyanModel(
        ckpt_dir=r'path/to/your/vision_model',
        MT='t6_standard_vision',  # Note: model type is vision model
        vocab=r'path/to/your/vocab.txt',
        imgzip_model_path='path/to/img_zip/model.keras'  # Specify img_zip model path
    )

    temp = 0.28  # Temperature parameter
    
    while True:
        inp = input('[Q]: ')
        if inp == '[CLEAN]':
            print('[Context Cleared]\n\n')
            model.clean_his()
            continue
        # Use chat_vision for dialogue
        ret = model.chat_vision(inp, temp=temp, pre_text='', pass_start_char=[])
        print('\n[A]:', ret, '\n')
```

**Important Notes**:
- Vision models must use the `chat_vision` method; do not use `chat` (which is only for text-only models)
- You must prepare an img_zip image compression encoder model that matches the vision model
- Mismatched models will cause the model to fail to understand the meaning of encoded tokens

### Image Processing Interfaces

Two new image processing interfaces have been added:

1. **img2ms** (for non-native vision models):
   ```python
   description = model.img2ms('path/to/image.jpg', temp=0.28)
   print(description)
   ```

2. **img2ms_vision** (for native vision models):
   ```python
   description = model.img2ms_vision('path/to/image.jpg', temp=0.28, max_shape=224)
   print(description)
   ```

### Image Reference Syntax

In dialogue, use the following syntax to reference images:
```python
<img>image path or URL</img>Please describe this image
```

The model will automatically parse the image path, extract features, and answer based on the image content.

**Notes**:
1. Image paths should use absolute paths to ensure correct parsing
2. Native vision models only support analyzing the most recent image
3. img_zip technology supports referencing multiple images in the context

---

## Image Compression to Feature Technology (img_zip)

The `img_zip` module provides advanced image and video compression/decompression functions based on deep learning feature extraction technology. Below are the detailed usage methods:

### 1. Command-line Interactive Mode

```bash
python -m xiaothink.llm.img_zip.img_zip
```

After running, you will enter an interactive command-line interface:

```
===== img_zip Image Video Compression Tool =====
Please enter .keras model path: path/to/your/imgzip_model.keras
Model loaded successfully!

Please select a function:
1. Compress image
2. Decompress image
3. Compress video
4. Decompress video
0. Exit

Please select (0-4):
```

### 2. Python Code Invocation

```python
from xiaothink.llm.img_zip.img_zip import ImgZip

# Initialize instance
img_zip = ImgZip(model_path='path/to/your/imgzip_model.keras')

# Compress image
compressed_path = img_zip.compress_image(
    img_path='input.jpg',
    patch=True,  # Whether to use patch processing
    save_path='compressed_img',  # Save path prefix
    ability=0.02,# New feature in 1.2.5: Set custom compression rate to 0.02 (when ability is 0, it means not using custom compression rate). The algorithm calculates and compresses to a close size (there may be errors between theoretical calculation and actual size)
)

# Generates two files: compressed_img.npy and compressed_img.shape

# Decompress image
img_zip.decompress_image(
    compressed_input='compressed_img',  # Compressed file prefix
    patch=True,  # Whether to use patch processing
    save_path='decompressed.jpg'  # Output path
)

# Compress video
compressed_paths, metadata_path = img_zip.compress_video(
    video_path='input.mp4',
    output_dir='compressed_video',  # Output directory
    patch=True  # Whether to use patch processing
)

# Decompress video
img_zip.decompress_video(
    compressed_dir='compressed_video',  # Compressed file directory
    output_path='decompressed.mp4'  # Output path
)

# Convert image to array and save
img_array = img_zip.image_to_array('input.jpg')
img_zip.save_image_array(img_array, 'image_array.npy')

# Load image from array
loaded_array = img_zip.load_image_array('image_array.npy')
img = img_zip.array_to_image(loaded_array)
img.save('restored.jpg')
```

### 3. Key Function Descriptions

1. **Compress Image** (`compress_image`)
   - `patch=True`: Split large images into 80x80 patches for separate processing
   - Outputs two files: `.npy` (feature vectors) and `.shape` (original size information)

2. **Decompress Image** (`decompress_image`)
   - Requires both `.npy` and `.shape` files
   - Automatically restores original dimensions

3. **Video Processing** (`compress_video`/`decompress_video`)
   - Automatically extracts video frames and processes them in batches
   - Preserves original video frame rate and resolution information
   - Uses temporary directories for intermediate file processing

#### 4. Parameter Descriptions

| Parameter | Type | Description |
|-----------|------|-------------|
| `model_path` | str | Path to img_zip model (.keras file) |
| `patch` | bool | Whether to use patch processing (default: True) |
| `save_path` | str | Output file path prefix |
| `img_path` | str | Input image path |
| `video_path` | str | Input video path |
| `output_dir` | str | Output directory path |
| `output_path` | str | Output file path |

#### 5. Processing Flow Features

1. **Patch Processing**:
   - Automatically splits large images into 80x80 patches
   - Each patch is independently encoded into feature vectors
   - Preserves original size information

2. **Video Processing**:
   - Automatically extracts frames and processes them in batches
   - Preserves original video parameters (fps, resolution)
   - Uses temporary directories for intermediate file processing

3. **Progress Display**:
   - All operations come with detailed progress bars
   - Displays current processing step and remaining time

4. **Error Handling**:
   - Comprehensive exception catching mechanism
   - Detailed error information prompts

#### 6. Usage Recommendations

1. For images larger than 80x80, it is recommended to use patch processing (`patch=True`)
2. Video processing requires sufficient disk space for temporary frame files
3. Ensure the input model matches the processing task
4. Use absolute paths to avoid file location issues

This module is the core component of Xiaothink vision models (especially non-native ones). Based on efficient image feature representation and compression, it can enable any text-only AI model to have basic vision capabilities through fine-tuning.

---

## Sentiment Classification Tool

The sentiment classification tool is based on loaded dialogue models and provides text sentiment tendency analysis functionality, which can quickly determine the sentiment category of input text (e.g., positive, negative, neutral, etc.).

### Feature Description
- This tool is a customized interface based on Xiaothink framework (Xiaothink T6 series, etc.) models
- Implements sentiment classification based on Xiaothink framework language models without the need to load additional classification models
- Supports input of ultra-long text and returns sentiment analysis results
- It is recommended to use single-turn dialogue enhanced models, such as: Xiaothink-T6-0.15B-ST

### Usage Example

```python
from xiaothink.llm.inference.test_formal import *
from xiaothink.llm.tools.classify import *

if __name__ == '__main__':
    # Initialize basic dialogue model
    model = QianyanModel(
        ckpt_dir=r'path/to/your/t6_model',  # Model weight directory  It is recommended to use _ST version models
        MT='t6_standard',  # Model type (must match weights)
        vocab=r'path/to/your/vocab.txt',  # Vocabulary path
        use_patch=0  # Do not use patch processing (text-only model)
    )
    
    # Initialize sentiment classification model (depends on basic dialogue model)
    cmodel = ClassifyModel(model)
    
    # Loop input text for sentiment classification
    while True:
        inp = input('Enter text: ')
        res = cmodel.emotion(inp)  # Call sentiment classification interface
        print(res)  # Output sentiment analysis results
```

### Notes
1. The sentiment classification model depends on an initialized `QianyanModel`; ensure the base model is loaded successfully
2. It is recommended to use instruction-tuned models (e.g., `t6_standard`); non-tuned models may affect classification accuracy
3. The output result format is: {'Positive': 0.6667, 'Negative': 0.1667, 'Neutral': 0.1667}

---
## AI Rate Detection Tool
The AI rate detection tool is based on loaded detection models and provides text AI generation probability analysis functionality. It can accurately determine the AI generation probability of each character in the text, output the overall AI rate average, and return detailed character-level detection information, achieving comprehensive traceability analysis of text AI generation traces.

### Feature Description
- This tool is a customized interface based on Xiaothink framework (Xiaothink T series, etc.) models
- Implements text AI rate analysis based on Xiaothink framework detection models without the need to load independent detection models
- Supports ultra-long text detection and batch text detection, returning multi-dimensional complete detection results
- Can output **four levels of results: overall AI rate average, detection conclusion, probability statistics information, and character-level detailed information**

### Usage Example
```python
if __name__ == "__main__" and 1:
    # 1. Initialize detector
    detector = AIDetector(
        ckpt_dir=r'E:\Xiaothink Framework\Paper\ganskchat\ckpt_test_t7',
        model_type='t7',
        print_load_info=True
    )

    # 2. Detect text (Some models may not support Chinese text detection)
    test_texts = [
        "This is a sentence that a car repair blogger active on mobile internet used to start many of his videos before being sued by BYD. Finally, this 'most miserable repairman in history' has received the first-instance judgment of being sued by BYD.",
        "\"Isn't it,\" Grandma looked up at the osmanthus tree, her eyes filled with gentle memories, \"This was planted by your grandfather back then, almost thirty years ago. At that time, he said, planting an osmanthus tree, it will bloom in autumn, fragrant and beautiful, and when we have children, we can make osmanthus cake to eat.\"",
        "These days, my heart has been quite unsettled. Sitting in the yard enjoying the cool air tonight, I suddenly thought of the lotus pond I pass by every day. In this moonlight of the full moon, it should have a different appearance. The moon gradually rose higher, and the laughter of children on the road outside the wall could no longer be heard; my wife was patting Run'er inside the house, humming a lullaby drowsily. I quietly put on my large shirt and went out the door."
    ]

    # 3. Execute detection
    for text in test_texts:
        print(f"\n{'='*60}")
        print(f"Detected Text: {text}")
        result = detector.detect_ai_rate(text)
        
        print(f"AI Rate (Probability Average): {result['AI Rate (Probability Average)']}")
        print(f"Detection Conclusion: {result['Detection Conclusion']}")
        print(f"Probability Statistics: Min={result['Probability Statistics']['Minimum Probability']} | Max={result['Probability Statistics']['Maximum Probability']}")
        
        # Optional: Print character-level details
        print("\nCharacter-level Details:")
        for detail in result['Character-level Details']:
            print(f"  Position {detail['Character Position']}: Previous Text 「{detail['Complete Previous Text']}」→ Character 「{detail['Target Character']}」→ Probability {detail['Prediction Probability']}")

    # 4. Release resources
    detector.close()
```

### Notes
1. When initializing the AI rate detector, ensure `ckpt_dir` points to the correct model weight directory; otherwise, model loading will fail
2. **Automatic Backend Selection**: Based on `model_type`, the backend is automatically selected:
   - `t17` series models → PyTorch backend
   - `t7.5` / `paddle` series models → PaddlePaddle backend
   - Other models → TensorFlow backend
3. **Core Accuracy Note**: This tool has **relatively accurate** AI rate detection results for **small model-generated text**, which can meet the traceability needs of small model-generated content; however, it has poor AI rate detection effect for **large model-generated text**, and the detection results have low reference value. It is strictly prohibited to use this tool for AI determination scenarios of large model-generated content
4. After detection is completed, you must call the `detector.close()` method to release resources such as video memory and hardware handles to avoid memory leaks and excessive video memory usage caused by long-term operation
5. Character-level details are optional output items. For ultra-long text of ten thousand characters, printing these details will significantly increase output time and can be selectively printed according to actual needs
6. When detecting a large number of texts in batches, it is recommended to process them in batches according to text length to avoid detection lag caused by passing too many ultra-long texts in a single batch
7. Enabling `print_load_info=True` when loading the model allows you to view loading progress and hardware adaptation information, which is convenient for troubleshooting model loading exceptions

### PyTorch Backend Example (T17 Models)

```python
from xiaothink.llm.tools.ai_possibility import AIDetector

if __name__ == "__main__":
    # Initialize detector (automatically selects PyTorch backend)
    detector = AIDetector(
        ckpt_dir=r'path/to/your/t17_model',
        model_type='t17_tiny',
        print_load_info=True
    )

    # Detect text
    test_text = "This is a text to be detected."
    result = detector.detect_ai_rate(test_text)
    
    print(f"AI Rate: {result['AI Rate (Probability Average)']}")
    print(f"Detection Conclusion: {result['Detection Conclusion']}")

    # Release resources
    detector.close()
```

### Threshold Calibration (v1.4.2 New)

The AI rate detection tool provides flexible threshold configuration and calibration to adapt to different detection scenarios.

#### 1. Default Thresholds

| Threshold | Default | Description |
|-----------|---------|-------------|
| `threshold_high` | 0.01 | AI rate >= this value → "High probability AI generated" |
| `threshold_low` | 0.005 | AI rate >= this value → "Suspected AI generated" |

#### 2. Manual Threshold Adjustment

Developers can set custom thresholds during initialization or dynamically:

```python
# Method 1: Set during initialization
detector = AIDetector(
    ckpt_dir=r'path/to/your/t17_model',
    model_type='t17_tiny',
    threshold_high=0.02,   # Custom high threshold
    threshold_low=0.008    # Custom low threshold
)

# Method 2: Dynamic adjustment
detector.set_thresholds(high=0.015, low=0.006)  # Modify high threshold only
detector.set_thresholds(low=0.004)               # Modify low threshold only
```

#### 3. User Calibration

Automatically calculate optimal thresholds by providing AI-generated and human text samples. It is recommended to provide at least 3 samples per category; longer texts yield better calibration.

**Interactive Calibration** (suitable for terminal use):
```python
detector = AIDetector(
    ckpt_dir=r'path/to/your/t17_model',
    model_type='t17_tiny'
)
# Enter interactive calibration: follow prompts to input 3 AI + 3 human texts
calib_result = detector.auto_calibrate()
print(f"Calibrated high threshold: {calib_result['Calibrated Thresholds']['High Threshold']:.6f}")
print(f"Calibrated low threshold: {calib_result['Calibrated Thresholds']['Low Threshold']:.6f}")
```

**Programmatic Calibration** (suitable for code integration):
```python
detector = AIDetector(ckpt_dir=r'path/to/your/t17_model', model_type='t17_tiny')

ai_samples = [
    "This is the first AI-generated text for calibration.",
    "This is the second AI-generated text with different content.",
    "This is the third AI-generated text for better accuracy."
]
human_samples = [
    "This is a human-written text for distinguishing AI from human.",
    "This is the second human-written text in natural style.",
    "This is the third human sample for improved reliability."
]

calib_result = detector.calibrate(ai_samples, human_samples)
print(f"Before: high={calib_result['Pre-calibration Thresholds']['High Threshold']}, "
      f"low={calib_result['Pre-calibration Thresholds']['Low Threshold']}")
print(f"After: high={calib_result['Calibrated Thresholds']['High Threshold']}, "
      f"low={calib_result['Calibrated Thresholds']['Low Threshold']}")
```

#### 4. View Current Threshold Configuration

```python
info = detector.get_threshold_info()
print(f"High Threshold: {info['High AI Threshold']}")
print(f"Low Threshold: {info['Low AI Threshold']}")
print(f"Calibrated: {info['Calibrated']}")
```

Each detection result also includes the current threshold configuration:
```python
result = detector.detect_ai_rate("Test text")
print(result['Threshold Configuration'])
# Output: {'High Threshold': 0.01, 'Low Threshold': 0.005, 'Calibrated': False}
```

---

## TinySkill Toolbox (v1.4.2 New)

TinySkill is a lightweight skill registration and execution mechanism provided by the Xiaothink library. By registering prompt templates and execution types, users can quickly perform text generation or text classification tasks with the T17 model.

Each TinySkill can set its own **temperature, max_length, repetition_penalty** defaults, which can be overridden at runtime via `**kwargs`.

> **Note**: As of the Xiaothink-T17 series release, all models only support **Chinese** text processing. TinySkill prompts and keywords must be in Chinese.

### 1. Core Concepts

| Concept | Description |
|---------|-------------|
| **TinySkill** | A task definition containing an id, name, prompt template, and execution type |
| **generate type** | Text generation task — the model generates text based on the prompt |
| **classify type** | Text classification task — calculates probabilities based on keyword hit rates in the model output |
| **TinyToolbox** | Manages registration, querying, and execution of TinySkills |

### 2. Quick Start

```python
from xiaothink.llm.inference_torch.torch_formal import TorchModel
from xiaothink.llm.tools.tiny_skill import TinyToolbox, register_default_skills

# 1. Load the model
model = TorchModel(ckpt_dir="model_weight_directory", MT="t17_tiny")
model.set_form("ua_chat")

# 2. Create toolbox and register built-in skills
toolbox = TinyToolbox(model)
register_default_skills(toolbox)

# 3. Execute by skill name
result = toolbox.run("题目生成", "春天来了，万物复苏。")
print(result["text"])  # Generated title

# 4. Execute by skill id
result = toolbox.run("generate_title", "人工智能正在改变世界")
print(result["text"])

# 5. List all built-in skills
for sk in toolbox.list_skills():
    print(f"[{sk['id']}] {sk['name']} - {sk['type']} - {sk['description']}")
```

### 3. Registering Custom TinySkills

#### 3.1 Text Generation (generate)

Default params for generate type: **temperature=0.5, max_length=64, repetition_penalty=1.1**

```python
from xiaothink.llm.tools.tiny_skill import TinySkill

skill = TinySkill(
    id_="summarize_cn",
    name="内容概括",
    prompt="{input}请用一句话概括以下内容",
    type_="generate",
    description="用一句话概括输入文本",
    temperature=0.5,
    max_length=128,
    repetition_penalty=1.1,
)
toolbox.register(skill)

result = toolbox.run("内容概括", "今天天气真好，阳光明媚。")
print(result["text"])
```

#### 3.2 Text Classification (classify)

Default params for classify type: **temperature=0.001, max_length=128, repetition_penalty=1.0**

```python
skill = TinySkill(
    id_="spam_detect",
    name="垃圾邮件检测",
    prompt="{input}判断以下内容是否为垃圾邮件，给出分析过程",  # Chinese prompt only
    type_="classify",
    keywords=[
        {"name": "垃圾邮件", "id": "0", "key": ["垃圾", "诈骗", "中奖", "点击链接"]},
    ],
    other={"id": "1", "name": "正常邮件"},
    description="判断内容是否为垃圾邮件",
    temperature=0.001,
    repetition_penalty=1.0,
)
toolbox.register(skill)

result = toolbox.run("spam_detect", "恭喜您中奖了！点击领取奖品")
print(f"Probabilities: {result['probabilities']}")  # {'0': 0.XX, '1': 0.XX}
print(f"Prediction: {result['prediction']['name']}")
```

### 4. Classification Logic

Classify-type skills use **keyword hit probability**:

1. Call the model with an extremely low temperature (0.001) and repetition_penalty=1.0 for deterministic output
2. Count keyword occurrences for each category: `text.count(keyword)`
3. Probability = category hit count / total hit count
4. If total hits = 0, falls back to the `other` category (probability = 1.0)

**Example**: Classifier definition:
- Category `"0"` (广告推销): keywords `["广告", "营销", "推销"]`
- Category `"1"` (生活通知): `other` category

Model output: "这条短信涉嫌营销推广，属于广告宣传。"
Hit stats: `"广告"×1 + "营销"×1 + "推销"×0 = 2` hits
Probability: `{"0": 1.0, "1": 0.0}` → Classified as 广告推销.

### 5. Built-in TinySkills

| ID | Name | Type | Prompt | Default Params | Description |
|----|------|------|--------|---------------|-------------|
| `summarize_word` | 词语概括 | generate | `{input}请你用一个词语概括内容` | temp=0.5, maxlen=64, rp=1.1 | Summarize with one word |
| `summarize_short` | 短词概括 | generate | `{input}请你用简短的词语概括内容` | temp=0.5, maxlen=64, rp=1.1 | Summarize with short words |
| `generate_title` | 题目生成 | generate | `{input}为这段文本生成题目` | temp=0.5, maxlen=64, rp=1.1 | Generate a title |
| `generate_poem` | 古诗生成 | generate | `{input}为以上内容生成一段优美古诗` | temp=0.5, maxlen=64, rp=1.1 | Generate a classical poem |
| `generate_philosophy` | 哲理文本 | generate | `{input}为以上内容生成一段富有哲理的文本` | temp=0.5, maxlen=64, rp=1.1 | Generate philosophical text |
| `generate_jueju` | 绝句生成 | generate | `关于"{input}"请生成一首绝句` | temp=0.5, maxlen=64, rp=1.1 | Generate a jueju poem |
| `sms_classify` | 短信分类 | classify | `{input}请你分析这段短信是广告推销还是生活通知？给出推理过程` | temp=0.001, rp=1.0 | Classify ads vs. notifications. Keywords: 广告、营销、推销、销、宣传; other: 生活通知 |
| `emotion_analysis` | 情感分析 | classify | `{input}分析以上文本的情感倾向` | temp=0.001, rp=1.0 | Analyze positive/negative/neutral. Positive keys: 正、积极、好、愉快、开心、喜欢; Negative keys: 反、消极、坏、差、烂、不、讨厌; other: 中性 |

### 6. Custom Registration Example

```python
# Custom classification: Feedback type analysis
feedback_skill = TinySkill(
    id_="feedback_classify",
    name="反馈分类",
    prompt="{input}分析以下用户反馈的类型",  # Chinese only
    type_="classify",
    keywords=[
        {"name": "投诉", "id": "0", "key": ["投诉", "不满", "差评", "失望"]},
        {"name": "建议", "id": "1", "key": ["建议", "希望", "改进", "优化"]},
        {"name": "咨询", "id": "2", "key": ["请问", "怎么", "如何"]},
    ],
    other={"id": "-1", "name": "其他"},
    description="分析用户反馈类型（投诉/建议/咨询/其他）",
)
toolbox.register(feedback_skill)
```

### 7. Parameter Priority

**Priority**: `run(**kwargs)` > `TinySkill defaults` > `Toolbox internal defaults`

**Toolbox internal defaults**:

| Type | temperature | max_length | repetition_penalty |
|------|-------------|------------|-------------------|
| generate | 0.5 | 64 | 1.1 |
| classify | 0.001 | 128 | 1.0 |

```python
result = toolbox.run(
    "古诗生成",
    "秋天景色",
    temperature=0.9,           # Overrides built-in 0.5
    max_length=256,            # Overrides built-in 64
    repetition_penalty=1.05,   # Overrides built-in 1.1
    top_p=0.9,
    stop_conditions=["<", "\n", "。"]  # Custom stop conditions
)
```

---

Xiaothink framework series model names, their corresponding MT (model architecture version), and form (model prompt input format) list:
| Model Name (by release time)              | mt parameter           | form parameter   |
|-----------------------|------------------|-------------|
| Xiaothink-T17-Tiny | mt='t17_tiny' | No form parameter needed |
| Xiaothink-T7.5-0.1B | mt='t7.5_paddle_small_instruct' | form=2 |
| Xiaothink-T7-ART(0.07B)| mt='t7_cpu_standard'    | form=1 |
| Xiaothink-T6-0.08B       | mt='t6_beta_dense'| form=1      |
| Xiaothink-T6-0.15B       | mt='t6_standard' | form=1      |
| Xiaothink-T6-0.02B       | mt='t6_fast'     | form=1      |
| Xiaothink-T6-0.5B        | mt='t6_large'    | form=1      |
| Xiaothink-T6-0.5B-pretrain| mt='t6_large'    | form='pretrain' |

---

## Changelog
### Version 1.4.2 (2026-06-10)
- **New Module**:
  - Added `xiaothink.llm.inference_torch` module for PyTorch-based inference
  - Supports Xiaothink-T17 series models (GRU/LSTM architecture + history retrieval mechanism)
  - Provides `TorchModel` class for PyTorch inference
  - **New TinySkill Toolbox**: `xiaothink.llm.tools.tiny_skill` with skill registration and execution (generate/classify types), 8 built-in skills (word summary, short summary, title generation, poem generation, philosophical text, jueju poem, SMS classification, emotion analysis)
- **Model Features**:
  - Innovative history retrieval mechanism: retrieves relevant historical context during generation
  - Gate fusion: learnable gate mechanism for blending RNN output with retrieved history
  - Causal masking: ensures no future information leakage during training and inference
  - Efficient inference: optimized for on-device deployment with low memory footprint
- **Supported MT Architectures**:
  - 't17_tiny': GRU-based model with history retrieval
- **AI Rate Detection**:
  - Added PyTorch backend support for AI rate detection
  - Automatic backend selection based on model_type
  - **New Calibration Feature**: `calibrate()` and `auto_calibrate()` methods for automatic threshold optimization using AI/human text samples
  - **Manual Threshold Adjustment**: `set_thresholds(high, low)` method for custom thresholds
  - New `threshold_high` and `threshold_low` parameters in `__init__`
  - Detection results now include `Threshold Configuration` field with current thresholds and calibration status
  - **Import Order Fix**: Reordered backend imports (PyTorch > PaddlePaddle > TensorFlow) to prevent numpy 2.0 TensorFlow crash from contaminating other backends
  - **New `XIAOTHINK_BACKEND` Env Var**: Set to `torch`/`paddle`/`tensorflow`/`auto` to import only the specified backend, completely skipping others to avoid compatibility issues

### Version 1.4.1 (2026-02-16)
- **New Module**:
  - Added `xiaothink.llm.inference_paddle` module for PaddlePaddle-based inference
  - Supports Xiaothink-T7.5 series models with RWKV architecture
  - Provides `TextGenerator` and `QianyanModel` classes for PaddlePaddle
- **Updated Dependencies**:
  - Removed TensorFlow as a required dependency (now optional)
  - Added PaddlePaddle as the primary deep learning framework
  - Added jieba for Chinese word segmentation
- **Model Support**:
  - Added support for MT architectures: 't7.5_paddle_small_instruct', 't7.5_paddle_small_instruct_pro', etc.
  - Supports automatic device selection (CPU/GPU) based on memory usage

### Version 1.4.0 (2026-02-16)[Yanked]
- **Note**: This version has been yanked due to issues with the README.md file content. Please use the updated version instead.

### Version 1.3.2 (2025-12-27)
- **Updated Interfaces**:
  - Added "AI Rate Detection" interface based on Xiaothink-T series models.
- **New Models**:
  - Added support for MT architectures "t7" and "t7_cpu_standard" in the Xiaothink-T7 series models.

### Version 1.3.1 (2025-10-31)
- **Updated Interfaces**:
  - Added custom input shape (must be supported by the corresponding model) for vision-related interfaces instead of the fixed 80*80*3 in previous versions
  - The ImgZIP command-line interface also added custom input shape (must be supported by the corresponding model) instead of the fixed 80*80*3 in previous versions, and added comprehensive quality scores based on SNR, PSNR, and SSIM.

### Version 1.3.0 (2025-10-17)[Yanked]
- **New Models**:
  - Added support for the Xiaothink-T7 series model architecture.

### Version 1.2.5 (2025-09-02)
- **Updated Interfaces**:
  - Added "custom compression rate" function to the ImgZIP command-line interface, supporting other compression rates beyond the model's native compression rate (implemented based on calculating and scaling the original image).

### Version 1.2.4 (2025-08-30)
- **Updated Interfaces**:
  - Updated the import method of ImgZIP-related interfaces in the documentation to: from xiaothink.llm.img_zip.img_zip import ImgZip

### Version 1.2.3 (2025-08-30)
- **New Features**:
  - Added Xiaothink-T6-0.02B series models (MT='t6_fast')
  - Added Xiaothink-T6-0.5B series models (MT='t6_large')
  - Added support for form='pretrain' in the model.chat method. For instruction-tuned models in the T6 series, form=1 should be used; for pre-trained models, form='pretrain' should be used

### Version 1.2.2 (2025-08-18)
- **New Features**:
  - Added sentiment classification tool to implement text sentiment tendency analysis through `ClassifyModel`
  - Added `xiaothink.llm.tools.classify` module to support sentiment classification based on basic dialogue models
  - Provided `cmodel.emotion(inp)` interface to return real-time text sentiment results

### Version 1.2.1 (2025-08-16)
- **New Models**:
  - Added Xiaothink-T6-0.15B series models (MT='t6_standard')

### Version 1.2.0 (2025-08-08)
- **Breakthrough Innovation**:
  - Added support for native vision models using an innovative dual-vision solution
  - Dual-path processing of image compression to feature tokens (img_zip) + native vision encoder
  - Retains multi-image context understanding capability while achieving single-image detail analysis

- **New Interfaces**:
  - `model.chat_vision`: Specialized dialogue interface for vision models
  - `model.img2ms`: Image description interface for non-native vision models
  - `model.img2ms_vision`: Image description interface for native vision models (supports max_shape parameter)
  
- **Module Expansion**:
  - Added `xiaothink.llm.img_zip.img_zip` command-line tool
  - Supports compression and decompression of images and videos
  - Provides rich parameters to adjust compression quality

- **Usage Guidelines**:
  - Vision models must use the `chat_vision` method
  - Must use a matching img_zip encoder model
  - Image paths should use absolute paths

### Version 1.1.0 (2025-08-02)
- **New Features**:
  - Added `img2ms` and `ms2img` interfaces to achieve high compression ratio lossy compression of images
  - Supports converting images into AI-readable feature tokens
  - Extended dialogue models to support multimodal input (image + text)
  - In test_formal, it supports converting feature tokens generated by multimodal AI into images and saving them to the system temporary folder by default.
  
- **Technical Upgrades**:
  - Based on Xiaothink framework's self-developed img_zip technology
  - Supports intelligent compression of 80x80x3 image patches
  - When outputting 96 feature values, combined with .7z algorithm, it can achieve an ultra-high compression ratio of 10%
  
- **Usage Method**:
  - Insert images using the `<img>{image_path}</img>` tag in dialogue
  - Need to specify the img_zip model path when initializing the model
  - Supports multimodal dialogue (image description, image Q&A, and other scenarios)

---

The above covers the main functions and usage methods of the Xiaothink Python module.

If you have any questions or suggestions, please feel free to contact us: xiaothink@foxmail.com.
