No project description provided
Project description
MistralAI Questionnaire
This project provides a toolkit for generating questionnaire from documents: [txt
, docx
, pdf
] to .csv
dataset format.
Requirements
Before starting, you need to install the following libraries: .. code-block:: python
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
langchain
langchain_community
langchain-huggingface
playwright
html2text
sentence_transformers
faiss-cpu
pandas
peft==0.4.0
trl==0.4.7
pypdf
bitsandbytes
accelerate
Description
ModelManager
This class is responsible for loading mistralai model and generating QA.
Constructor
^^^^^^^^^^^
.. code-block:: python
__init__(self, model_name)
- **model_name**: The path or name of the pre-trained model.
Methods
^^^^^^^
- **setup_tokenizer()**: Loads and configures the tokenizer for the model.
- **setup_bitsandbytes_parameters()**: Configures parameters for bit quantization (BitsAndBytes).
- **from_pretrained()**: Loads the model with pre-trained weights and quantization configuration.
- **print_model_parameters(examples)**: Prints the number of trainable and total parameters of the model.
- **__call__(self, *args, **kwargs)**: The main method for running the generate tasks.
Usage
-----
To start generating QA, you should create an instance of the ``ModelManager`` class and call its ``__call__`` method, passing the necessary arguments.
.. code-block:: python
from questionnaire_mistral.models import ModelManager
model = ModelManager(model_name="path_to_model")
model(document=document, task=task, document_content=document_content, task_count=task_count)
License
-------
The project is distributed under the MIT License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file questionnaire_mistral-3.1.tar.gz
.
File metadata
- Download URL: questionnaire_mistral-3.1.tar.gz
- Upload date:
- Size: 9.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.15
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7a681a17bb83dbe8e4faae8232c3179614226ff76030861ce2c5ddf458d5599 |
|
MD5 | c1ca963266847f6cf9d8a567803788ba |
|
BLAKE2b-256 | 296a365fc60ea532448fa17dbe0ad3d370b3ba22ecda708e9047bcd6412126cc |
File details
Details for the file questionnaire_mistral-3.1-py3-none-any.whl
.
File metadata
- Download URL: questionnaire_mistral-3.1-py3-none-any.whl
- Upload date:
- Size: 9.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.15
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 946e42b63053bdc0e6f070631addad81b0e42e78f011dc0072e24d24c22d1c04 |
|
MD5 | 4a208c1df22e4ea32152063bad21d714 |
|
BLAKE2b-256 | 2137188cfc2c3b3523be5e6ecd2819cf8b30d53a5ba9a6d701edebccab819970 |