Skip to main content

No project description provided

Project description

MistralAI Questionnaire

This project provides a toolkit for generating questionnaire from documents: [txt, docx, pdf] to .csv dataset format.

Requirements

Before starting, you need to install the following libraries: .. code-block:: python

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

  • langchain
  • langchain_community
  • langchain-huggingface
  • playwright
  • html2text
  • sentence_transformers
  • faiss-cpu
  • pandas
  • peft==0.4.0
  • trl==0.4.7
  • pypdf
  • bitsandbytes
  • accelerate

Description

ModelManager


This class is responsible for loading mistralai model and generating QA.

Constructor
^^^^^^^^^^^

.. code-block:: python

   __init__(self, model_name)

- **model_name**: The path or name of the pre-trained model.


Methods
^^^^^^^

- **setup_tokenizer()**: Loads and configures the tokenizer for the model.
- **setup_bitsandbytes_parameters()**: Configures parameters for bit quantization (BitsAndBytes).
- **from_pretrained()**: Loads the model with pre-trained weights and quantization configuration.
- **print_model_parameters(examples)**: Prints the number of trainable and total parameters of the model.
- **__call__(self, *args, **kwargs)**: The main method for running the generate tasks.

Usage
-----

To start generating QA, you should create an instance of the ``ModelManager`` class and call its ``__call__`` method, passing the necessary arguments.

.. code-block:: python
   from questionnaire_mistral.models import ModelManager
   model = ModelManager(model_name="path_to_model")
   model(document=document, task=task, document_content=document_content, task_count=task_count)

License
-------

The project is distributed under the MIT License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

questionnaire_mistral-3.1.tar.gz (9.4 kB view details)

Uploaded Source

Built Distribution

questionnaire_mistral-3.1-py3-none-any.whl (9.5 kB view details)

Uploaded Python 3

File details

Details for the file questionnaire_mistral-3.1.tar.gz.

File metadata

  • Download URL: questionnaire_mistral-3.1.tar.gz
  • Upload date:
  • Size: 9.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.15

File hashes

Hashes for questionnaire_mistral-3.1.tar.gz
Algorithm Hash digest
SHA256 f7a681a17bb83dbe8e4faae8232c3179614226ff76030861ce2c5ddf458d5599
MD5 c1ca963266847f6cf9d8a567803788ba
BLAKE2b-256 296a365fc60ea532448fa17dbe0ad3d370b3ba22ecda708e9047bcd6412126cc

See more details on using hashes here.

File details

Details for the file questionnaire_mistral-3.1-py3-none-any.whl.

File metadata

File hashes

Hashes for questionnaire_mistral-3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 946e42b63053bdc0e6f070631addad81b0e42e78f011dc0072e24d24c22d1c04
MD5 4a208c1df22e4ea32152063bad21d714
BLAKE2b-256 2137188cfc2c3b3523be5e6ecd2819cf8b30d53a5ba9a6d701edebccab819970

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page