talktollm

PyPI version License: MIT

A Python utility for interacting with large language models (LLMs) through browser automation. It leverages image recognition to automate interactions with LLM web interfaces, enabling seamless conversations and task execution.

Features

  • Simple Interface: Provides a single, intuitive function for interacting with LLMs.
  • Automated Image Recognition: Employs image recognition (optimisewait) to identify and interact with elements on the LLM interface.
  • Multi-LLM Support: Supports DeepSeek, Gemini, and Google AI Studio.
  • Automated Conversations: Facilitates automated conversations and task execution by simulating user interactions.
  • Image Support: Allows sending one or more images (as base64 data URIs) to the LLM.
  • Robust Clipboard Handling: Includes retry mechanisms for setting and getting clipboard data, handling common access errors and timing issues.
  • Self-Healing Image Cache: Creates a clean, temporary image cache for each run, preventing issues from stale or corrupted recognition assets.
  • Easy to use: Designed for simple setup and usage.

Core Functionality

The core function is talkto(llm, prompt, imagedata=None, debug=False, tabswitch=True).

Arguments:

  • llm (str): The LLM name ('deepseek', 'gemini', or 'aistudio').
  • prompt (str): The text prompt to send.
  • imagedata (list[str] | None): Optional list of base64 encoded image data URIs (e.g., "data:image/png;base64,...").
  • debug (bool): Enable detailed console output. Defaults to False.
  • tabswitch (bool): Switch focus back to the previous window after closing the LLM tab. Defaults to True.

Steps:

  1. Validates the LLM name.
  2. Ensures a clean temporary image cache is ready for optimisewait.
  3. Opens the LLM's website in a new browser tab.
  4. Waits for and clicks the message input area.
  5. If imagedata is provided, it pastes each image into the input area.
  6. Pastes the prompt text.
  7. Clicks the 'run' or 'send' button.
  8. Sets a placeholder value on the clipboard.
  9. Waits for the 'copy' button to appear (indicating the response is ready) and clicks it.
  10. Polls the clipboard until its content changes from the placeholder value.
  11. Closes the browser tab (Ctrl+W).
  12. Switches focus back if tabswitch is True (Alt+Tab).
  13. Returns the retrieved text response, or an empty string if the process times out.

Helper Functions

Clipboard Handling:

  • set_clipboard(text: str, retries: int = 5, delay: float = 0.2): Sets text to the clipboard. Retries on common access errors.
  • set_clipboard_image(image_data: str, retries: int = 5, delay: float = 0.2): Sets a base64 encoded image to the clipboard. Retries on common access errors.
  • _get_clipboard_content(...): Internal helper to read text from the clipboard with retry logic.

Image Path Management:

  • copy_images_to_temp(llm: str, debug: bool = False): Deletes and recreates the LLM-specific temporary image folder to ensure a clean state. Copies necessary .png images from the package's internal images/ directory to the temporary location.

Installation

pip install talktollm

Note: Requires optimisewait for image recognition. Install separately if needed (pip install optimisewait).

Usage

Here are some examples of how to use talktollm.

Example 1: Simple Text Prompt

Send a basic text prompt to Gemini.

import talktollm

prompt_text = "Explain quantum entanglement in simple terms."
response = talktollm.talkto('gemini', prompt_text)
print("--- Simple Gemini Response ---")
print(response)

Example 2: Text Prompt with Debugging

Send a text prompt to AI Studio and enable debugging output.

import talktollm

prompt_text = "What are the main features of Python 3.12?"
response = talktollm.talkto('aistudio', prompt_text, debug=True)
print("--- AI Studio Debug Response ---")
print(response)

Example 3: Preparing Image Data

Load an image file, encode it in base64, and format it correctly for the imagedata argument.

import base64

# Load your image (replace 'path/to/your/image.png' with the actual path)
try:
    with open("path/to/your/image.png", "rb") as image_file:
        # Encode to base64
        encoded_string = base64.b64encode(image_file.read()).decode('utf-8')
        # Format as a data URI
        image_data_uri = f"data:image/png;base64,{encoded_string}"
        print("Image prepared successfully!")
except FileNotFoundError:
    print("Error: Image file not found. Please check the path.")
    image_data_uri = None

# This 'image_data_uri' variable holds the string needed for the next example

Example 4: Text and Image Prompt

Send a text prompt along with a prepared image to DeepSeek. (Assumes image_data_uri was successfully created in Example 3).

import talktollm

# Assuming image_data_uri is available from the previous example
if image_data_uri:
    prompt_text = "Describe the main subject of this image."
    response = talktollm.talkto(
        'deepseek',
        prompt_text,
        imagedata=[image_data_uri], # Pass the image data as a list
        debug=True
    )
    print("--- DeepSeek Image Response ---")
    print(response)
else:
    print("Skipping image example because image data is not available.")

Dependencies

  • pywin32: For Windows API access (clipboard).
  • pyautogui: For GUI automation (keystrokes).
  • Pillow: For image processing.
  • optimisewait (Recommended): For robust image-based waiting and clicking.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

License

MIT

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.10.4)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for AMAMazing from gravatar.com AMAMazing

Credits

Author: Alex M

License

MIT License

Requires

Python >=3.6

Tags

llm automation gui pyautogui gemini deepseek clipboard aistudio

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

talktollm-0.10.0.tar.gz (89.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

talktollm-0.10.0-py3-none-any.whl (94.9 kB view details)

Uploaded Python 3

File details

Details for the file talktollm-0.10.0.tar.gz.

File metadata

  • Download URL: talktollm-0.10.0.tar.gz
  • Upload date:
  • Size: 89.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.6

File hashes

Hashes for talktollm-0.10.0.tar.gz
Algorithm Hash digest
SHA256 ced0bf4d221244854cda98b003fbf76d07559c050b08a4fbab05326641a434c8
MD5 8a1da9e31069f3248def7e9c5d020694
BLAKE2b-256 ccf68eea0053c6da9068776c8b711c5bc69aced0f38eb6bfb67edbafb966f8ff

See more details on using hashes here.

File details

Details for the file talktollm-0.10.0-py3-none-any.whl.

File metadata

  • Download URL: talktollm-0.10.0-py3-none-any.whl
  • Upload date:
  • Size: 94.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.6

File hashes

Hashes for talktollm-0.10.0-py3-none-any.whl
Algorithm Hash digest
SHA256 87bf5d83078fb1048b79162e59963d65de9aea80b382c5574876a3cc5cf865a4
MD5 e66498da97488758eab522d8289bbc9a
BLAKE2b-256 61c0ac81aa83baefb093872c2fc8b40edf6728a7fad68b7bce399a2d9c79a7ab

See more details on using hashes here.

Release history Release notifications | RSS feed

0.10.4

2 files

0.10.3

2 files

0.10.2

2 files

0.10.1

2 files

This release

0.10.0 This release

2 files

0.9.1

2 files

0.9.0

2 files

0.8.6

2 files

0.8.5

2 files

0.8.4

2 files

0.8.3

2 files

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.1

2 files

0.7.0

2 files

0.6.10

2 files

0.6.9

2 files

0.6.8

2 files

0.6.7

2 files

0.6.6

2 files

0.6.5

2 files

0.6.4

2 files

0.6.3

2 files

0.6.2

2 files

0.6.1

2 files

0.5.5

2 files

0.5.4

2 files

0.5.3

2 files

0.5.2

2 files

0.5.1

2 files

0.5.0

2 files

0.4.9

2 files

0.4.8

2 files

0.4.7

2 files

0.4.6

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.1

2 files

0.3.0

2 files

0.2.7

2 files

0.2.6

2 files

0.2.5

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.0

2 files

0.1.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page