rapidinstall 1.0.1

Downloads, Installs in paralell - A utility to run multiple shell command sequences (e.g., installations) in parallel with status updates.

rapidinstall

A simple Python utility to run multiple shell command sequences (e.g., complex installations, downloads, build steps) in parallel, providing status updates and aggregated output directly to the console.

Features

• Parallel Execution: Runs multiple tasks concurrently using subprocess.
• Real-time Output: Captures stdout and stderr from tasks.
• Status Updates: Periodically shows which tasks are still running and recent output.
• Clear Formatting: Presents startup, status, and completion information in distinct blocks.
• Simple API: Easy to integrate into existing Python scripts.
• Pure Python: Relies only on the standard library with optionally requiring pySmartDL if you use the aria2-like download manager.

Installation

pip install rapidinstall pySmartDL # pySmartDL is optional

Examples

Example: first look

import rapidinstall

my_tasks = [
    {'name': 'Pip Update', 'commands': 'pip install --upgrade pip'},
    {'name': 'Download Example', 'download': "echo 'Simulating download...'; sleep 30; echo 'Download complete!'"}, # using pySmartDL, similar to aria2
    {'name': 'Run Setup', 'commands': "echo 'Running setup...'; sleep 20; echo 'Setup finished.'"}
]

rapidinstall.install(my_tasks)

Runs the 3 tasks in paralell and updates you on the progress and result.

Example: Git installs

import rapidinstall

clone_and_install = '''clone https://github.com/tomOfBerlin/rapidinstall
cd rapidinstall
python setup.py demo-install
cd ..'''

my_tasks = [
    {'name': 'Pip Update', 'commands': 'pip install --upgrade pip'},
    {'name': 'Download Example', 'commands': "echo 'Simulating download...'; sleep 30; echo 'Download complete!'"},
    {'name': 'Run Setup', 'commands': clone_and_install}
]

rapidinstall.install(my_tasks)

Example: Some Parameters

# Completely silent - verbose is turned on by default
rapidinstall.install(my_tasks, verbose=False)

# Shows Updates every 60 seconds
rapidinstall.install(my_tasks, update_interval=60)

# No Updates
results = rapidinstall.install(my_tasks, update_interval=0)

# Show results your way
rc = result.get('returncode')
duration = result.get('duration_sec')
status = "Success" if rc == 0 else f"Failed (RC={rc})"
duration_str = f"{duration:.2f} sec" if duration is not None else "N/A"
print(f"Task '{name}': {status} (Duration: {duration_str})")
print(f"Full Stdout:\n{result['stdout']}") # full "console" output
print(f"Full Stderr:\n{result['stderr']}") # full error output

RapidInstaller Class

Example

The class interface provides more flexibility, especially if tasks are generated dynamically. Tasks start immediately when added.

from rapidinstall import RapidInstaller

installer = RapidInstaller(verbose=True, update_interval=10)

# Start the downloads even before basic setup.
installer.add_download(name='HF model1', url='https://example.com/model1.zip')
installer.add_download(name='HF model2', url='https://example.com/model1.zip')
installer.add_download(name='HF model3', url='https://example.com/model1.zip')

# Basic Setup, e.g. connect to gdrive on colab
#
#
#

# Rest of the install that for example requires a drive connection
# Note: Please test if paralell pip installs work together or have conflicts.
clone_and_install = '''clone https://github.com/tomOfBerlin/rapidinstall
cd rapidinstall
python setup.py demo-install
pip install -r requirements.txt
cd ..'''

setup_tasks = [
    {'name': 'Pip installs', 'commands': 'pip install a b c'},
    {'name': 'Setup Demo', 'commands': clone_and_install}
]

installer.add_tasks(setup_tasks)

# Some more setup that creates new directories
#
#
#

installer.add_download(name='extra model 123', url='https://example.com/model1.zip', directory='new_directory/models')

# Wait for everything to finish
installer.wait()

This example demonstrates adding tasks incrementally. The wait() call ensures the script doesn't exit until all tasks and downloads have completed execution.

Error Handling

• rapidinstall (both the function and the class) runs all initiated tasks to completion, regardless of individual task failures (non-zero exit codes).
• Failure information (non-zero returncode, stderr output) is captured for each task and included in the final results dictionary returned by install() or wait().
• If verbose=True, the console output during execution and in the final summary will typically highlight failed tasks with their return code and stderr.
• Programmatically check the returncode key in the results dictionary for each task to determine success or failure. A returncode of 0 indicates success.

License

This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.◊