runsawa 0.1.0

A simple library for running loops concurrently with progress.

Run Sawa

A simple Python library for running functions concurrently with progress tracking. Perfect for speeding up I/O-bound operations like API calls, file processing, or any task that can benefit from parallel execution.

Sawa is Arabic slang for together.

Features

• Concurrent Execution: Uses ThreadPoolExecutor for parallel processing
• Progress Tracking: Built-in progress bar with tqdm
• Error Handling: Graceful handling of individual task failures
• Configurable Workers: Adjust the number of concurrent workers
• Simple API: Easy-to-use generator-based interface
• Python 3.8+: Compatible with modern Python versions

Installation

From PyPI (recommended)

pip install runsawa

From Source

git clone https://github.com/orasik/runsawa.git
cd runsawa
pip install -e .

Quick Start

from runsawa import runsawa

# Basic usage - parallel processing with progress bar
def process_item(item):
    # Your processing logic here
    return item * 2

# Process items concurrently
for result in runsawa(process_item, range(100), workers=10):
    print(result)

Usage Examples

Example 1: CPU-bound Tasks

import time
from runsawa import runsawa

def compute_task(n):
    time.sleep(0.001)  # Simulate computation
    return n * n

# Sequential vs Parallel comparison
items = range(1000)

# With runsawa (parallel)
results = list(runsawa(compute_task, items, workers=20))

Example 2: API Calls (I/O-bound)

import requests
from runsawa import runsawa

def fetch_post(post_id):
    response = requests.get(f"https://jsonplaceholder.typicode.com/posts/{post_id}")
    return response.json()

# Fetch multiple posts concurrently
post_ids = range(1, 51)
posts = list(runsawa(fetch_post, post_ids, workers=15))

Example 3: File Processing

from runsawa import runsawa
import json

def process_file(filename):
    with open(filename, 'r') as f:
        data = json.load(f)
    # Process the data...
    return len(data)

files = ['data1.json', 'data2.json', 'data3.json']
results = list(runsawa(process_file, files, workers=3))

API Reference

runsawa(func, iterable, workers=5)

Runs a function concurrently on an iterable with progress tracking.

Parameters:

• func (callable): The function to apply to each item in the iterable
• iterable (iterable): The items to process
• workers (int, optional): Number of concurrent workers. Default is 5

Returns:

• Generator yielding results as they complete

Example:

from runsawa import runsawa

def square(x):
    return x ** 2

results = list(runsawa(square, range(10), workers=3))

Performance

Run Sawa is particularly effective for I/O-bound tasks. Here's a typical performance comparison:

# Sequential processing: 45.2 seconds
# With runsawa (15 workers): 3.8 seconds
# Speedup: ~12x faster for API calls

The actual speedup depends on:

• Task type (I/O-bound tasks see better improvements)
• Network latency (for API calls)
• Number of workers vs available resources
• Task complexity

Running Tests

# Run all tests
python -m pytest tests/

# Run specific test file
python -m unittest tests.test_sawa

# Run tests with coverage
pip install coverage
coverage run -m pytest tests/
coverage report

Running Examples

# Basic example
python example.py

# API example (requires internet connection)
python api_example.py

Limitations

• Threading Model: Uses ThreadPoolExecutor, which is ideal for I/O-bound tasks but limited by Python's GIL for CPU-bound tasks
• Memory Usage: Converts input iterable to a list, which may consume significant memory for very large datasets
• Result Order: Results are yielded as they complete, not necessarily in input order
• Error Handling: Individual task failures are logged but don't stop the overall execution
• Dependencies: Requires tqdm for progress bars and requests for HTTP examples

Use Cases

Ideal for:

• API calls and web scraping
• File I/O operations
• Database queries
• Network requests
• Image processing (I/O operations)

Less ideal for:

• Pure CPU-bound mathematical computations (consider multiprocessing instead)
• Very large datasets that don't fit in memory
• Tasks requiring strict result ordering

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.