A Python package for handling multiple concurrent tasks using multiprocessing and multithreading.
Project description
CD Multi Tasker
cd_multi_tasker
is a Python package designed to handle multiple concurrent tasks efficiently. It supports both CPU-bound and I/O-bound tasks, using multiprocessing for CPU-intensive operations and multithreading for I/O-bound operations.
Features
- Multiprocessing for CPU-bound tasks such as file processing.
- Multithreading for I/O-bound tasks such as HTTP requests.
- Easy-to-use interface for running concurrent tasks.
Installation
To install the package, use:
pip install cd_multi_tasker
Usage
Importing the Package
from cd_multi_tasker.cd_multi_tasker import MultiTasking
Example: CPU-bound Tasks (File Processing)
# Initialize the MultiTasking class
multitasker = MultiTasking(max_workers=4)
# Define a sample task function for file processing
def process_file(file_path):
"""
Dummy function to simulate file processing.
Replace with actual file processing logic.
Parameters:
file_path (str): Path to the file to process.
Returns:
str: Dummy result message.
"""
return f"Processed file {file_path}"
# List of file paths to process
file_paths = ["/path/to/file1", "/path/to/file2", "/path/to/file3"]
# Run the tasks
results = multitasker.run_cpu_bound_tasks(file_paths, process_file)
print("File processing results:", results)
Example: I/O-bound Tasks (HTTP Requests)
# Initialize the MultiTasking class
multitasker = MultiTasking(max_workers=4)
# Define a sample task function for HTTP requests
def fetch_data(url):
"""
Dummy function to simulate fetching data from a URL.
Replace with actual HTTP request logic.
Parameters:
url (str): URL to fetch data from.
Returns:
str: Dummy response message.
"""
return f"Fetched data from {url}"
# List of URLs to fetch data from
urls = ["http://example.com/data1", "http://example.com/data2"]
# Run the tasks
results = multitasker.run_io_bound_tasks(urls, fetch_data)
print("HTTP fetching results:", results)
Class and Methods
MultiTasking
__init__(self, max_workers)
- Initializes the MultiTasking class with the maximum number of workers.
- Parameters:
max_workers
(int): Maximum number of concurrent tasks.
run_cpu_bound_tasks(self, tasks, task_func, *args, **kwargs)
- Executes multiple CPU-bound tasks concurrently using multiprocessing.
- Parameters:
tasks
(list): List of tasks to be processed (e.g., file paths for processing).task_func
(callable): Function to run for each task.*args
: Variable length argument list for the task function.**kwargs
: Arbitrary keyword arguments for the task function.
- Returns:
list
: Results of the tasks.
run_io_bound_tasks(self, tasks, task_func, *args, **kwargs)
- Executes multiple I/O-bound tasks concurrently using multithreading.
- Parameters:
tasks
(list): List of tasks to be processed (e.g., URLs for web requests).task_func
(callable): Function to run for each task.*args
: Variable length argument list for the task function.**kwargs
: Arbitrary keyword arguments for the task function.
- Returns:
list
: Results of the tasks.
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
License
This project is licensed under the MIT License.
More Info
For more information, visit codedocta.com.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
cd_multi_tasker-0.1.1.tar.gz
(3.5 kB
view hashes)
Built Distribution
Close
Hashes for cd_multi_tasker-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c3497ec78247c680669b2dab912464dd7c39c1323bddff719a16c36f43e3bdf0 |
|
MD5 | 80f59c76371f99b8e0d312ca76595a5e |
|
BLAKE2b-256 | 6a43fef28bc5706cf010a2a0f11d2606a65d2a54b3c08d921b51b382102cd8b4 |