Skip to main content

Master-Worker Connection Manager Tutorial

This tutorial will guide you through setting up and using the Master-Worker Connection Manager system.

Prerequisites

  • Python 3.x installed on all machines (master and workers)
  • Network connectivity between the master and worker machines

Setup

  1. Ensure all necessary files are in place:

    • connection_manager.py (contains the MasterManager class)
    • Any additional dependencies
  2. Install required Python packages:

    pip install socket threading sqlite3
    

Usage

Starting the Master Server

  1. Create a Python script (e.g., run_master.py) with the following content:

    from connection_manager import MasterManager
    
    master = MasterManager()
    master.start_master_server(ip_address='0.0.0.0', port=5000, authorized_workers=['192.168.1.100', '192.168.1.101'])
    
  2. Run the master server:

    python run_master.py
    

    This will start the master server on all available network interfaces (0.0.0.0) on port 5000.

Connecting Workers

  1. Ensure workers have the necessary client-side code to connect to the master.

  2. Workers should send their local IP addresses to the master upon connection.

  3. Only authorized workers (as specified in authorized_workers list) will be allowed to connect.

Features

  • Authorization: The master server authorizes workers based on their IP addresses.
  • Multi-threading: The server can handle multiple worker connections simultaneously.
  • SQLite Database: Client IDs and specifications are stored in an SQLite database.

Troubleshooting

  • If connections fail, check firewall settings and ensure the correct IP addresses and ports are being used.
  • Verify that all workers' IP addresses are correctly listed in the authorized_workers list.

Extending the System

To add more functionality:

  1. Modify the MasterManager class in connection_manager.py.
  2. Implement additional methods for task distribution, result collection, etc.
  3. Update the client-side code on workers to handle new features.

For more detailed information, refer to the comments in the connection_manager.py file.

Release files for multipal 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for multipal 0.1.1
File Size Uploaded
multipal-0.1.1.tar.gz 9.7 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for multipal 0.1.1
File Interpreter ABI Platform
multipal-0.1.1-py3-none-any.whl Python 3 none any Details

Total release size:19.7 kB

Release files / multipal-0.1.1.tar.gz

Download URL multipal-0.1.1.tar.gz
Size 9.7 kB
Tags Source
SHA-256 checksum
How to use checksums
da55ab1b2519c8d14bdbd2bf0edb39922799babde4e5cdaa297f0af2e1a444f4
BLAKE2b-256 checksum
How to use checksums
5bc722a33e3fd463c7f779ccf6ea1f9a1b046054db8f81ac1bc5c63cc4f20ab5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.10.4

Release files / multipal-0.1.1-py3-none-any.whl

Download URL multipal-0.1.1-py3-none-any.whl
Size 10.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b3f09e9bb4d35706f0ef6173c249a09aa09a851f6f56a92da80fc37f690a70e8
BLAKE2b-256 checksum
How to use checksums
6ffca8088b4214e0403631bcac6f28022b4171b72e916c3c4adbecfe3004570c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.10.4

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page