Skip to main content

Client application to interface with the BranchKey system

Project description

BranchKey Python Client Application

BK_logo

This application runs against the BranchKey backend aggregation service for Federated Learning. It provides a Python interface to login/logout a client, upload files to the system for aggregation, and download aggregated output files.

Install

  • pip install branchkey

Build Instructions

  • To build the dependencies:
    • make setup, or
    • pip install -r requirements.txt
  • To run the tests: make test
    • make test, or
    • python3 -m unittest -v

Usage instructions

  • To use a client:

    import json
    from branchkey.client import Client
    
    credentials = {"leaf_name": "leaf-1",
                   "leaf_id": "46780841-9787-41e6-ac14-e3ee160e158a",
                   "leaf_session_token": "46780841-9787-41e6-ac14-e3ee160e158a",
                   "response_host":"response.branchkey.com"}
    
    host = "https://api.branchkey.com"
    
    proxy_servers = {
          'http': 'http://user:password@proxyserver.com:8080',
          'https': 'http://user:password@proxyserver.com:8080',
          }
    
    '''initialise the client
    it implicitly authenticates the leaf_session
    and fetches the run_details of the parent branch
    
    ssl: Whether to verify the SSL certificates of the
    remote host or not. Default it True
    
    wait_for_run: When trying to upload file, if the
    run is stopped/paused, this parameter decides whether
    to throw exception and stop the process, or wait for
    the run to be started again. Default is False
    
    run_check_interval_s: if wait_for_run=True, this
    parameter decides the sleeping interval of the
    program until the run status is checked again.
    Default is 30 seconds
    '''
    c = Client(credentials,host, ssl=True, wait_for_run=True, run_check_interval_s=15, proxies=proxy_servers)
    
    '''
    upload the file to the system
    '''
    c.file_upload("./file/path.npy")
    
    '''Download a file with the file_id value
    same as the one received from the consumer
    It downloads the files in the ./aggregated_files directory
    '''
    if not c.queue.empty():
          aggregation_id = c.queue.get(block=False)
          c.file_download(aggregation_id)
    
          '''To push performance analysis metrics for this aggregation:
          mode can be test, train or non-federated
          '''
          data = json.dumps({"key1":"val1","key2":"val2"})
          mode = "test"
          c.send_performance_metrics(aggregation_id, data, mode)
    

File format

Weights file in a numpy .npy format:

with open("./test.npy", "wb") as f:
    np.save(f, parameter_array)
[num_samples, [n_d parameter matrix]]
num_samples - the number of samples that contributed to this update
n_d parameter matrix - parameters

Required file format

The required numpy arrays after exports

[1329, list([array([[[[ 1.71775490e-01,    [[[ 8.74867663e-02,  5.19692302e-02, -1.64664671e-01,,          -2.23452481e-03,  1.11475676e-01],,    [-1.75505821e-02, -1...
(1329, [array([[[[ 1.71775490e-01,  3.02851666e-02,  2.90171858e-02,
          -4.27578250e-03,  1.14474617e-01],
         [-8.07138346e-03,  1.44909814e-01, -5.36724664e-02,
          -3.51673253e-02, -1.82426855e-01],
         [ 6.75795972e-02, -1.72839850e-01, -7.25025982e-02,
          -1.59504730e-02,  1.60634145e-01],
         [ 6.62277341e-02, -2.26575769e-02, -1.65369093e-01,
          -8.67117420e-02,  1.80021569e-01],
         [-6.11407161e-02, -1.59245610e-01,  1.45820528e-01,
          -5.40512279e-02, -5.19061387e-02]]],
        ....
         [-1.44068539e-01,  6.15987852e-02,  1.83321223e-01,
          -1.79076958e-02, -1.53445438e-01],
         [-7.76787996e-02,  7.64556080e-02,  9.43044946e-02,
           1.63337544e-01, -1.69042274e-01],
         [-8.55994076e-02, -1.23661250e-01,  1.48442864e-01,
          -1.35983482e-01,  2.05254350e-02]]]], dtype=float32), array([ 0.13065006,  0.12797254, -0.12818147, -0.09621437,  0.04100017,
       -0.07248228,  0.02753541,  0.00476395, -0.11270998,  0.11353076,
       -0.0167569 ,  0.12654744, -0.05019006, -0.07281244,  0.03892357,
       -0.09698197, -0.06845284, -0.04604543, -0.01372138, -0.052395  ,
        0.04833373,  0.16228785,  0.09982517,  0.19556762,  0.10631064,
        0.02496212, -0.14297573, -0.10442089,  0.01970248, -0.1684099 ,
       -0.05076171,  0.19325127], dtype=float32), array([[[[-3.42470817e-02,  8.76816106e-04, -2.13724039e-02,
          -2.62880027e-02, -1.86583996e-02],
         [ 2.56936941e-02, -1.97169576e-02, -3.45735364e-02,
          -4.32738848e-03, -1.22306980e-02],
         [ 8.36322457e-03,  3.26042138e-02, -1.50063485e-02,
          -1.85401291e-02,  2.39207298e-02],
         [-1.15280924e-02, -3.47947963e-02,  2.17274204e-02,
           1.80862695e-02,  2.19682772e-02],
...
etc

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

branchkey-2.6.1.tar.gz (9.8 kB view hashes)

Uploaded Source

Built Distribution

branchkey-2.6.1-py3-none-any.whl (8.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page