Skip to main content

README

The Molgenis EMX2 Pyclient is a Python package developed to be used for data management on Molgenis EMX2 servers. A detailed overview of the capabilities is presented in the MOLGENIS documentation.

Installation

The releases of the package are hosted at PyPI. The recommended way to install the latest version is through pip.

pip install molgenis-emx2-pyclient

Changelog

Releases of the Molgenis EMX2 Pyclient follow the release number of the accompanying release of the Molgenis EMX2 software. Therefore, releases of the Pyclient are less frequent than those of EMX2 and the latest version of the Pyclient may differ from the latest version of Molgenis EMX2.

13.204.1

  • Improved preparing CSV data for reading as DataFrame

13.196.1

  • Fixed pandas package dependency
  • Improved formatting of source files

13.187.2

  • Fixed input check for delete_records
  • Added utility functions validate_graphql_response and response_to_dataframe

13.165.2

  • Fixed compatibility issue with pandas v2

13.162.0

  • Added option parse_arrays to get, which parses array-type data to Python lists
  • Added helper functions for csv array to/from Python list conversion
  • Fixed issue with reading columns of types LONG and LONG_ARRAY

13.158.2

  • Fixed problem with accidental conversion of integers to floats when null values are present
  • Add new helper function for data-to-CSV-output

13.84.2

  • Added: method export_schema for exporting a schema's metadata

13.77.1

  • Improved error handling in 'columns' filter in get method
  • Fixed asynchronous API calls in export
  • Fixed issue with 'equals' filter for references in get method
  • Fixed greater_than and smaller_than filters for LONG type values in get method
  • Renamed __prepare_unequal_filter to prepare_not_equals_filter and added NotImplementedError for certain data types
  • Fixed issue with incorrect column names in get_graphql
  • Removed default value None for 'name' in create_schema as schema name cannot be null
  • Fixed parameter include_demo_data in recreate_schema to default to False instead of None
  • Improved error handling when creating schema with non-existent template

13.75.1

  • Fixed: issue when running get_schema_metadata on a schema with insufficient permissions to see members
  • Added: methods get_schema_settings
  • Added: methods get_schema_members
  • Added: methods get_schema_roles

13.55.4

Added: method save_table. Replaces save_schema which is now deprecated.

13.50.0

Fixed: issue with loading table data from get_graphql method

13.40.4

Fixed: no longer convert NA-like strings to pandas' NaN, only do this for missing values (i.e. empty strings).

11.61.2

Fixed: problem where the schema metadata could not be updated with upload_file and a molgenis.csv file.

11.57.0

  • Added: feature 'truncate' to remove all entries from a table
  • Added: option to filter results of get method by columns
  • Added: method get_graphql implements the GraphQL API
  • Improved: added additional parsing for data returned from the CSV API to pandas DataFrame in get method
  • Fixed: log level was set to DEBUG without possibility to change this. The user can now set the log level again at their preferred level

11.47.1

Fixed: updated GraphQL queries to be in line with EMX2 database metadata

11.23.0

Added: an optional job argument to the Client initialization, allowing the Pyclient to run asynchronous methods within a job in EMX2."

11.11.1

Added: option to specify filename in method export and to return the exported data in a function

11.8.0

Breaking: introduced asynchronous methods. Users need to explicitly address the asynchronous methods

10.109.3

Added: the ability to use the Pyclient on an EMX2 instance running on a local machine.

10.98.0

Added: the option to filter the results obtained from the get method based on columns

10.92.1

Added: the method upload_file which allows the direct upload of files of multiple types to the API

10.72.1

Fixed: issue where NA values were not properly read in or returned by the API

10.48.9

Fixed: issue in managing data of a table where the table id differs from the table name

10.48.0

Added: option to save data from the get method as a pandas DataFrame

10.47.2

Fixed: exceptions for metadata classes Column and Schema

10.47.0

Added: metadata classes for Column, Table, Schema

10.13.1

Added: methods create_schema, delete_schema, update_schema, recreate_schema, get_schema_metadata

How to use

Within your Python project import the class Client and instantiate it as a context manager. Operations and queries can then be executed from within the context.

from molgenis_emx2_pyclient import Client

username = 'username'
password = '...'

with Client('https://example.molgeniscloud.org') as client:
    client.signin(username, password)

    # Retrieve signin information
    print(client.status)
    """ Output:
    Host: https://example.molgeniscloud.org
    Status: Signed in
    Schemas:
        CatalogueOntologies
        catalogue
        ExampleSchema
        ...
    Version: v10.10.1
    """

    # Retrieve data from a table on a schema
    data = client.get(schema='ExampleSchema', table='Cohorts')

    # Create a new schema on the server
    client.create_schema(name='New Schema')

    # Delete a schema from the server
    client.delete_schema(name='New Schema')

Instead of signing in with a username and password the client can also be used while authorized by a (temporary) token that is generated on the server. See the MOLGENIS documentation for generating tokens

from molgenis_emx2_pyclient import Client

token = '...'

with Client('https://example.molgeniscloud.org', token=token) as client:

    # Retrieve signin information
    print(client.status)
    """ Output:
    Host: https://example.molgeniscloud.org
    User: token
    Status: session-less
    Schemas:
        CatalogueOntologies
        catalogue
        ExampleSchema
        ...
    Version: v10.32.1
    """

    ...
    ...

Development

The Pyclient requires a Python installation with version 3.10 or higher. Clone the molgenis-emx2 repository from GitHub

git clone git@github.com:molgenis/molgenis-emx2.git

Change the working directory to .../tools/pyclient

Create a virtual Python environment

On macOS:

python -m venv venv

On Linux:

python3.10 -m venv venv

On Windows:

py -3.10 venv venv

Activate the virtual environment

On macOS and Linux:

source venv/bin/activate

On Windows:

.venv\Scripts\activate.bat

Install the script dependencies

pip install -r requirements.txt

Build

Before building the source, the package build needs to be installed.

(venv) $ pip install build

(venv) $ python -m build

(venv) $ pip install dist/molgenis_emx2_pyclient*.whl

Test

Tests for the Pyclient have been created with the Pytest framework. In order to test the functionality of the Pyclient copy the '.env-example' file to '.env' and modify the parameters for the server you want to test on.

Then execute the following to run all the tests

(venv) molgenis-emx2/tools/pyclient$ pytest

Metadata

Release files for molgenis-emx2-pyclient 13.204.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for molgenis-emx2-pyclient 13.204.1
File Size Uploaded
molgenis_emx2_pyclient-13.204.1.tar.gz 33.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for molgenis-emx2-pyclient 13.204.1
File Interpreter ABI Platform
molgenis_emx2_pyclient-13.204.1-py3-none-any.whl Python 3 none any Details

Total release size: 59.3 kB

Release files / molgenis_emx2_pyclient-13.204.1.tar.gz

Download URL molgenis_emx2_pyclient-13.204.1.tar.gz
Size 33.9 kB
Tags Source
SHA-256 checksum
How to use checksums
2be0efd995a23e75e3408ad82383e57e2ee9127668744e9676b4545e146f5850
BLAKE2b-256 checksum
How to use checksums
09ba7b2018a8d058f5496e0051598b6c4be84515aafb7626074f2482a7df616c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.3

Release files / molgenis_emx2_pyclient-13.204.1-py3-none-any.whl

Download URL molgenis_emx2_pyclient-13.204.1-py3-none-any.whl
Size 25.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6a760cfec33f0568942fccb1fd2d2d173d6ea5ea9ddb7328c5005b7f7dd02308
BLAKE2b-256 checksum
How to use checksums
e08f14cb744a23b06002cf45322ae6c94daf1e1207ee4ed3583cb4b0c557dec7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.3

Release history Release notifications | RSS feed

This release

13.204.1 This release

2 release files

0.2.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page