Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Python wrapper for the Xenon API.

Project Description

Python interface to the Xenon middleware library, v. 2.0. Xenon provides a simple programming interface to various pieces of software that can be used to access distributed compute and storage resources.

Underneath it uses GRPC, to connect to the Xenon-GRPC service. We’ve taken care to mirror the original Java API in this Python module as much as possible.

Installing

Clone this repository, and do:

pip install .

The code will appear on PyPI when it is ready for release.

Documentation

The compiled documentation is hosted on Read the Docs. This includes a quick-start guide.

Development

PyXenon ships with the Xenon-GRPC jar-file and command-line executable. If these need upgrading, build them manually, following instructions at Xenon-GRPC, and place the contents of the build/install/xenon-grpc-shadow folder (lib and bin) here.

To generate the GRPC code, run scripts/protoc.sh from the project root.

Testing

Unit tests all run against the local scheduler and the file adaptor for filesystems. To run them, just do:

$ pytest ./tests

For faster testing it may be useful to start the xenon-grpc daemon manually; start it in a separate terminal as it may give useful output for debugging.

For integration testing, run the following docker container to test against remote slurm

docker run --detach --publish 10022:22 nlesc/xenon-slurm:17

An example of some code running against this container is in examples/tutorial.py.

Example

import xenon
import os

xenon.init()

# create a new job scheduler, using SSH to localhost to submit new jobs.
with xenon.Scheduler.create(
        adaptor='ssh', location='localhost') as scheduler:

    # make a new job description. The executable must already be present on
    # the target host.
    target = xenon.Path('.') / 'stdout.txt'
    desc = xenon.JobDescription(
        executable='hostname',
        stdout=str(target))

    # submit a job
    job = scheduler.submit_batch_job(desc)
    status = scheduler.wait_until_done(job, 1000)

    # read the standard output of the job. We can do this directly because
    # we ran on localhost, otherwise, we need to transfer the file first.
    with open(target) as f:
        print(f.read())

Contributing

Contributions can be made using GitHub pull requests. To add a feature, first install the test requirements

pip install -U tox

and then run

tox

until all tests succeed. The command checks against flake8 code standards and syntax errors on Python 3.5 and 3.6. Then commit, to make sure the change didn’t break any code. The pull request will be evaluated in Travis.

Release History

Release History

This version
History Node

2.2

History Node

0.3

History Node

0.2.2

History Node

0.2.1

History Node

0.2.0

History Node

0.1.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
pyxenon-2.2.tar.gz (19.6 MB) Copy SHA256 Checksum SHA256 Source Nov 6, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting