Skip to main content

(Python) Process and File Controller

Project description

https://badge.fury.io/py/pfcon.svg https://travis-ci.org/FNNDSC/pfcon.svg?branch=master https://img.shields.io/badge/python-3.5%2B-blue.svg

Overview

This repository provides pfcon – a controlling service that speaks to remote pman and pfioh services.

pfcon

Most simply, pfcon pushes local data to a remote location (by talking to a remote pfioh service), runs some process on this data in the remote space using pman, and then copies the resultant data back to a local target space.

It can be used to query and control the following (for example):

  • state: Is job <XYZ> still running?

  • result: What is the stdout (or stderr) from job <XYZ>?

  • control: Kill job <XYZ>

Installation

Installation is relatively straightforward, and we recommend using either python virtual environments or docker.

Python Virtual Environment

On Ubuntu, install the Python virtual environment creator

sudo apt install virtualenv

Then, create a directory for your virtual environments e.g.:

mkdir ~/python-envs

You might want to add to your .bashrc file these two lines:

export WORKON_HOME=~/python-envs
source /usr/local/bin/virtualenvwrapper.sh

Then you can source your .bashrc and create a new Python3 virtual environment:

source .bashrc
mkvirtualenv --python=python3 python_env

To activate or “enter” the virtual env:

workon python_env

To deactivate virtual env:

deactivate

Using the fnndsc/ubuntu-python3 dock

We provide a slim docker image with python3 based off Ubuntu. If you want to play inside this dock and install pman manually, do

docker pull fnndsc/ubuntu-python3

This docker has an entry point python3. To enter the dock at a different entry and install your own stuff:

docker run -ti --entrypoint /bin/bash fnndsc/ubuntu-python3

Now, install pman and friends using pip

apt update && \
apt install -y libssl-dev libcurl4-openssl-dev librtmp-dev && \
pip install pfcon

If you do the above, remember to commit your changes to the docker image otherwise they’ll be lost when you remove the dock instance!

 docker commit <container-ID> local/ubuntu-python3-pfcon

where ``<container-ID>`` is the ID of the above container.

Using the fnndsc/pfcon dock

The easiest option however, is to just use the fnndsc/pfcon dock.

docker pull fnndsc/pfcon

and then run

docker run --name pfcon -v /home:/Users --rm -ti fnndsc/pfcon --forever --httpResponse

Usage

For usage of pfcon, consult the relevant wiki pages.

pfcon usage

For pfcon detailed information, see the pfcon wiki page.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pfcon-1.2.2.tar.gz (32.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page