No project description provided
Project description
===============
python-grid5000
===============
``python-grid5000`` is a python package wrapping the Grid’5000 REST API.
.. warning::
The code is currently being developed heavily. Jump to the contributing section
if you want to be involved.
1 Thanks
--------
The core code is borrowed from `python-gitlab <https://github.com/python-gitlab/python-gitlab>`_ with small adaptations to
conform with the Grid5000 API models (with an ’s’!)
2 Contributing
--------------
- To contribute, you can drop me an email or open an issue for a bug report, or feature request.
- There are many areas where this can be improved some of them are listed here:
- The complete coverage of the API isn’t finished (yet) but this should be fairly easy to reach.
Most of the logic go in ```grid5000.objects`` <https://gitlab.inria.fr/msimonin/python-grid5000/blob/master/grid5000/objects.py>`_. And to be honnest I only
implemented the feature that I needed the most.
- Returned `status code <https://www.grid5000.fr/mediawiki/index.php/API#Status_Codes>`_ aren’t yet well treated.
3 Comparison with ...
---------------------
- `RESTfully <https://api.grid5000.fr/doc/4.0/tools/restfully.html>`_:
It consumes REST API following the `HATEOAS <https://en.m.wikipedia.org/wiki/HATEOAS>`_ principles. This allows the client to
fully discover the resources and actions available. Most of the G5K API follow
theses principles but, for instance the `Storage API <https://www.grid5000.fr/mediawiki/index.php/Storage_Manager>`_ don’t. Thus RESTfully
isn’t compatible with all the features offered by the Grid’5000 API.
- `Execo <http://execo.gforge.inria.fr>`_:
Written in Python. The api module gathers a lot of utils functions leveraging
the Grid’5000 API. Resources aren’t exposed in a syntax friendly manner,
instead functions for some classical operations are exposed (mainly getters).
It has a convenient way of caching the reference API.
- `Raw requests <http://docs.python-requests.org>`_:
**The** reference for HTTP library in python. Good for prototyping but low-level.
python-grid5000 encapsulates this library.
4 Examples
----------
- Please refer to `https://api.grid5000.fr/doc/4.0/reference/spec.html <https://api.grid5000.fr/doc/4.0/reference/spec.html>`_ for
the complete specification.
- All the examples are exported in the examples subdirectory so you can
easily test and adapt them.
- The configuration is read from a configuration file located in the home
directory (should be compatible with the restfully one).
It can be created with the following:
::
echo '
username: MYLOGIN
password: MYPASSWORD
' > ~/.python-grid5000.yaml
4.1 Get node information
~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
node_info = gk.sites["nancy"].clusters["grisou"].nodes["grisou-1"]
print("grisou-1 has {threads} threads and has {ram} bytes of RAM".format(
threads=node_info.architecture["nb_threads"],
ram=node_info.main_memory["ram_size"]))
4.2 Job filtering
~~~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
# state=running will be placed in the query params
running_jobs = gk.sites["rennes"].jobs.list(state="running")
print(running_jobs)
# get a specific job by its uid
job = gk.sites["rennes"].jobs.get("424242")
print(job)
4.3 Submit a job
~~~~~~~~~~~~~~~~
.. code:: python
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
# This is equivalent to gk.sites.get("rennes")
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
})
while job.state != "running":
job.refresh()
print("Waiting for the job to be running")
time.sleep(10)
print(job)
print("Assigned nodes : %s" % job.assigned_nodes)
4.4 Deploy an environment
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
# This is equivalent to gk.sites.get("rennes")
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"types": ["deploy"]
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be running" % job.uid)
time.sleep(10)
print("Assigned nodes : %s" % job.assigned_nodes)
deployment = site.deployments.create({"nodes": job.assigned_nodes,
"environment": "debian9-x64-min"})
while deployment.status != "terminated":
deployment.refresh()
print("Waiting for the deployment [%s] to be finished" % deployment.uid)
time.sleep(10)
print(deployment.result)
4.5 Get Storage accesses
~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
print(gk.sites["rennes"].storage["msimonin"].access.list())
4.6 Set storage accesses (e.g for vms)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
from netaddr import IPNetwork
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"resources": "slash_22=1+nodes=1"
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be running" % job.uid)
time.sleep(5)
subnet = job.resources_by_type['subnets'][0]
ip_network = [str(ip) for ip in IPNetwork(subnet)]
# create acces for all ips in the subnet
access = site.storage["msimonin"].access.create({'ipv4': ip_network,
'termination': {"job": job.uid,
"site": site.uid}})
4.7 Get vlan(s)
~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
# Get all vlans
vlans = site.vlans.list()
print(vlans)
# Get on specific
vlan = site.vlans.get("4")
print(vlan)
vlan = site.vlans["4"]
print(vlan)
# Get vlan of some nodes
print(site.vlansnodes.submit({"nodes": ["paravance-1.rennes.grid5000.fr", "paravance-2.rennes.grid5000.fr"]}))
# Get nodes in vlan
print(site.vlans["4"].nodes.list())
4.8 TODO Set nodes in vlan
~~~~~~~~~~~~~~~~~~~~~~~~~~
- Putting primary interface in a vlan
.. code:: python
from netaddr import IPNetwork
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"resources": "{type='kavlan'}/vlan=1+nodes=1",
"types": ["deploy"]
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be runnning" % job.uid)
time.sleep(5)
deployment = site.deployments.create({"nodes": job.assigned_nodes,
"environment": "debian9-x64-min",
"vlan": job.resources_by_type["vlans"][0]})
while deployment.status != "terminated":
deployment.refresh()
print("Waiting for the deployment [%s] to be finished" % deployment.uid)
time.sleep(10)
print(deployment.result)
- Putting the secondary interface in a vlan
.. code:: python
from netaddr import IPNetwork
import os
import time
from grid5000 import Grid5000
def _to_network_address(host, interface):
"""Translate a host to a network address
e.g:
paranoia-20.rennes.grid5000.fr -> paranoia-20-eth2.rennes.grid5000.fr
"""
splitted = host.split('.')
splitted[0] = splitted[0] + "-" + interface
return ".".join(splitted)
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"resources": "{type='kavlan'}/vlan=1+{cluster='paranoia'}nodes=1",
"types": ["deploy"]
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be runnning" % job.uid)
time.sleep(5)
vlanid = job.resources_by_type["vlans"][0]
# we hard code the interface but this can be discovered in the node info
# TODO: write the code here to discover
nodes = [_to_network_address(n, "eth2") for n in job.assigned_nodes]
print(nodes)
# set in vlan
site.vlans[vlanid].submit({"nodes": nodes})
python-grid5000
===============
``python-grid5000`` is a python package wrapping the Grid’5000 REST API.
.. warning::
The code is currently being developed heavily. Jump to the contributing section
if you want to be involved.
1 Thanks
--------
The core code is borrowed from `python-gitlab <https://github.com/python-gitlab/python-gitlab>`_ with small adaptations to
conform with the Grid5000 API models (with an ’s’!)
2 Contributing
--------------
- To contribute, you can drop me an email or open an issue for a bug report, or feature request.
- There are many areas where this can be improved some of them are listed here:
- The complete coverage of the API isn’t finished (yet) but this should be fairly easy to reach.
Most of the logic go in ```grid5000.objects`` <https://gitlab.inria.fr/msimonin/python-grid5000/blob/master/grid5000/objects.py>`_. And to be honnest I only
implemented the feature that I needed the most.
- Returned `status code <https://www.grid5000.fr/mediawiki/index.php/API#Status_Codes>`_ aren’t yet well treated.
3 Comparison with ...
---------------------
- `RESTfully <https://api.grid5000.fr/doc/4.0/tools/restfully.html>`_:
It consumes REST API following the `HATEOAS <https://en.m.wikipedia.org/wiki/HATEOAS>`_ principles. This allows the client to
fully discover the resources and actions available. Most of the G5K API follow
theses principles but, for instance the `Storage API <https://www.grid5000.fr/mediawiki/index.php/Storage_Manager>`_ don’t. Thus RESTfully
isn’t compatible with all the features offered by the Grid’5000 API.
- `Execo <http://execo.gforge.inria.fr>`_:
Written in Python. The api module gathers a lot of utils functions leveraging
the Grid’5000 API. Resources aren’t exposed in a syntax friendly manner,
instead functions for some classical operations are exposed (mainly getters).
It has a convenient way of caching the reference API.
- `Raw requests <http://docs.python-requests.org>`_:
**The** reference for HTTP library in python. Good for prototyping but low-level.
python-grid5000 encapsulates this library.
4 Examples
----------
- Please refer to `https://api.grid5000.fr/doc/4.0/reference/spec.html <https://api.grid5000.fr/doc/4.0/reference/spec.html>`_ for
the complete specification.
- All the examples are exported in the examples subdirectory so you can
easily test and adapt them.
- The configuration is read from a configuration file located in the home
directory (should be compatible with the restfully one).
It can be created with the following:
::
echo '
username: MYLOGIN
password: MYPASSWORD
' > ~/.python-grid5000.yaml
4.1 Get node information
~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
node_info = gk.sites["nancy"].clusters["grisou"].nodes["grisou-1"]
print("grisou-1 has {threads} threads and has {ram} bytes of RAM".format(
threads=node_info.architecture["nb_threads"],
ram=node_info.main_memory["ram_size"]))
4.2 Job filtering
~~~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
# state=running will be placed in the query params
running_jobs = gk.sites["rennes"].jobs.list(state="running")
print(running_jobs)
# get a specific job by its uid
job = gk.sites["rennes"].jobs.get("424242")
print(job)
4.3 Submit a job
~~~~~~~~~~~~~~~~
.. code:: python
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
# This is equivalent to gk.sites.get("rennes")
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
})
while job.state != "running":
job.refresh()
print("Waiting for the job to be running")
time.sleep(10)
print(job)
print("Assigned nodes : %s" % job.assigned_nodes)
4.4 Deploy an environment
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
# This is equivalent to gk.sites.get("rennes")
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"types": ["deploy"]
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be running" % job.uid)
time.sleep(10)
print("Assigned nodes : %s" % job.assigned_nodes)
deployment = site.deployments.create({"nodes": job.assigned_nodes,
"environment": "debian9-x64-min"})
while deployment.status != "terminated":
deployment.refresh()
print("Waiting for the deployment [%s] to be finished" % deployment.uid)
time.sleep(10)
print(deployment.result)
4.5 Get Storage accesses
~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
print(gk.sites["rennes"].storage["msimonin"].access.list())
4.6 Set storage accesses (e.g for vms)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: python
from netaddr import IPNetwork
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"resources": "slash_22=1+nodes=1"
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be running" % job.uid)
time.sleep(5)
subnet = job.resources_by_type['subnets'][0]
ip_network = [str(ip) for ip in IPNetwork(subnet)]
# create acces for all ips in the subnet
access = site.storage["msimonin"].access.create({'ipv4': ip_network,
'termination': {"job": job.uid,
"site": site.uid}})
4.7 Get vlan(s)
~~~~~~~~~~~~~~~
.. code:: python
import os
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
# Get all vlans
vlans = site.vlans.list()
print(vlans)
# Get on specific
vlan = site.vlans.get("4")
print(vlan)
vlan = site.vlans["4"]
print(vlan)
# Get vlan of some nodes
print(site.vlansnodes.submit({"nodes": ["paravance-1.rennes.grid5000.fr", "paravance-2.rennes.grid5000.fr"]}))
# Get nodes in vlan
print(site.vlans["4"].nodes.list())
4.8 TODO Set nodes in vlan
~~~~~~~~~~~~~~~~~~~~~~~~~~
- Putting primary interface in a vlan
.. code:: python
from netaddr import IPNetwork
import os
import time
from grid5000 import Grid5000
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"resources": "{type='kavlan'}/vlan=1+nodes=1",
"types": ["deploy"]
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be runnning" % job.uid)
time.sleep(5)
deployment = site.deployments.create({"nodes": job.assigned_nodes,
"environment": "debian9-x64-min",
"vlan": job.resources_by_type["vlans"][0]})
while deployment.status != "terminated":
deployment.refresh()
print("Waiting for the deployment [%s] to be finished" % deployment.uid)
time.sleep(10)
print(deployment.result)
- Putting the secondary interface in a vlan
.. code:: python
from netaddr import IPNetwork
import os
import time
from grid5000 import Grid5000
def _to_network_address(host, interface):
"""Translate a host to a network address
e.g:
paranoia-20.rennes.grid5000.fr -> paranoia-20-eth2.rennes.grid5000.fr
"""
splitted = host.split('.')
splitted[0] = splitted[0] + "-" + interface
return ".".join(splitted)
conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
gk = Grid5000.from_yaml(conf_file)
site = gk.sites["rennes"]
job = site.jobs.create({
"name":"pyg5k",
"command": "sleep 3600",
"resources": "{type='kavlan'}/vlan=1+{cluster='paranoia'}nodes=1",
"types": ["deploy"]
})
while job.state != "running":
job.refresh()
print("Waiting the job [%s] to be runnning" % job.uid)
time.sleep(5)
vlanid = job.resources_by_type["vlans"][0]
# we hard code the interface but this can be discovered in the node info
# TODO: write the code here to discover
nodes = [_to_network_address(n, "eth2") for n in job.assigned_nodes]
print(nodes)
# set in vlan
site.vlans[vlanid].submit({"nodes": nodes})
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for python_grid5000-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9f86ecc880247be308187dbc7279a2eadd4e2070bb017fb1eb9dcf96052e0d91 |
|
MD5 | b427b986576096266d089d6b74676cf3 |
|
BLAKE2b-256 | 09fb8626d58cd8b6603e6c78095914231dfeb83faf791515b692fe4de3b49797 |