oar-lib 0.4.1

Python OAR Library

Python OAR Common Library

Python version of OAR Common Library

• Free software: BSD license

• Documentation: https://oar-lib.readthedocs.org.

Read Configuration

>>> from oar import config
>>>
>>> config.load_file("/path/to/config")
True
>>> config["ENERGY_SAVING_INTERNAL"]
'no'
>>> config.get_namespace('OARSUB_')
{'default_resources': '/resource_id=1', 'force_job_key': 'no', 'nodes_resources': 'network_address'}

Query the database

>>> from oar import db, Resource
>>> db
<Database engine=None>

>>> db.query(Resource).first()
<oar.models.Resource object at 0x2becb10>
>>> Resource.query.first()
<oar.models.Resource object at 0x2becb10>

>>> db
<Database engine=Engine(postgresql://oar:***@server:5432/oar)>

>>> for r in Resource.query.filter(Resource.core > 3).limit(2):
...     print(r.id, r.network_address)
...
(12L, u'node3')
(9L, u'node3')

>>> req = db.query(Resource.id, Resource.network_address)
>>> req.filter(Resource.core > 3).limit(2).all()
[(12L, u'node3'), (9L, u'node3')]

Logger

Logging from the root logger:

>> from oar import logger as log
>> log.info("Init confiuration")
## [INFO] [2014-12-05 14:13:42,239] [oar]: Init confiuration

Logging from your own sublogger:

>> from oar import get_logger
>> log = get_logger("oar.kao")
>> log.error("fatal error")
## [ERROR] [2014-12-05 14:14:32,116] [oar.kao]: fatal error

OAR lib changes

Version 0.4.1

Released on September 30th 2015

• [all] Switched the license to BSD

• [all] Dropped python 3.3 support

• [all] Little code refactoring

• [database] Supported session read-only mode via oar_reader user

• [database] engine.execute is not used directly anymore, prefer session.execute

• [database] Used the same database connection with alembic and sqlalchemy

• [database] Added ephemeral session context manager that will rollback the transaction at the end

• [database] Added db.show() method to quickly show the database content

• [database] Used truncate instead of delete on postgresql in db.delete_all()

• [database] db.close() method closes the session and disposes the engine

• [database] Fixed db.close method by removing deadlock

• [database] Raised KeyError exception from Database if wanted Model or Table are missing

• [database] Configured automatically default table args with DeclarativeMeta class

• [database] Created DeferredReflectionModel metaclass per Database instance to avoid conflict during database reflection

• [database] Collected declared tables and models when they are defined

• [database] Avoided circular import of db object

• [database] Generated a tablename if none specified

• [models] The __repr__ method in Model classes include the identity of the object (primary key)

• [models] Set column Resource.scheduler_priority to BigInteger

• [models] Added BaseModel.to_json method() method to convert a sqlalchemy entity to json

• [models] Added BaseModel.to_dict() method to convert a sqlalchemy entity to a dictionary

• [models] Fixed BaseModel.create() method

• [utils] Added ability to run command without timeout

• [utils] Used decimal module to check if input is a number

• [utils] Reset ResultProxyIter iterator automatically

• [utils] Fixed ResultProxyIter rowcount

• [utils] Reset cached_property attribut when deleted

• [utils] Added merge_dicts() that merge given dictionaries into a new dictionary

• [basequery] Updated Query.render() method to return a special string that have useful __repr__

• [basequery] Added BaseQuery.get_jobs_for_user() query

• [test] Tested on postgresql and sqlite-file/memory with tox and travis.ci

• [test] Added test helper scripts to populate database

• [test] Fixed tests on python 3.4

• [test] More tests

• [compat] Removed unused SimpleNamespace class from compat module

Version 0.3.0

Released on July 07th 2015

• [database] Fixed Database.delete_all method to remove all database content

• [database] Listed all datetime columns in models.TIME_COLUMNS

• [database] Made table and model import easier with getitem syntax (Eg. db[‘table_name’])

• [database] Added an alembic operator as Database.op attribut

• [database] Kept columns order during dictionary conversion

• [database] Put MySQLdb SSCursor tweak on standby

• [models] Added ResultProxyIter class that make SQLAlchemy ResultProxies iterable by dicts

• [models] Fixed columns orders for admission_rules table

• [models] models.all_tables returns a dictionary with table names as keys now

• [models] Forwarded log to STDOUT if get_logger(..) receive stdout=True

• [configuration] New default configuration values for DB_PORT=5432 and DB_TYPE=Pg

• [configuration] Handled OARCONFFILE environment variable to load OAR configuration

• [basequery] Added Query.render that generate an SQL expression string from statement

• [basequery] Added get_gantt_visu_scheduled_jobs_resources method that returns all nodes allocated to a (waiting) reservation

• [basequery] Added get_assigned_jobs_resources and groups results by job_id

• [basequery] Optimized get_user_jobs query by using JOIN instead of IN operator

• [basequery] Loaded only some columns with Load ORM object

• [utils] Added utils.Command class to run subprocess commands with a timeout option

• [utils] Moved JSONEncoder class from oar-rest-api to oar-lib

• [utils] Removed unsed IterStream class

• [utils] Added utils.row2dict function helpers to convert a RowProxy to a dict

• [compat] Used simplejson if available

Version 0.2.0

Released on June 23rd 2015

• [config] Added clear parameter to clear the config before loading a new file

• [config] only one default configuration file

• [config] Made load file configuration atomic

• [config] Warned user when configuration loading failed

• [compat] Used iterator version of zip and range method

• [compat] Removed unused string_types

• [database] pg_bulk_insert that use COPY clause to perform batch inserts

• [database] Added db.queries object that included all oar-lib sql queries

• [database] Workaround to support table inheritance and __table_args__

• [database] Moved the BaseQuery class to the basequery module

• [models] Added missings relations between tables as dicts

• [models] Used BigInteger type on Accounting fields

• [models] Added all_tables method to get all tables

• [basequery] make models module easier to read

• [basequery] Added get_job_resources query

• [basequery] Added filter_jobs_for_user method to build jobs query

• [basequery] Added get_resources

• [basequery] Added get_jobs_for_user query

• [utils] Added IterStream class that give a stream like interface for any iterator

• Added alembic and sqlalchemy-utils requirements

Version 0.1.1

Released on April 30th 2015

• Minor bugfixes

Version 0.1.0

Released on April 21st 2015

First public release of oar-lib