oarepo-validate 1.0.0a1

OArepo Validate library for record metadata validation

oarepo-validate

OArepo Validate library for model-level matedata validation

Installation

    pip install oarepo-validate

Usage

The library provides mixins for enforcing json schema and marshmallow validation.

JSON schema validation

If $schema is present on metadata, invenio performs a json schema validation inside the validate() method. The problem is that $schema can be set/removed via the REST API. This means that an ill-written client can completely bypass the validation.

To mitigate this issue, create your own Record implementation:

from oarepo_validate import SchemaKeepingRecordMixin
from invenio_records import Record

class MyRecord(SchemaKeepingRecordMixin, Record):
    ALLOWED_SCHEMAS = ('records/record-v1.0.0.json', 'records/record-v2.0.0.json')
    PREFERRED_SCHEMA = 'records/record-v2.0.0.json'

And register the record in REST endpoints in configuration:

RECORD_PID = 'pid(recid,record_class="my:MyRecord")'

RECORDS_REST_ENDPOINTS = {
    'records': dict(
        pid_type='recid',
        pid_minter='recid',
        pid_fetcher='recid',
        record_class='my:MyRecord',
        item_route='/records/<{0}:pid_value>'.format(RECORD_PID),
        # ...
    )
}

Create record

When creating a new record, if $schema is not set, MyRecord.PREFERRED_SCHEMA is added automatically. If $schema is set, it is validated against MyRecord.ALLOWED_SCHEMAS and an exception is raised if the schema is not present in ALLOWED_SCHEMAS.

PUT / PATCH record

Before the result of the operation is committed, $schema is checked again.

Marshmallow validation

In invenio, REST create operation use the following sequence:

<flask>
<invenio_records_rest.views.RecordsListResource:post>
   <loader>
      <marshmallow>
   <permission factory>
   <pid minter>
   <record_class.create>
      <record.commit>
         <record.validate>

REST PUT operation then uses:

<flask>
<invenio_records_rest.views.RecordResource:put>
   <permission factory>
   <loader>
      <marshmallow>
   <record.update>
   <record.commit>
      <record.validate>

REST PATCH operation:

<flask>
<invenio_records_rest.views.RecordResource:put>
   <permission factory>
   <simple json loader>
   <record.patch>
   <record.commit>
      <record.validate>

As you can see, if you place any validation code in loader's marshmallow, it is not executed. An alternative is to have the validation code in validate and handle all validations there. This library does exactly this - it provides a record mixin that calls marshmallow schema's load method inside its validate method.

Usage

Create your own record and inherit from the mixin:

from oarepo_validate import MarshmallowValidatedRecordMixin
from invenio_records import Record
from marshmallow import Schema, fields

class TestSchema(Schema):
    name = fields.Str(required=True)

class MyRecord(MarshmallowValidatedRecordMixin, Record):
    MARSHMALLOW_SCHEMA = TestSchema

Do not forget to register it as in the previous example.

Now marshmallow schema will be processed before each commit method.

What about marshmallow in loader?

In most cases, marshmallow schema in loader can be removed and a simple json loader used instead. However, if you need a custom processing of input data that is independent of validation, you can keep the two marshmallows.

A special case is when the marshmallow in loader already includes validation marshmallow rules. Then you would want to use loader's marshmallow for create / replace and marshmallow in validation only for patch operation (so that the same marshmallow rules are not called twice). To accomplish this, set:

class MyRecord(MarshmallowValidatedRecordMixin, Record):
    MARSHMALLOW_SCHEMA = TestSchema

    VALIDATE_MARSHMALLOW = False
    VALIDATE_PATCH = True

VALIDATE_MARSHMALLOW will switch off marshmallow validation in validate method and VALIDATE_PATCH will switch on marshmallow validation in patch method.

Changes

Version 0.1.0 (released TBD)

• Initial public release.