halogen 2.0.0

Python HAL generation/parsing library

halogen

Python HAL generation/parsing library.

Halogen takes the advantage of the declarative style serialization with easily extendable schemas. Schema combines the knowledge about your data model, attribute mapping and advanced accessing, with complex types and data transformation.

Library is purposed in representing your data in HAL format in the most obvious way possible, but also of the generic web form-like functionality so that your schemas and types can be reused as much as possible.

Schema

Schema is the main building block of the serialization. It is also a type which means you can declare nested structures with schemas.

Serialization

>>> Schema.serialize({"hello": "Hello World"})
>>> {"hello": "Hello World"}

Simply call Schema.serialize() class method which can accept dict or any other object.

Validation

There’s no validation involved in the serialization. Your source data or your model is considered to be clean since it is coming from the storage and it is not a user input. Of course exceptions in the types or attribute accessors may occur but they are considered as programming errors.

Serializing dict

Dictionary values are automatically accessed by the schema attributes using their names as keys:

import halogen

class Hello(halogen.Schema):
    hello = halogen.Attr()


serialized = Hello.serialize({"hello": "Hello World"})

Result:

{
    "hello": "Hello World"
}

HAL is just JSON, but according to it’s specification it SHOULD have self link to identify the serialized resource. For this you should use HAL-specific attributes and configure the way the self is composed.

HAL example:

import halogen
from flask import url_for

spell = {
    "uid": "abracadabra",
    "name": "Abra Cadabra",
    "cost": 10,
}

class Spell(halogen.Schema):

    self = halogen.Link(attr=lambda spell: url_for("spell.get" uid=spell['uid']))
    name = halogen.Attr()

serialized = Spell.serialize(spell)

Result:

{
    "_links": {
        "self": {"href": "/spells/abracadabra"}
    },
    "name": "Abra Cadabra"
}

Serializing objects

Similar to dictionary keys the schema attributes can also access object properties:

import halogen
from flask import url_for

class Spell(object):
    uid = "abracadabra"
    name = "Abra Cadabra"
    cost = 10

spell = Spell()

class SpellSchema(halogen.Schema):
    self = halogen.Link(attr=lambda spell: url_for("spell.get" uid=spell.uid))
    name = halogen.Attr()

serialized = SpellSchema.serialize(spell)

Result:

{
    "_links": {
        "self": {"href": "/spells/abracadabra"}
    },
    "name": "Abra Cadabra"
}

Attribute

Attributes form the schema and encapsulate the knowledge how to get the data from your model, how to transform it according to the specific type.

Attr()

The name of the attribute member in the schema is the name of the key the result will be serialized to. By default the same attribute name is used to access the source model.

Example:

import halogen
from flask import url_for

class Spell(object):
    uid = "abracadabra"
    name = "Abra Cadabra"
    cost = 10

spell = Spell()

class SpellSchema(halogen.Schema):
    self = halogen.Link(attr=lambda spell: url_for("spell.get" uid=spell.uid))
    name = halogen.Attr()

serialized = SpellSchema.serialize(spell)

Result:

{
    "_links": {
        "self": {"href": "/spells/abracadabra"}
    },
    "name": "Abra Cadabra"
}

Attr(“const”)

In case the attribute represents a constant the value can be specified as a first parameter. This first parameter is a type of the attribute. If the type is not a instance or subclass of a halogen.types.Type it will be bypassed.

import halogen
from flask import url_for

class Spell(object):
    uid = "abracadabra"
    name = "Abra Cadabra"
    cost = 10

spell = Spell()

class SpellSchema(halogen.Schema):
    self = halogen.Link(attr=lambda spell: url_for("spell.get" uid=spell.uid))
    name = halogen.Attr("custom name")

serialized = SpellSchema.serialize(spell)

Result:

{
    "_links": {
        "self": {"href": "/spells/abracadabra"}
    },
    "name": "custom name"
}

In some cases also the attr can be specified to be a callable that returns a constant value.

Attr(attr=”foo”)

In case the attribute name doesn’t correspond your model you can override it:

import halogen
from flask import url_for

class Spell(object):
    uid = "abracadabra"
    title = "Abra Cadabra"
    cost = 10

spell = Spell()

class SpellSchema(halogen.Schema):
    self = halogen.Link(attr=lambda spell: url_for("spell.get" uid=spell.uid))
    name = halogen.Attr(attr="title")

serialized = SpellSchema.serialize(spell)

Result:

{
    "_links": {
        "self": {"href": "/spells/abracadabra"}
    },
    "name": "Abra Cadabra"
}

The attr parameter accepts strings of the source attribute name or even dot-separated path to the attribute. This works for both: nested dictionaries or related objects an Python properties.

import halogen

class SpellSchema(halogen.Schema):
    name = halogen.Attr(attr="path.to.my.attribute")

Attr(attr=lambda value: value)

The attr parameter accepts callables that take the entire source model and can access the neccessary attribute. You can pass a function or lambda in order to return the desired value which also can be just a constant.

import halogen
from flask import url_for

class Spell(object):
    uid = "abracadabra"
    title = "Abra Cadabra"
    cost = 10

spell = Spell()

class SpellSchema(halogen.Schema):
    self = halogen.Link(attr=lambda spell: url_for("spell.get" uid=spell.uid))
    name = halogen.Attr(attr=lambda value: value.title)

serialized = SpellSchema.serialize(spell)

Result:

{
    "_links": {
        "self": {"href": "/spells/abracadabra"}
    },
    "name": "Abra Cadabra"
}

Attribute as a decorator

Sometimes accessor functions are too big for lambdas. In this case it is possible to decorate a method of the class to be a getter accessor.

import halogen

class ShoppingCartSchema(halogen.Schema):

    @halogen.attr(AmountType(), default=None)
    def total(obj):
        return sum(
            (item.amount for item in obj.items),
            0,
        )

    @total.setter
    def set_total(obj, value):
        obj.total = value

Attr(attr=Acccessor)

In case the schema is used for both directions to serialize and to deserialize the halogen.schema.Accessor can be passed with both getter and setter specified. Getter is a string or callable in order to get the value from a model, and setter is a string or callable that knows where the deserialized value should be stored.

Attr(Type())

After the attibute gets the value it passes it to it’s type in order to complete the serialization. Halogen provides basic types for example halogen.types.List to implement lists of values or schemas. Schema is also a Type and can be passed to the attribute to implement complex structures.

Example:

import halogen
from flask import url_for

class Book(object):
    uid = "good-book-uid"
    title = "Harry Potter and the Philosopher's Stone"
    genres = [
        {"uid": "fantasy-literature", "title": "fantasy literature"},
        {"uid": "mystery", "title": "mystery"},
        {"uid": "adventure", "title": "adventure"},
    ]

book = Book()

class GenreSchema(halogen.Schema):
    self = halogen.Link(attr=lambda genre: url_for("genre.get" uid=genre['uid']))
    title = halogen.Attr()

class BookSchema(halogen.Schema):
    self = halogen.Link(attr=lambda book: url_for("book.get" uid=book.uid))
    title = halogen.Attr()
    genres = halogen.Attr(halogen.types.List(GenreSchema))

serialized = BookSchema.serialize(book)

Result:

{
    "_links": {
        "self": {"href": "good-book-uid"}
    },
    "genres": [
        {"_links": {"self": {"href": "fantasy-literature"}}, "title": "fantasy literature"},
        {"_links": {"self": {"href": "mystery"}}, "title": "mystery"},
        {"_links": {"self": {"href": "adventure"}}, "title": "adventure"}
    ],
    "title": "Harry Potter and the Philosopher's Stone"
}

Attr(Type(validators=[validator]))

Type gets optional validators parameter, which is a list of halogen.validators.Validator objects whose single interface method validate will be called for the given value during the deserialization. If the value is not valid, halogen.exceptions.ValidationError should be raised. Halogen provides basic validators, for example halogen.validators.Range to validate that the values is in certain range.

Attr(default=value)

If an attribute cannot be taken, provided default value will be used; if default value is a callable, it will be called to get the default value.

Attr(required=False)

By default, attributes are required, so when an attribute can not be taken during the serialization and default is not provided, an exception will be raised (AttributeError or KeyError, depending on the input). It’s possible to relax this restriction by passing required=False to the attribute constructor. For deserialization, the same logic applies, but the exception type will be halogen.exceptions.ValidationError for human readability (see Deserialization).

Type

Type is responsible in serialization of individual values such as integers, strings, dates. Also type is a base of Schema. It has both serialize() and deserialize() methods that convert the attribute’s value. Unlike Schema types are instantiated. You can configure serialization behavior by passing parameters to their constructors while declaring your schema.

Types can raise halogen.exceptions.ValidationError during deserialization, but serialization expects the value that this type knows how to transform.

Subclassing types

Types that are common in your application can be shared between schemas. This could be the datetime type, specific URL type, internationalized strings and any other representation that requires specific format.

Type.serialize

The default implementation of the Type.serialize is a bypass.

Serialization method of a type is the last opportunity to convert the value that is being serialized:

Example:

import halogen

class Amount(object):
    currency = "EUR"
    amount = 1


class AmountType(halogen.types.Type):
    def serialize(self, value):

        if value is None or not isinstance(value, Amount):
            return None

        return {
            "currency": value.currency,
            "amount": value.amount
        }


class Product(object):
    name = "Milk"

    def __init__(self):
        self.price = Amount()

product = Product()


class ProductSchema(halogen.Schema):

    name = halogen.Attr()
    price = halogen.Attr(AmountType())

serialized = ProductSchema.serialize(product)

Result:

{
    "name": "Milk",
    "price": {
        "amount": 1,
        "currency": "EUR"
    }
}

Nullable types

In case the accessor returns None and the further serialization by a type or a nested schema is not desired the type can be wrapped into Nullable type.

import halogen


class FreeProduct(object):
    """A free product, that doesn't have a price."""

    price = None


class AmountSchema(halogen.Schema):

    currency = halogen.Attr(required=True, default="USD")
    amount = halogen.Attr(required=True, default=0)


class FreeProductSchema(halogen.Schema):

    price_null = halogen.Attr(halogen.types.Nullable(AmountType()), attr="price")
    price_zero = halogen.Attr(AmountType(), attr="price")


serialized = FreeProductSchema.serialize(FreeProduct())

Result:

{
    "price_null": None,
    "price_zero": {
        "amount": 0,
        "currency": "USD"
    }
}

HAL

Hypertext Application Language.

RFC

The JSON variant of HAL (application/hal+json) has now been published as an internet draft: draft-kelly-json-hal

Link

Link objects at RFC: link-objects

href

The “href” property is REQUIRED.

halogen.Link will create href for you. You just need to point to halogen.Link either from where or what halogen.Link should put into href.

Static variant

import halogen

class EventSchema(halogen.Schema):

    artist = halogen.Link(attr="/artists/some-artist")

Callable variant

import halogen

class EventSchema(halogen.Schema):

    help = halogen.Link(attr=lambda: current_app.config['DOC_URL'])

deprecation

Links can be deprecated by specifying the deprecation URL attribute which points to the document describing the deprecation.

import halogen

class EventSchema(halogen.Schema):

    artist = halogen.Link(
        attr="/artists/some-artist",
        deprecation="http://docs.api.com/deprecations#artist",
    )

CURIE

CURIEs are providing links to the resource documentation.

import halogen

doc = halogen.Curie(
    name="doc,
    href="http://haltalk.herokuapp.com/docs/{rel}",
    templated=True
)

class BlogSchema(halogen.Schema):

    lastest_post = halogen.Link(attr="/posts/latest", curie=doc)

{
    "_links": {
        "curies": [
            {
              "name": "doc",
              "href": "http://haltalk.herokuapp.com/docs/{rel}",
              "templated": true
            }
        ],

        "doc:latest_posts": {
            "href": "/posts/latest"
        }
    }
}

Schema also can be a param to link

import halogen

class BookLinkSchema(halogen.Schema):
    href = halogen.Attr("/books")

class BookSchema(halogen.Schema):

    books = halogen.Link(BookLinkSchema)

serialized = BookSchema.serialize({"books": ""})

{
    "_links": {
        "books": {
            "href": "/books"
        }
    }
}

Embedded

The reserved “_embedded” property is OPTIONAL. It is an object whose property names are link relation types (as defined by [RFC5988]) and values are either a Resource Object or an array of Resource Objects.

Embedded Resources MAY be a full, partial, or inconsistent version of the representation served from the target URI.

For creating _embedded in your schema you should use halogen.Embedded.

Example:

import halogen

em = halogen.Curie(
    name="em",
    href="https://docs.event-manager.com/{rel}.html",
    templated=True,
    type="text/html"
)


class EventSchema(halogen.Schema):
    self = halogen.Link("/events/activity-event")
    collection = halogen.Link("/events/activity-event", curie=em)
    uid = halogen.Attr()


class PublicationSchema(halogen.Schema):
    self = halogen.Link(attr=lambda publication: "/campaigns/activity-campaign/events/activity-event")
    event = halogen.Link(attr=lambda publication: "/events/activity-event", curie=em)
    campaign = halogen.Link(attr=lambda publication: "/campaign/activity-event", curie=em)


class EventCollection(halogen.Schema):
    self = halogen.Link("/events")
    events = halogen.Embedded(halogen.types.List(EventSchema), attr=lambda collection: collection["events"], curie=em)
    publications = halogen.Embedded(
        attr_type=halogen.types.List(PublicationSchema),
        attr=lambda collection: collection["publications"],
        curie=em
    )


collections = {
    'events': [
        {"uid": 'activity-event'}
    ],
    'publications': [
        {
            "event": {"uid": "activity-event"},
            "campaign": {"uid": "activity-campaign"}
        }
    ]
}

serialized = EventCollection.serialize(collections)

Result:

{
    "_embedded": {
        "em:events": [
            {
                "_links": {
                    "curies": [
                        {
                            "href": "https://docs.event-manager.com/{rel}.html",
                            "name": "em",
                            "templated": true,
                            "type": "text/html"
                        }
                    ],
                    "em:collection": {"href": "/events/activity-event"},
                    "self": {"href": "/events/activity-event"}
                },
                "uid": "activity-event"
            }
        ],
        "em:publications": [
            {
                "_links": {
                    "curies": [
                        {
                            "href": "https://docs.event-manager.com/{rel}.html",
                            "name": "em",
                            "templated": true,
                            "type": "text/html"
                        }
                    ],
                    "em:campaign": {"href": "/campaign/activity-event"},
                    "em:event": {"href": "/events/activity-event"},
                    "self": {"href": "/campaigns/activity-campaign/events/activity-event"}
                }
            }
        ]
    },
    "_links": {
        "curies": [
            {
                "href": "https://docs.event-manager.com/{rel}.html",
                "name": "em",
                "templated": true,
                "type": "text/html"
            }
        ],
        "self": {"href": "/events"}
    }
}

By default, embedded resources are required, you can make them not required by passing required=False to the constructor, and empty values will be omitted in the serialization:

import halogen

class Schema(halogen.Schema):
    user1 = halogen.Embedded(PersonSchema, required=False)
    user2 = halogen.Embedded(PersonSchema)

serialized = Schema.serialize({'user2': Person("John", "Smith")})

Result:

{
    "_embedded": {
        "user2": {
            "name": "John",
            "surname": "Smith"
        }
    }
}

Deserialization

Schema has deserialize method. Method deserialize will return dict as a result of deserialization if you wont pass any object as a second param.

Example:

import halogen

class Hello(halogen.Schema):
    hello = halogen.Attr()

result = Hello.deserialize({"hello": "Hello World"})
print(result)

Result:

{
    "hello": "Hello World"
}

However, if you will pass object as the second param of deserialize method then data will be assigned on object’s attributes.

Example:

import halogen

class HellMessage(object):
    hello = ""


hello_message = HellMessage()


class Hello(halogen.Schema):
    hello = halogen.Attr()


Hello.deserialize({"hello": "Hello World"}, hello_message)
print(hello_message.hello)

Result:

"Hello World"

Type.deserialize

How you already know attributes launch serialize method from types which they are supported in moment of serialization but in case of deserialization the same attributes will launch deserialize method. It means that when you write your types you should not forget about deserialize methods for them.

Example:

import halogen
import decimal


class Amount(object):
    currency = "EUR"
    amount = 1

    def __init__(self, currency, amount):
        self.currency = currency
        self.amount = amount

    def __repr__(self):
        return "Amount: {currency} {amount}".format(currency=self.currency, amount=str(self.amount))


class AmountType(halogen.types.Type):

    def serialize(self, value):

        if value is None or not isinstance(value, Amount):
            return None

        return {
            "currency": value.currency,
            "amount": value.amount
        }

    def deserialize(self, value):
        return Amount(value["currency"], decimal.Decimal(str(value["amount"])))


class ProductSchema(halogen.Schema):
    title = halogen.Attr()
    price = halogen.Attr(AmountType())


product = ProductSchema.deserialize({"title": "Pencil", "price": {"currency": "EUR", "amount": 0.30}})
print(product)

Result:

{"price": Amount: EUR 0.3, "title": "Pencil"}

Deserialization validation errors

On deserialization failure, halogen raises special exception (halogen.exceptions.ValidationError). That exception class has __unicode__ method which renders human readable error result so user can easily track down the problem with his input.

Example:

import halogen

class Hello(halogen.Schema):
    hello = halogen.Attr()

try:
    result = Hello.deserialize({})
except halogen.exceptions.ValidationError as exc:
    print(exc)

Result:

{
    "errors": [
        {
            "errors": [
                    {
                        "type": "str",
                        "error": "Missing attribute."
                    }
                ],
            "attr": "hello"
        }
    ],
    "attr": "<root>"
}

In case when you have nested schemas, and use List, halogen also adds the index (counting from 0) in the list so you see where exactly the validation error happened.

Example:

import halogen

class Product(halogen.Schema):

    """A product has a name and quantity."""

    name = halogen.Attr()
    quantity = halogen.Attr()


class NestedSchema(halogen.Schema):

    """An example nested schema."""

    products = halogen.Attr(
        halogen.types.List(
            Product,
        ),
        default=[],
    )

try:
    result = NestedSchema.deserialize({
        "products": [
            {
                "name": "name",
                "quantity": 1
            },
            {
                "name": "name",
            }

        ]
    })
except halogen.exceptions.ValidationError as exc:
    print(exc)

Result:

{
    "errors": [
        {
            "errors": [
                {
                    "index": 1,
                    "errors": [
                        {
                            "errors": [
                                {
                                    "type": "str",
                                    "error": "Missing attribute."
                                }
                            ],
                            "attr": "quantity"
                        }
                    ]
                }
            ],
            "attr": "products"
        }
    ],
    "attr": "<root>"
}

Note that should ValueError exception happen on the attribute deserialization, it will be caught and reraized as halogen.exceptions.ValidationError. This is to eliminate the need of raising halogen specific exceptions in types and attributes during the deserialization.

Providing context

When serializing or deserializing an object, not all data required for (de)serialization may be available in the object itself. You can pass this data as separate keyword arguments to serialize or deserialize to provide context. This context will be available in all nested schema, types and attributes.

Serialize example:

class ErrorSchema(halogen.Schema):
    message = halogen.Attr(
        attr=lambda error, language: error["message"][language]
    )

error = ErrorSchema.serialize({
    "message": {
        "dut": "Ongeldig e-mailadres",
        "eng": "Invalid email address"
    }
}, language="dut")

print error

Result:

{"message": "Ongeldig e-mailadres"}

Deserialize example:

import halogen


class Book(halogen.Schema):

    @halogen.attr()
    def title(obj, language):
        return obj['title'][language]

class Author(halogen.Schema):
    name = halogen.Attr(attr='author.name')
    books = halogen.Attr(
        halogen.types.List(Book),
        attr='author.books',
    )

author = Author.deserialize({
    "author": {
        "name": "Roald Dahl",
        "books": [
            {
                "title": {
                    "dut": "De Heksen",
                    "eng": "The Witches"
                }
            },
            {
                "title": {
                    "dut": "Sjakie en de chocoladefabriek",
                    "eng": "Charlie and the Chocolate Factory"
                }
            }
        ]
    }
}, language="eng")

print author

Result:

{
    "name": "Roald Dahl",
    "books": [
        {"title": "The Witches"},
        {"title": "Charlie and the Chocolate Factory"}
    ]
}

Vendor media types

Handling validation and business logic errors are as important as handling HAL responses. Halogen provides support for the vendor error media type, which is fully HAL-compatible.

vnd.error

The vendor error (application/vnd.error+json) has now been published as an internet draft: draft-vnd-error

This mediatype is attempting to standartise the format in which the problem can be represented to many clients so that it can be expressed and understood. Multiple deserialization errors can be mapped to the relevant keys of the payload via the path attribute, which represents the JSON Pointer to the payload key, and therefore to the UI element that is serialized with that key.

import halogen
from halogen.vnd.error import Error, VNDError


class AuthorSchema(halogen.Schema):
    name = halogen.Attr(required=True)


class PublisherSchema(halogen.Schema):
    name = halogen.Attr(required=True)
    address = halogen.Attr()


class BookSchema(halogen.Schema):
    title = halogen.Attr(required=True)
    year = halogen.Attr(halogen.types.Int(), required=True)
    authors = halogen.Attr(halogen.types.List(AuthorSchema), required=True)
    publisher = halogen.Attr(PublisherSchema)

try:
    BookSchema.deserialize(
        dict(
            # title is skipped
            year="abc",  # Not an integer
            authors=[dict(name="John Smith"), dict()],  # Second author has no name
            publisher=dict(address="Chasey Lane 42, Los Angeles, US"),  # No name
        ),
    )
except halogen.exceptions.ValidationError as e:
    error = Error.from_validation_exception(e)

>>> error.errors
>>>
    [
        {"path": "/authors/1/name", "message": "Missing attribute."),
        {"path": "/title", "message": "Missing attribute."),
        {"path": "/year", "message": "'abc' is not an integer"),
        {"path": "/publisher/name", "message": "Missing attribute."),
    }

The errors may or may not be related to the payload, but sometimes to another resources. In this case the about link should be returned within the error.

{
    "_links": {
        "about": {"href": "/products/1"}
    },
    "message": "The product is sold out."
}

i18n

The error messages should be internationalized and respect Accept-Language and Content-Language HTTP headers.

Contact

If you have questions, bug reports, suggestions, etc. please create an issue on the GitHub project page.

License

This software is licensed under the MIT license

See License file

© 2013 Oleg Pidsadnyi, Paylogic International and others.

Changelog

2.0.0

• Added Enum type

• Added OneOfValidator

• Breaking change: All types are now non-nullable instead of possibly returning a default or accepting None, you should adapt your code accordingly by using Nullable()

• Dropped support Python 2

1.7.0

• Fix List type raising unrelated errors (or no error at all, when passing a dictionary) when deserializing objects that are not lists.

1.6.1

• Hotfix for 1.6.0: Fix serialization/deserialization error Schema.serialize(…) is invoked with kwargs and there are attributes using callables that don’t require those kwargs.

1.6.0

• Fix DeprecationWarnings being raised (youtux)

• Dropped support for python 2.6 and add declare support for python 3.5, 3.6, 3.7, 3.8

1.5.0

• Allow passing context keyword arguments to deserialize methods (blaise-io)

1.4.1

• Fix the package setup (olegpidsadnyi)

1.4.0

• Support for vnd.error responses (olegpidsadnyi)

1.3.5

• Add ISO Date Time type (moisesribeiro)

1.3.4

• Nullable type (olegpidsadnyi)

1.3.3

• Strict validation of the ISO8601 (olegpidsadnyi)

1.3.2

• Improve serialization performance (youtux)

1.3.1

• Fix for the String.deserialize to force the text type (olegpidsadnyi)

1.3.0

• Attribute as a decorator (olegpidsadnyi)

1.2.1

• Use native datetime.isoformat for datetime serialization (bubenkoff)

1.1.3

• Correctly handle schema class derivation (bubenkoff)

1.1.2

• Correct deserialization for String and Int types (bubenkoff)

1.1.1

• Deprecation attribute is added to Link (olegpidsadnyi)

1.1.0

• Add common-use types (bubenkoff)

1.0.8

• Correctly handle and document required and default (bubenkoff)

• Properly get validator’s comparison values (lazy and constant) (bubenkoff)

• Increase test coverage (bubenkoff)

1.0.6

• Respect ValueError in deserialization of the attributes (bubenkoff)

1.0.4

• Correctly render and document deserialization errors (bubenkoff)

1.0.3

• Allow Embedded fields to be marked as not required (mattupstate)

• Field order is preserved in serialized documents (mattupstate)

1.0.0

• Initial public release