A simple Python interface for AppSync resolvers and Gremlin traversals.
Project description
AppSync - Gremlin
Overview
Through the AppSync-Gremlin, developers can write powerful queries in GraphQL without having to worry too much about the underlying database query language in AWS Neptune. The AppSync-Gremlin provides lambda function code that converts query operation types (from GraphQL) to a gremlin traversal.
Definitions
-
Property field: A field corresponding to a property of a vertex in the AWS Neptune graph database. In the below example, the
name
field is a property field.{ User { name following { name } } }
Query 1 : (Vertex Field Example)
-
Vertex field: A field corresponding to a vertex in the AWS Neptune graph database. In the above example,
location
is a vertex field. -
Vertex list fields: A field corresponding to a list of vertices in the AWS Neptune graph database. In the above example,
following
is a vertex list field. -
Result set: An assignment of vertices in the graph to fields in the query. As the database processes the query, new result sets may be created (e.g. when traversing edges), and result sets may be discarded when they do not satisfy filters. After all parts of the query are processed by the database, all remaining result sets are used to form the query result, by taking their values at all properties marked for output (anything in an output scope).
-
Scope: The part of a query between any pair of parentheses or curly braces. We often refer to the parts between parentheses as the input scope and the parts between curly braces as the output scope or payload scope. For example, consider the query
{ User ( input: { name: { eq: "John" } } ) { name following { name } } }
Query 2 : (Scope Example)
Filtering Operations and Pagination
Filtering
We define a filtering standard on the following scalar fields:
- ID:
For ID filtering we define the following input for filtering:
input IDFilterInput { ne: ID eq: ID in: [ID!] not_in: [ID!] }
- String: For String filtering we define the following input for filtering
input StringFilterInput { ne: String eq: String in: [String!] not_in: [String!] contains: String not_contains: String begins_with: String not_begins_with: String ends_with: String not_ends_with: String }
- Int: For Integer filtering we define the following input for filtering
input IntFilterInput { ne: Int eq: Int le: Int lt: Int ge: Int gt: Int in: [Int!] not_in: [int!] }
- Float: For Float filtering we define the following input for filtering
input FloatFilterInput { ne: Float eq: Float le: Float lt: Float ge: Float gt: Float in: [Float!] not_in: [Float!] }
- Boolean: For Boolean filtering we define the following input for filtering
input BooleanFilterInput { eq: Boolean ne: Boolean }
- DateTime: For DateTime we first have to define our own DateTime input definition. To avoid confusion and to prevent the use of different DateTime formats
in this interface, we have defined the following
DateTimeInput
to expose the individual date components (such as day, month, year, etc) as well as aformatted
field which is the ISO 8601 string representation of the DateTime value:
Using this input definition, we can then create the following input for filtering:input DateTimeInput { year: Int month: Int day: Int hour: Int minute: Int second: Int formatted: DateTime #custom datetime scalar }
input DateTimeFilterInput { eq: DateTimeInput ne: DateTimeInput in: [DateTimeInput!] not_in: [DateTimeInput!] le: DateTimeInput lt: DateTimeInput ge: DateTimeInput gt: DateTimeInput }
and a filtering standard on enum types:
enum ENUM_FIELD_TYPE {
E_1
E_2
.
.
.
E_n
}
input EnumFilterInput {
eq: ENUM_FIELD_TYPE
ne: ENUM_FIELD_TYPE
in: [ENUM_FIELD_TYPE!]
not_in: [ENUM_FIELD_TYPE!]
}
Note that these standards must be manually implemented in the original GraphQL schema. In future we may devise some method of augmenting a GraphQL schema so we don't have to manually implement them.
Each of these scalar filters has a corresponding filter in the AppSync-Gremlin library. For example, the StringFilterInput
has
the scalar filter string_filter
, etc.
Implementing Filters
The AppSync-Gremlin library also allows custom vertex
, relationship
and scalar
filters to be implemented.
-
Scalar filters: We define a
scalar
filter as a filter that maps GraphQL fields to Gremlin predicates. Consider anid_filter
. In GraphQL, theIDFilterInput
has the following fields:eq, ne, in, not_in
. The Gremlin predicates that correspond to these fields areeq, neq, within, without
. So we construct the following dictionary to map the GraphQL fields to the Gremlin predicates:from gremlin_python.process.traversal import eq, neq, within, without def id_scalar_filter(): return { "eq": eq, "ne": neq, "in": within, "not_in": without }
By using this dictionary, we can construct a filter function that applies the predicates to some traversal
g
, such that the traverser is at some verticesv
with propertyproperty_name
, producing a filtered traversal with the general form ,`g' = g.has(property_name, p_1).has(property_name, p_2). ... .has(property_name, p_n)`,
where
p_1, p_2, ..., p_n
are predicates that are applied to the propertyproperty_name
atv
.The
@scalar_filter
decorator provided by the AppSync-Gremlin library produces a function that takes aproperty_name
and returns the function described above. So by applying this to the aboveid_filter
gives usfrom gremlin_python.process.traversal import eq, neq, within, without from appsync_gremlin import scalar_filter @scalar_filter def id__filter(): return { "eq": eq, "ne": neq, "in": within, "not_in": without }
However, at this stage the filter cannot actually be used, since it requires a
property_name
. The AppSync-Gremlin library provides two different ways of applying aproperty_name
to a filter. Either we can simply callid_scalar_filter
with out desiredproperty_name
, or we can apply the@name
decorator.For scalar filters, we advice that the former is used. Since this provides a better generalisation for your scalar filter. Since a vertex could implement it's own custom id with property name
"id"
or simply make use of the built in vertex idT.id
. -
Vertex filter: We define a
vertex
filter as a filter that maps GraphQL fields toscalar
orrelationship
(see below) filters. Let us consider the following GraphQLUser
type:type User { id: ID! email: String! name: String about: String following: [User]! followed_by: [User]! }
We have 4 scalar fields (
id, email, name, about
) and 2 vertex list fields (following, followed_by
). In the Graph database, we define aUser
vertex with propertiesemail, name, about
(string) and a single relationshipUser -> FOLLOWS -> User
. So in a similar fashion to a scalar filter, let us define some function that returns a dictionary mapping the GraphQL fields to filters:from gremlin_python.process.traversal import T from appsync_gremlin import id_filter, string_filter def user_filter(): return { "id": id_filter(T.id), "email": string_filter("email"), "name": string_filter("name"), "about": string_filter("about"), "following": ?, "followed_by": ? }
By using this dictionary we can construct a filter function that applies the
scalar
andrelationship
filters to some supplied traversalg
, such that the traverser is at some verticesv
which matches the desired filter (in this case the travesers must all be atUser
vertices), producing a filtered traversalg'
with the general form:g' = r_1(r_2( ... (r_n(s_1( ... s_m(g) ... ))) ... ))
where
r_1, r_2, ..., r_n
are relationship filters ands_1, s_2, ..., s_m
are scalar filters. Note that the order of application of scalar and relationship filters does not matter, however, we advice that scalar filters should be applied first as it is more efficient.The
@vertex_filter
decorator provided by the AppSync-Gremlin library produces a function that takes avertex_label
and returns the function described above. So by applying this to the aboveuser_filter
gives usfrom gremlin_python.process.traversal import T from appsync_gremlin import id_filter, string_filter, vertex_filter, name @name("User") @vertex_filter def user_filter(): return { "id": id_filter(T.id), "email": string_filter("email"), "name": string_filter("name"), "about": string_filter("about"), "following": ?, "followed_by": ? }
Similarly to the
scalar_filter
, we can apply the@name
decorator to pass thevertex_label
to the filter function. This label should match the label of theUser
vertex in the Graph database.With the vertex filter now completely implemented, we can now define our relationship filters (see below). In this case we have the relationship filters:
from gremlin_python.process.traversal import T from appsync_gremlin import id_filter, string_filter, vertex_filter, name, relationship_filter, RelationshipDirection @name("User") @vertex_filter def user_filter(): return { "id": id_filter(T.id), "email": string_filter("email"), "name": string_filter("name"), "about": string_filter("about"), "following": relationship_filter(("FOLLOWS", RelationshipDirection.OUT), user_filter), "followed_by": relationship_filter(("FOLLOWS", RelationshipDirection.IN), user_filter) }
-
Relationship filter: We define a relationship
R
from some vertexu
tov
asR = (name, direction)
. We can filteru
based on the whether a relationshipR
exists and whetherv
satisfies certain conditions.Consider a Gremlin traversal
g
, such that the traverser is at some verticesu
. We can produce a filtered traversalg'
based on whether the vertices selected byg
have the relationshipR
and the verticesv
(the other vertex inR
) satisfy a vertex filterv_f
. This filtered traversal has the general form:g' = g.where(v_f(direction(name))).
The AppSync-Gremlin library produces a function
relationship_filter
that takes arelationship
, a tuple consisting of a edge label (name
) and a edge direction (direction
), and a vertex filterv_f
for the other vertex in the relationship.(See above for example).
Pagination
We also implement a pagination standard. Note that pagination can only be applied to vertex list fields.
For simplicity, we've decided to implement an offset based pagination, as it allows us to make us
of the Gremlin traversal step .range(first, offset)
. The stanardised pagination input is defined as follows:
input PaginationInput {
page: Int!
per_page: Int!
}
We then use page
and per_page
to compute first
and offset
using the function get_range
, shown below.
from typing import Tuple
def get_range(page: int, per_page: int) -> Tuple[int, int]:
"""
Returns the Gremlin range from page options in the format:
(first, last)
:param page: (Integer)
:param per_page: (Integer)
:return: (Integer, Integer)
"""
return (page - 1) * per_page, page * per_page
Once the traversal has been submitted and the result set has been return, we format the response into a pagination
response object. The GraphQL type for this response object for some GraphQL type Type
is
type Type {
.
.
.
}
type TypePage {
data: [Type]!
page: Int!
per_page: Int!
total: Int!
}
where total
is the total
number of pages available.
Error Handling and Request / Response Mapping Template
The AppSync-Gremlin library provides automatic error handling for AppSync. The library does this via the user of the AppSyncException
.
The AppSyncException
requires 3 arguments when instantiated: error_type
, error_message
and error_data
for type
string, string and dictionary respectively.
For example, consider the mutation resolver that creates a User
vertex. Naturally we want to ensure that a user doesn't have a duplicate vertex,
therefore we must add some form of validation within the resolver code which raises an AppSyncException
with the relevant error information
if the validation fails.
from gremlin_python.process.graph_traversal import GraphTraversal
from appsync_gremlin import ResolverInput, AppSyncException, mutation_resolver
@mutation_resolver
def create_user(traversal: GraphTraversal, resolver_input: ResolverInput) -> GraphTraversal:
username = resolver_input.arguments.get("username")
user = traversal.V().hasLabel("User").has("username", username)
if user.hasNext():
raise AppSyncException(
error_type="BAD_REQUEST",
error_message="A user with username {} is already stored in the AWS Neptune database.".format(username),
error_data={
"username": username
}
)
.
.
.
In order to ensure our AppSyncException
work's with AppSync, we've had to define a request / response template mapping standard.
For all resolvers, we must have the request template mapping:
{
"version" : "2018-05-29",
"operation": "(Invoke|BatchInvoke)",
"payload": {
"type_name": String!,
"field_name": String!,
"arguments": $util.toJson($context.args),
"identity": $util.toJson($context.identity),
"source": $util.toJson($context.source)
}
}
and the response mapping template:
#if ($context.result && $context.result.error)
$utils.error($context.result.error.error_message, $context.result.error.error_type, $context.result.error.data)
#else
$utils.toJson($context.result.data)
#end
Usage
The AppSync-Gremlin library currently provides 4 different resolver types in the form of function decorators:
vertex_list_field_resolver
vertex_field_resolver
calculated_field_resolver
mutation_resolver
The first 3 resolvers are Query based resolvers, that is to say they are designed to be used for resolving GraphQL queries.
Each resolver has the same function type signature:
resolver : (GraphTraversal, ResolverInput) -> GraphTraversal
Note the custom ResolverInput
object. A ResolverInput
object simply stores the data passed from the Apache VTL
request mapping template described in the section above. Hence the ResolverInput
object has the following properties:
type_name
:(String)
field_name
:(String)
arguments
:(Dictionary)
identity
:(Dictionary | None)
source
:(Dictionary | None)
Hence these properties can be referenced in the resolvers to build the Gremlin traversals.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file appsync-gremlin-0.0.19.tar.gz
.
File metadata
- Download URL: appsync-gremlin-0.0.19.tar.gz
- Upload date:
- Size: 12.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d548cfe1c114df3bd249a7456d824ff89ec294523b2131d45389202a2a574d17 |
|
MD5 | 124cb50a9300979488e2bc42f4b97a40 |
|
BLAKE2b-256 | 09ae5f0ae88df9da2f57f9b1a4bfa0eb462c7ec9c0794fb9490efdf3e4434a8e |
File details
Details for the file appsync_gremlin-0.0.19-py3-none-any.whl
.
File metadata
- Download URL: appsync_gremlin-0.0.19-py3-none-any.whl
- Upload date:
- Size: 14.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/2.0.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.6.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2401e7ff208b8a4e35480da445b688b40e872bfc4474a60875f36542aa6e44d4 |
|
MD5 | 53743582416cc5cd252b4b5292ec54fe |
|
BLAKE2b-256 | 2766d7b29c12f1f88d93d7b56ebe2d8d19fe59dd55ea782c02c3e3f2dc3c0fa5 |