hypothesis-graphql

Generate queries matching your GraphQL schema, and use them to verify your backend implementation

It is a Python library that provides a set of Hypothesis strategies that let you write tests parametrized by a source of examples. Generated queries have arbitrary depth and may contain any subset of GraphQL types defined in the input schema. They expose edge cases in your code that are unlikely to be found otherwise.

Schemathesis provides a higher-level interface around this library and finds server crashes automatically.

Usage

hypothesis-graphql provides the from_schema function, which takes a GraphQL schema and returns a Hypothesis strategy for GraphQL queries matching the schema:

from hypothesis import given
from hypothesis_graphql import from_schema
import requests

# Strings and `graphql.GraphQLSchema` are supported
SCHEMA = """
type Book {
  title: String
  author: Author
}

type Author {
  name: String
  books: [Book]
}

type Query {
  getBooks: [Book]
  getAuthors: [Author]
}

type Mutation {
  addBook(title: String!, author: String!): Book!
  addAuthor(name: String!): Author!
}
"""


@given(from_schema(SCHEMA))
def test_graphql(query):
    # Will generate samples like these:
    #
    # {
    #   getBooks {
    #     title
    #   }
    # }
    #
    # mutation {
    #   addBook(title: "H4Z\u7869", author: "\u00d2"){
    #     title
    #   }
    # }
    response = requests.post("http://127.0.0.1/graphql", json={"query": query})
    assert response.status_code == 200
    assert response.json().get("errors") is None

It is also possible to generate queries or mutations separately with hypothesis_graphql.queries and hypothesis_graphql.mutations.

Customization

To restrict the set of fields in generated operations use the fields argument:

@given(from_schema(SCHEMA, fields=["getAuthors"]))
def test_graphql(query):
    # Only `getAuthors` will be generated
    ...

You can customize the string generation with these arguments to from_schema:

• allow_x00 (default True): Determines whether to allow the generation of \x00 bytes within strings. It is useful to avoid rejecting tests as invalid by some web servers.
• codec (default utf-8): Specifies the codec used for generating strings. It helps if you need to restrict the inputs to, for example, the ASCII range.

@given(from_schema(SCHEMA, allow_x00=False, codec="ascii"))
def test_graphql(query):
    assert "\0" not in query
    query.encode("ascii")

It is also possible to generate custom scalars. For example, Date:

from hypothesis import strategies as st, given
from hypothesis_graphql import from_schema, nodes

SCHEMA = """
scalar Date

type Query {
  getByDate(created: Date!): Int
}
"""


@given(
    from_schema(
        SCHEMA,
        custom_scalars={
            # Standard scalars work out of the box, for custom ones you need
            # to pass custom strategies that generate proper AST nodes
            "Date": st.dates().map(nodes.String)
        },
    )
)
def test_graphql(query):
    # Example:
    #
    #  { getByDate(created: "2000-01-01") }
    #
    ...

The hypothesis_graphql.nodes module includes a few helpers to generate various node types:

• String -> graphql.StringValueNode
• Float -> graphql.FloatValueNode
• Int -> graphql.IntValueNode
• Object -> graphql.ObjectValueNode
• List -> graphql.ListValueNode
• Boolean -> graphql.BooleanValueNode
• Enum -> graphql.EnumValueNode
• Null -> graphql.NullValueNode (a constant, not a function)

They exist because classes like graphql.StringValueNode can't be directly used in map calls due to kwarg-only arguments.

Negative Testing

hypothesis-graphql supports generating invalid GraphQL queries for negative testing. This is useful for testing error handling and validation logic in your GraphQL server.

Use the mode=Mode.NEGATIVE parameter to generate queries that should be rejected:

from hypothesis import given
from hypothesis_graphql import queries, Mode
import requests

SCHEMA = """
type Query {
    getUser(id: Int!, name: String!): User
}

type User {
    id: Int!
    name: String!
}
"""


@given(queries(SCHEMA, mode=Mode.NEGATIVE))
def test_server_rejects_invalid_queries(query):
    # Generates queries with violations like:
    # - Wrong types (String where Int expected)
    # - Missing required arguments
    # - Out-of-range integers
    # - Null for non-null fields
    # - Invalid enum values
    response = requests.post("http://127.0.0.1/graphql", json={"query": query})

    # Server should reject with 400 or return errors
    if response.status_code == 200:
        assert "errors" in response.json()

The mode parameter works with queries(), mutations(), and from_schema().

License

The code in this project is licensed under MIT license. By contributing to hypothesis-graphql, you agree that your contributions will be licensed under its MIT license.