Skip to main content

Specmatic Python

This is a Python library to run Specmatic. Specmatic is a contract driven development tool that allows us to turn OpenAPI contracts into executable specifications.
Click below to learn more about Specmatic and Contract Driven Development

Specmatic - Contract Driven Development

The specmatic python library provides three main functions:

  • The ability to start and stop a python web app like flask/sanic.
  • The ability to run specmatic in test mode against an open api contract/spec.
  • The ability to mock out an api dependency using the specmatic mock feature.

Running Contract Tests

A contract test validates an open api specification against a running api service.
The open api specification can be present either locally or in a Central Contract Repository
Click here to learn more about contract tests.

How to use

  • Create a file called test_contract.py in your test folder.
  • Declare an empty class in it called 'TestContract'.
    This is could either be a normal class like:
    class TestContract:
      pass
    
    Or you could also have a class which inherits from unittest.TestCase:
    class TestContract(unittest.TestCase):
      pass
    

How does it work

  • Specmatic uses the TestContract class defined above to inject tests dynamically into it when you run it via PyTest or UnitTest.
  • The Specmatic Python package, invokes the Specmatic executable jar (via command line) in a separate process to start mocks and run tests.
  • It is the specmatic jar which runs the contract tests and generates a JUnit test summary report.
  • The Specmatic Python package ingests the JUnit test summary report and generates test methods corresponding to every contract test.
  • These dynamic test methods are added to the TestContract class and hence we seem them reported seamlessly by PyTest/Unittest like this:
test/test_contract_with_coverage.py::TestContract::test_Scenario: GET /products -> 200 | SEARCH_2 PASSED
test/test_contract_with_coverage.py::TestContract::test_Scenario: GET /products -> 500 | SEARCH_ERROR PASSED
test/test_contract_with_coverage.py::TestContract::test_Scenario: GET /products -> 200 | SEARCH_1 PASSED

WSGI Apps

To run contract tests with a mock for a wsgi app (like Flask):

class TestContract:
    pass


(
    Specmatic(PROJECT_ROOT)
    .with_mock()
    .with_wsgi_app(app, app_host, app_port)
    .test(TestContract)
    .run()
)

if __name__ == '__main__':
    pytest.main()
  • In this, we are passing:
    • The root directory of our project as an argument to the Specmatic constructor. This is required because specmatic needs to be able to find the specmatic config file, examples etc in your project directory to start the mock and run tests.
    • an instance of your wsgi app like flask
    • app_host and app_port. If they are not specified, the app will be started on a random available port on 127.0.0.1.
    • You would need a specmatic config file to be present in the root directory of your project.
    • an empty test class. The mock will be started based on the run options configured in specmatic.yaml.
      Click here to learn more about mocking/service virtualization.
  • You can run this test from either your IDE or command line by pointing pytest to your test folder: pytest test -v -s
  • NOTE: Please ensure that you set the '-v' and '-s' flags while running pytest as otherwise pytest may swallow up the console output.

To run contract tests without a mock:

class TestContract:
    pass


(
    Specmatic(PROJECT_ROOT)
    .with_wsgi_app(app, app_host, app_port)
    .test(TestContract)
    .run()
)

ASGI Apps

To run contract tests with a mock for an asgi app (like sanic):

  • If you are using an asgi app like sanic, fastapi, use the with_asgi_app function and pass it a string in the 'module:app' format.
class TestContract:
    pass


(
    Specmatic(PROJECT_ROOT)
    .with_mock()
    .with_asgi_app('main:app', app_host, app_port)
    .test(TestContract)
    .run() 
)

Coverage

Specmatic can generate a coverage summary report which will list out all the apis exposed by your app/service with a status next to it indicating if it has been covered in your contract tests.

Enabling api coverage for Flask apps

class TestContract:
    pass


(
    Specmatic(PROJECT_ROOT)
    .with_mock()
    .with_wsgi_app(app, app_host, app_port)
    .test_with_api_coverage_for_flask_app(TestContract, app)
    .run()
)

Enabling api coverage for Sanic apps

class TestContract:
    pass


(
    Specmatic(PROJECT_ROOT)
    .with_mock()
    .with_asgi_app('main:app', app_host, app_port)
    .test_with_api_coverage_for_sanic_app(TestContract, app)
    .run()
)

Enabling api coverage for FastApi apps

class TestContract:
    pass


(
    Specmatic(PROJECT_ROOT)
    .with_mock()
    .with_asgi_app('main:app', app_host, app_port)
    .test_with_api_coverage_for_fastapi_app(TestContract, app)
    .run()
)

Enabling api coverage for any other type of app

For any app other than Flask, Sanic, and FastApi, you would need to implement an AppRouteAdapter class.
The idea is to implement to_coverage_routes method, which returns a list of CoverageRoute objects corresponding to all the routes defined in your app.
The CoverageRoute class has two properties:
url : This represents your route url in this format: /orders/{order_id}
method : A list of HTTP methods supported on the route, for instance : ['GET', 'POST']

You can then enable coverage by passing your adapter like this:

(
    Specmatic(PROJECT_ROOT)
    .with_mock()
    .with_asgi_app('main:app', app_host, app_port)
    .test_with_api_coverage(TestContract, MyAppRouteAdapter(app))
    .run()
)

Enabling api coverage by setting the EndPointsApi property

You can also start your coverage server externally and use the EndPointsApi method to enable coverage.
We have provided ready to use Coverage Server classes for:
Flask: FlaskAppCoverageServer
Sanic: SanicAppCoverageServer
FastApi FastApiAppCoverageServer

You can also easily implement your own coverage server if you have written a custom implementation of the AppRouteAdapter class. The only point to remember in mind is that the EndPointsApi url should return a list of routes in the format used buy Spring Actuator's /actuator/mappings endpoint as described here.

Here's an example where we start both our FastApi app and coverage server outside the specmatic api call.

app_server = ASGIAppServer('test.apps.fast_api:app', app_host, app_port)
coverage_server = FastApiAppCoverageServer(app)

app_server.start()
coverage_server.start()


class TestContract:
    pass


(
    Specmatic(PROJECT_ROOT)
    .with_mock()
    .with_endpoints_api(coverage_server.endpoints_api)
    .test(TestContract, app_host, app_port)
    .run()
)

app_server.stop()
coverage_server.stop()

Common Issues

  • 'Error loading ASGI app' This error occurs when an incorrect app module string is passed to the with_asgi_app function.

    Solutions:

    • Try to identify the correct module in which your app variable is instantiated/imported.
      For example if your 'app' variable is declared in main.py, try passing 'main:app'.
    • Try running the app using uvicorn directly:
      uvciron 'main:app'
      If you are able to get the app started using uvicorn, it will work with specmatic too.

Sample Projects

Metadata

Release files for specmatic 2.56.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for specmatic 2.56.3
File Size Uploaded Provenance Actions
specmatic-2.56.3.tar.gz 78.1 MB Without provenance Details for specmatic-2.56.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for specmatic 2.56.3
File Interpreter ABI Platform Actions
specmatic-2.56.3-py3-none-any.whl Python 3 none any Details for specmatic-2.56.3-py3-none-any.whl

Total release size: 156.1 MB

Release files / specmatic-2.56.3.tar.gz

Download URL specmatic-2.56.3.tar.gz
Size 78.1 MB
Tags Source
SHA-256 checksum
How to use checksums
a0d99055a89a1fb52b13216712dc495e2f47e0900636141f0416832157e978ac
BLAKE2b-256 checksum
How to use checksums
12bb0a654a9f80239c5a5d1fbbadc8f261771e6476ae4e3a0105180d50f48bc1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / specmatic-2.56.3-py3-none-any.whl

Download URL specmatic-2.56.3-py3-none-any.whl
Size 78.1 MB
Tags Python 3
SHA-256 checksum
How to use checksums
13d53c9e1333e59c310c6c1de5674df1726dfd6bc90eeb8044a52dd671c27af4
BLAKE2b-256 checksum
How to use checksums
9ca9fc1ab4e37e6f82659e9d613f53b75ab9363db06fa5c1a1c28e82375c1189
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for specmatic 2.56.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

2.56.3 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page