meeshkan 0.2.5

Reverse engineer services with style

Meeshkan

Meeshkan is a library and command-line tool for reverse engineering APIs from recorded traffic.

It is used at Meeshkan to build enterprise-level API sandboxes.

The supported input format for HTTP recordings is the HTTP Types JSON format.

Meeshkan requires Python 3.6 or later for the new async/await syntax.

Meeshkan is statically typed using the pyright type checker.

Table of Contents

1. Installation
2. Python API
3. Command-line interface
4. Development
5. Contributing

Installation

Install via pip:

$ pip install meeshkan

Note that meeshkan requires Python 3.6+.

Python API

Quick start

from openapi_typed import OpenAPIObject
import meeshkan
from typing import List
import json
from http_types import HttpExchange, Request, Response, RequestBuilder


def read_http_exchanges() -> List[HttpExchange]:
    """Read HTTP exchanges from the source of your choice.
    """
    request: Request = RequestBuilder.from_url(
        "https://example.org/v1", method="get", headers={})

    response: Response = Response(
        statusCode=200, body=json.dumps({"hello": "world"}), headers={})

    http_exchange: HttpExchange = {'request': request, 'response': response}

    return [http_exchange]


http_exchanges = read_http_exchanges()

# Build OpenAPI schema from a list of recordings
openapi: OpenAPIObject = meeshkan.build_schema_batch(http_exchanges)

# Build schema from an iterator
openapi: OpenAPIObject = meeshkan.build_schema_online(iter(http_exchanges))

# Update OpenAPI schema one `HttpExchange` at a time
http_exchange = http_exchanges[0]
openapi: OpenAPIObject = meeshkan.update_openapi(openapi, http_exchange)

Command-line interface

To list available commands, execute meeshkan or meeshkan --help.

Create OpenAPI schema from HTTP recordings

Build OpenAPI schema from a single recordings.jsonl file:

$ meeshkan build --source file -i path/to/recordings.jsonl [-o path/to/output_directory]

The input file should be in JSON Lines format and every line should be in HTTP Types JSON format.For an example input file, see recordings.jsonl. The libraries listed at HTTP Types can be used to generate input files in your language of choice.

Use dash (-i -) to read from standard input:

$ meeshkan build --source file -i - < recordings.jsonl

Reading from Kafka

Install kafka bundle: pip install meeshkan[kafka] and set --source kafka:

$ meeshkan build --source kafka [-o path/to/output_directory]

TODO: Configuration for Kafka

Converting from pcap

You can convert packet capture files to HTTP Types format using the convert command:

meeshkan convert -i /path/to/file.pcap -o recordings.jsonl

Executable tshark must be present in your PATH.

Converter does not decrypt captured packages, so only files containing plain unencrypted HTTP traffic are currently supported.

Development

Getting started

1. Create virtual environment
2. Install dependencies: pip install --upgrade -e '.[dev]'

Tests

Run tests/ with pytest:

pytest
# or
python setup.py test

Configuration for pytest is found in pytest.ini.

Type-checking

Run type-checking by installing pyright globally and running

pyright --lib
# or
python setup.py typecheck

Using the Pyright extension is recommended for development in VS Code.

Automated builds

Configuration for CircleCI build pipeline can be found in .circleci/config.yml.

Publishing package

1. Bump the version in setup.py if the version is the same as in the published package. Commit and push.
2. Run python setup.py test, python setup.py typecheck and python setup.py dist to check everything works
3. To build and upload the package, run python setup.py upload. Insert PyPI credentials to upload the package to PyPI. The command will also run git tag to tag the commit as a release and push the tags to remote.

To see what the different commands do, see Command classes in setup.py.

Contributing

Thanks for wanting to contribute! Take a look at our development guide for notes on how to develop the package locally.

Please note that this project is governed by the Meeshkan Community Code of Conduct. By participating in this project, you agree to abide by its terms.