Reverse engineer services with style
Project description
Meeshkan
Meeshkan is a library and command-line tool for reverse engineering APIs from recorded traffic.
It is used at Meeshkan to build enterprise-level API sandboxes.
The supported input format for HTTP recordings is the HTTP Types JSON format.
Meeshkan requires Python 3.6 or later for the new async/await syntax.
Meeshkan is statically typed using the pyright type checker.
Table of Contents
Installation
Install via pip:
$ pip install meeshkan
Note that meeshkan
requires Python 3.6+.
Python API
Quick start
from openapi_typed import OpenAPIObject
import meeshkan
from typing import List
import json
from http_types import HttpExchange, Request, Response, RequestBuilder
def read_http_exchanges() -> List[HttpExchange]:
"""Read HTTP exchanges from the source of your choice.
"""
request: Request = RequestBuilder.from_url(
"https://example.org/v1", method="get", headers={})
response: Response = Response(
statusCode=200, body=json.dumps({"hello": "world"}), headers={})
http_exchange: HttpExchange = {'request': request, 'response': response}
return [http_exchange]
http_exchanges = read_http_exchanges()
# Build OpenAPI schema from a list of recordings
openapi: OpenAPIObject = meeshkan.build_schema_batch(http_exchanges)
# Build schema from an iterator
openapi: OpenAPIObject = meeshkan.build_schema_online(iter(http_exchanges))
# Update OpenAPI schema one `HttpExchange` at a time
http_exchange = http_exchanges[0]
openapi: OpenAPIObject = meeshkan.update_openapi(openapi, http_exchange)
Builder
Using the Meeshkan CLI, you can build OpenAPI schema from a single recordings.jsonl
file:
$ pip install meeshkan # if not done yet
$ meeshkan build --source file -i path/to/recordings.jsonl [-o path/to/output_directory]
The input file should be in JSON Lines format and every line should be in HTTP Types JSON format.For an example input file, see recordings.jsonl. The libraries listed at HTTP Types can be used to generate input files in your language of choice.
Use dash (-i -
) to read from standard input:
$ meeshkan build --source file -i - < recordings.jsonl
Building modes
You can use a mode flag to indicate how the OpenAPI spec should be built, ie:
meeshkan build --mode gen -i path/to/recordings.jsonl
Supported modes are:
- gen [default] - infer a schema from the recorded data
- replay - replay the recorded data based on exact matching
The OpenAPI schemas can be manually edited to mix the two modes.
Reading from Kafka
Install kafka
bundle: pip install meeshkan[kafka]
and set --source kafka
:
$ meeshkan build --source kafka [-o path/to/output_directory]
TODO: Configuration for Kafka
Mocking
You can use an existing OpenAPI spec, an OpenAPI spec you've built using meeshkan build
, and recordings to create a mock server using Meeshkan.
$ pip install meeshkan # if not installed yet
$ meeshkan mock
Common command-line arguments
The following commands are available in mock mode:
Argument | Description | Default |
---|---|---|
port |
Server port | 8000 |
admin_port |
Admin port | 8999 |
log_dir |
The directory containing .jsonl files for mocking directly from recorded fixtures |
logs |
schema_dir |
The directory containing .yml or .yaml OpenAPI specs used for mocking, including ones built using meeshkan build |
__unmock__ |
Recording
In addition to the builder, Meeshkan provides a recorder that can capture API traffic using a proxy and, like the builder, automatically assembles it into an OpenAPI schema in addition to storing the raw recordings.
$ pip install meeshkan # if not installed yet
$ meeshkan record
This starts Meeshkan as a reverse proxy on the default port of 8000
. For example, with curl, you can use Meeshkan as a proxy like so.
$ curl http://localhost:8000/http://api.example.com
By default the recording proxy uses the first path items to navigate to a target host.
Now you should have the logs
folder with jsonl files and the __unmock__
folder with ready openapi schemes.
For more advanced information about recording, including custom middleware, see the server documentation.
Converting
Meeshkan provides utilities to convert from certain popular recording formats to the .jsonl
format.
Converting from pcap
You can convert packet capture files to HTTP Types format using the convert
command:
meeshkan convert -i /path/to/file.pcap -o recordings.jsonl
Executable tshark must be present in your PATH.
Converter does not decrypt captured packages, so only files containing plain unencrypted HTTP traffic are currently supported.
Development
Getting started
- Create virtual environment
- Install dependencies:
pip install --upgrade -e '.[dev]'
Tests
Run tests/ with pytest
:
pytest
# or
python setup.py test
Configuration for pytest
is found in pytest.ini.
Type-checking
Run type-checking by installing pyright globally and running
pyright --lib
# or
python setup.py typecheck
Using the Pyright extension is recommended for development in VS Code.
Automated builds
Configuration for CircleCI build pipeline can be found in .circleci/config.yml.
Publishing package
- Bump the version in setup.py if the version is the same as in the published package. Commit and push.
- Run
python setup.py test
,python setup.py typecheck
andpython setup.py dist
to check everything works - To build and upload the package, run
python setup.py upload
. Insert PyPI credentials to upload the package toPyPI
. The command will also rungit tag
to tag the commit as a release and push the tags to remote.
To see what the different commands do, see Command
classes in setup.py.
Contributing
Thanks for wanting to contribute! Take a look at our development guide for notes on how to develop the package locally.
Please note that this project is governed by the Meeshkan Community Code of Conduct. By participating in this project, you agree to abide by its terms.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for meeshkan-0.2.7-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 36c1cfb31e01d21cba86bc03fb9aa0c9fe84312e94c3f93aa63f00abde4ee618 |
|
MD5 | 61156616a2747a2e21f5a588f50aca1a |
|
BLAKE2b-256 | 4257df5df66130ef1584445560e3697f633ce3b1e953ce9fc426cf3652e6f4a0 |