Skip to main content

A toolset for reverse engineering and fuzzing Protobuf-based apps

Project description

pbtk - Reverse engineering Protobuf apps

Protobuf is a serialization format developed by Google and used in an increasing number of Android, web, desktop and more applications. It consists of a language for declaring data structures, which is then compiled to code or another kind of structure depending on the target implementation.

pbtk (Protobuf toolkit) is a full-fledged set of scripts, accessible through an unified GUI, that provides two main features:

  • Extracting Protobuf structures from programs, converting them back into readable .protos, supporting various implementations:

    • All the main Java runtimes (base, Lite, Nano, Micro, J2ME), with full Proguard support, (2026: this still works well but mostly with old APKs)
    • Binaries containing embedded reflection metadata (typically C++, sometimes Java and most other bindings), (2026: this still works well)
    • Web applications using the JsProtoUrl runtime. (2026: this needs an update)
  • Editing, replaying and fuzzing data sent to Protobuf network endpoints, through a handy graphical interface that allows you to edit live the fields for a Protobuf message and view the result.

The pbtk editor GUI

Installation

PBTK requires Python ≥ 3.5, PySide 6, Python-Protobuf 3, and a handful of executable programs (chromium, jad, dex2jar...) for running extractor scripts.

Ubuntu users can install it using snap:

$ sudo snap install pbtk
$ pbtk

Archlinux users can install directly through the package:

$ yay -S pbtk-git
$ pbtk

On most other distributions, you'll want to run it directly:

# For Ubuntu/Debian testing derivates:
$ sudo apt install python3-pip git openjdk-8-jre python3-qtpy-pyside6

# Then, using UV:
$ sudo snap install --classic astral-uv
$ uv tool install pbtk
$ pbtk

# Or using pipx:
$ sudo apt install pipx
$ pipx install pbtk
$ pbtk

Windows is also supported (with the same modules required). Once you run the GUI, it should warn you on what you are missing depending on what you try to do.

Command line usage (installing through package manager)

The GUI can be lanched through the main script:

pbtk

The following scripts can also be used standalone, without a GUI:

pbtk-jar-extract [-h] input_file [output_dir]
pbtk-from-binary [-h] input_file [output_dir]
pbtk-web-extract [-h] input_url [output_dir] # Needs update to work as of 2026

When install from snap, the exact commands differ:

pbtk.jar-extract [-h] input_file [output_dir]
pbtk.from-binary [-h] input_file [output_dir]
pbtk.web-extract [-h] input_url [output_dir]

Command line usage (local)

The GUI can be lanched through the main script:

uv sync # Download dependencies to the .venv folder
source .venv/bin/activate # Put the local scripts in $PATH for the current shell session
uv tool install -e . # Put the local scripts in $PATH all time
pbtk

The following scripts can also be used standalone, without a GUI:

pbtk-jar-extract [-h] input_file [output_dir]
pbtk-from-binary [-h] input_file [output_dir]
pbtk-web-extract [-h] input_url [output_dir] # Needs update to work as of 2026

Typical workflow

Let's say you're reverse engineering an Android application. You explored a bit the application with your favorite decompiler, and figured it transports Protobuf as POST data over HTTPS in a typical way.

You open PBTK and are greeted in a meaningful manner:

The welcome screen

The first step is getting your .protos into text format. If you're targeting an Android app, dropping in an APK and waiting should do the magic work! (unless it's a really exotic implementation)

Done screen

This being done, you jump to ~/.pbtk/protos/<your APK name> (either through the command line, or the button on the bottom of the welcome screen to open your file browser, the way you prefer). All the app's .protos are indeed here.

Back in your decompiler, you stumbled upon the class that constructs data sent to the HTTPS endpoint that interests you. It serializes the Protobuf message by calling a class made of generated code.

Your decompiler

This latter class should have a perfect match inside your .protos directory (i.e com.foo.bar.a.b will match com/foo/bar/a/b.proto). Either way, grepping its name should enable you to reference it.

That's great: the next thing is going to Step 2, selecting your desired input .proto, and filling some information about your endpoint.

Endpoint creation form

You may also give some sample raw Protobuf data, that was sent to this endpoint, captured through mitmproxy or Wireshark, and that you'll paste in a hex-encoded form.

Step 3 is about the fun part of clicking buttons and seeing what happens! You have a tree view representing every field in the Protobuf structure (repeated fields are suffixed by "+", required fields don't have checkboxes).

Endpoint creation form

Just hover a field to have focus. If the field is an integer type, use the mouse wheel to increment/decrement it. Enum information appears on hover too.

Here it is! You can determine the meaning of every field with that. If you extracted .protos out of minified code, you can rename fields according to what you notice they mean, by clicking their names.

Happy reversing! 👌 🎉

Local data storage

PBTK stores extracted .proto information into ~/.pbtk/protos/ (or %APPDATA%\pbtk\protos on Windows).

You can move in, move out, rename, edit or erase data from this directory directly through your regular file browser and text editor, it's the expected way to do it and won't interfere with PBTK.

HTTP-based endpoints are stored into ~/.pbtk/endpoints/ as JSON objects. These objects are arrays of pairs of request/response information, which looks like this:

[{
    "request": {
        "transport": "pburl",
        "proto": "www.google.com/VectorTown.proto",
        "url": "https://www.google.com/VectorTown",
        "pb_param": "pb",
        "samples": [{
            "pb": "!....",
            "hl": "fr"
        }]
    },
    "response": {
        "format": "other"
    }
}]

Source code structure

PBTK uses two kinds of pluggable modules internally: extractors, and transports.

  • An extractor supports extracting .proto structures from a target Protobuf implementation or platform.

Extractors are defined in src/extractors/*.py. They are defined as a method preceded by a decorator, like this:

@register_extractor(name = 'my_extractor',
                    desc = 'Extract Protobuf structures from Foobar code (*.foo, *.bar)',
                    depends={'binaries': ['foobar-decompiler']})
def my_extractor(path):
    # Load contents of the `path` input file and do your stuff...
    
    # Then, yield extracted .protos using a generator:
    for i in do_your_extraction_work():
        yield proto_name + '.proto', proto_contents
    
    # Other kinds of information can be yield, such as endpoint information or progress to display.
  • A transport supports a way of deserializing, reserializing and sending Protobuf data over the network. For example, the most commonly used transport is raw POST data over HTTP.

Transports are defined in src/utils/transports.py. They are defined as a class preceded by a decorator, like this:

@register_transport(
    name = 'my_transport',
    desc = 'Protobuf as raw POST data',
    ui_data_form = 'hex strings'
)
class MyTransport():
    def __init__(self, pb_param, url):
        self.url = url
    
    def serialize_sample(self, sample):
        # We got a sample of input data from the user.
        # Verify that it is valid in the form described through "ui_data_form" parameter, fail with an exception or return False otherwise.
        # Optionally modify this data prior to returning it.
        bytes.fromhex(sample)
        return sample
    
    def load_sample(self, sample, pb_msg):
        # Parse input data into the provided Protobuf object.
        pb_msg.ParseFromString(bytes.fromhex(sample))
    
    def perform_request(self, pb_data, tab_data):
        # Perform a request using the provided URL and Protobuf object, and optionally other transport-specific side data.
        return post(url, pb_data.SerializeToString(), headers=USER_AGENT)

Forthcoming improvements

The following could be coming for further releases:

  • Finishing the automatic fuzzing part.
  • Support for extracting extensions out of Java code.
  • Support for the JSPB (main JavaScript) runtime.
  • If there's any other platform you wish to see supported, just drop an issue and I'll look at it.

I've tried to do my best to produce thoroughly readable and commented code (except for parts that are mostly self-describing, like connecting GUI signals) for most modules, so you can contribute.

Licensing

pbtk is released under the GNU GPL license (I, hereby, etc.).

There's no formalized rule for the letter case of the project name, the rule is just about following your heart ❤

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pbtk-1.1.1.post1.tar.gz (8.3 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pbtk-1.1.1.post1-py3-none-any.whl (8.3 MB view details)

Uploaded Python 3

File details

Details for the file pbtk-1.1.1.post1.tar.gz.

File metadata

  • Download URL: pbtk-1.1.1.post1.tar.gz
  • Upload date:
  • Size: 8.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for pbtk-1.1.1.post1.tar.gz
Algorithm Hash digest
SHA256 6d561dd447fc721be68e313aea331c9b97535b2441b96913cc6ddc834a70dff5
MD5 079d9266b05e8050903e5644809e7463
BLAKE2b-256 d0b5f8032da807e0242b6df7c19c3a95752d438f75df2a20c9b887880b52dcf9

See more details on using hashes here.

Provenance

The following attestation bundles were made for pbtk-1.1.1.post1.tar.gz:

Publisher: python-publish.yml on marin-m/pbtk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pbtk-1.1.1.post1-py3-none-any.whl.

File metadata

  • Download URL: pbtk-1.1.1.post1-py3-none-any.whl
  • Upload date:
  • Size: 8.3 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for pbtk-1.1.1.post1-py3-none-any.whl
Algorithm Hash digest
SHA256 7fc0695bb6fae41c19a69d3f62cbdc61d615695b61f3d3b98a564fe79c57ebf1
MD5 ca870b00a722d4f64137a85defdf2f1b
BLAKE2b-256 343c2c301441f85c04f789c0fc5ee494147d45ff5e7e297609e344c2c395c9ad

See more details on using hashes here.

Provenance

The following attestation bundles were made for pbtk-1.1.1.post1-py3-none-any.whl:

Publisher: python-publish.yml on marin-m/pbtk

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page