Skip to main content

Experimental extraction of Socorro signature generation

Project description

This is an experimental extraction of the Socorro signature generation code.

Code:

https://github.com/willkg/socorro-siggen

Documentation:

Check the README.rst file

Changelog:

Check the HISTORY.rst file

Issue tracker:

https://github.com/willkg/socorro-siggen/issues

License:

MPLv2

Status:

Alpha

Installing

socorro-siggen is available on PyPI. You can install it with:

$ pip install siggen

Basic use

You can use socorro-siggen as a command line:

$ signify <JSONFILE>
SIGNATURE HERE

Alternatively:

$ cat <JSONFILE> | signify

You can use socorro-siggen as a library:

from siggen.generator import SignatureGenerator

generator = SignatureGenerator()

crash_data = {
    ...
}

ret = generator.generate(crash_data)
print(ret['signature'])

Crash data schema

This is the schema for the crash data structure:

{
  crashing_thread: <int or null>,    // Optional, The index of the crashing thread in threads.
                                     // This defaults to None which indicates there was no
                                     // crashing thread identified in the crash report.

  threads: [                         // Optional, list of stack traces for c/c++/rust code.
    {
      frames: [                      // List of one or more frames.
        {
          function: <string>,        // Optional, The name of the function.
                                     // If this is ``None`` or not in the frame, then signature
                                     // generation will calculate something using other data in
                                     // the frame.

          module: <string>,          // Optional, name of the module
          file: <string>,            // Optional, name of the file
          line: <int>,               // Optional, line in the file
          module_offset: <string>,   // Optional, offset in hex in the module for this frame
          offset: <string>           // Optional, offset in hex for this frame

                                     // Signature parts are computed using frame data in this
                                     // order:

                                     // 1. if there's a function (and optionally line)--use
                                     //    that
                                     // 2. if there's a file and a line--use that
                                     // 3. if there's an offset and no module/module_offset--use
                                     //    that
                                     // 4. use module/module_offset
        }
        // ... additional frames
      ],

      thread_name: <string>,         // Optional, The name of the thread.
                                     // This isn't used, yet, but might be in the future for
                                     // debugging purposes.

      frame_count: <int>             // Optional, This is the total number of frames. This
                                     // isn't used.
    },
    // ... additional threads
  ],

  java_stack_trace: <string>,        // Optional, If the crash is a Java crash, then this will
                                     // be the Java traceback as a single string. Signature
                                     // generation will split this string into lines and then
                                     // extract frame information from it to generate the
                                     // signature.

                                     // FIXME(willkg): Write up better description of this.

  oom_allocation_size: <int>,        // Optional, The allocation size that triggered an
                                     // out-of-memory error. This will get added to the
                                     // signature if one of the indicator functions appears in
                                     // the stack of the crashing thread.

  abort_message: <string>,           // Optional, The abort message for the crash, if there is
                                     // one. This is added to the beginning of the signature.

  hang_type: <int>,                  // Optional.
                                     // 1 here indicates this is a chrome hang and we look at
                                     // thread 0 for generation.
                                     // -1 indicates another kind of hang.

  async_shutdown_timeout: <text>,    // Optional, This is a text field encoded in JSON with
                                     // "phase" and "conditions" keys.
                                     // FIXME(willkg): Document this structure better.

  jit_category: <string>,            // Optional, If there's a JIT classification in the
                                     // crash, then that will override the signature

  ipc_channel_error: <string>,       // Optional, If there is an IPC channel error, it
                                     // replaces the signature.

  ipc_message_name: <string>,        // Optional, This gets added to the signature if there
                                     // was an IPC message name in the crash.

  additional_minidumps: <string>,    // Optional, A crash report can contain multiple minidumps.
                                     // This is a comma-delimited list of minidumps other than
                                     // the main one that the crash had.

                                     // Example: "browser,flash1,flash2,content"

  mdsw_status_string: <string>,      // Optional, Socorro-generated
                                     // This is the minidump-stackwalk status string. This
                                     // gets generated when the Socorro processor runs the
                                     // minidump through minidump-stackwalk. If you're not
                                     // using minidump-stackwalk, you can ignore this.

  moz_crash_reason: <string>,        // Optional, This is the MOZ_CRASH_REASON value. This
                                     // doesn't affect anything unless the value is
                                     // "MOZ_RELEASE_ASSERT(parentBuildID == childBuildID)".

  os: <string>,                      // Optional, The name of the operating system. This
                                     // doesn't affect anything unless the name is "Windows
                                     // NT" in which case it will lowercase module names when
                                     // iterating through frames to build the signature.
}

Missing keys in the structure are treated as None, so you can pass in a minimal structure with just the parts you define.

Examples

Example almost minimal, somewhat nonsense crash_data.json:

{
    "os": "Linux",
    "crashing_thread": 0,
    "threads": [
        {
            "frames": [
                {
                    "frame": 0,
                    "function": "SomeFunc",
                    "line": 20,
                    "file": "somefile.cpp",
                    "module": "foo.so.5.15.0",
                    "module_offset": "0x37a92",
                    "offset": "0x7fc641052a92"
                },
                {
                    "frame": 1,
                    "function": "SomeOtherFunc",
                    "line": 444,
                    "file": "someotherfile.cpp",
                    "module": "bar.so",
                    "module_offset": "0x39a55",
                    "offset": "0x7fc641044a55"
                }
            ]
        }
    ]
}

That produces this output:

$ cat crash_data.json | signify
{
  "notes": [],
  "proto_signature": "SomeFunc | SomeOtherFunc",
  "signature": "SomeFunc"
}

Release process

  1. Create branch

  2. Update version and release date in siggen/__init__.py

  3. Update HISTORY.rst

  4. Push the branch, create a PR, review it, merge it

  5. Create a signed tag, push to github:

    git tag -s v0.1.0
    git push --tags [REMOTE] master
  6. Build:

    python setup.py sdist bdist_wheel
  7. Upload to PyPI:

    twine upload dist/*

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

siggen-0.2.0.tar.gz (45.7 kB view details)

Uploaded Source

Built Distribution

siggen-0.2.0-py3-none-any.whl (52.1 kB view details)

Uploaded Python 3

File details

Details for the file siggen-0.2.0.tar.gz.

File metadata

  • Download URL: siggen-0.2.0.tar.gz
  • Upload date:
  • Size: 45.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.18.4 setuptools/39.1.0 requests-toolbelt/0.8.0 tqdm/4.23.3 CPython/2.7.14

File hashes

Hashes for siggen-0.2.0.tar.gz
Algorithm Hash digest
SHA256 4a59e52e93fff8965775565d1193c47a5d94aa6e403f519e86b09244e01de7eb
MD5 d7b250eeb2c621dbe37856d31c7a6648
BLAKE2b-256 3ead16bf0738934293f43d7fc837f21e052c582408857af57c71a11ee28bb45f

See more details on using hashes here.

File details

Details for the file siggen-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: siggen-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 52.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.18.4 setuptools/39.1.0 requests-toolbelt/0.8.0 tqdm/4.23.3 CPython/2.7.14

File hashes

Hashes for siggen-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 4024e7360d48ce12c852eb3e4d6ad634853b9e5f65df187d18e695a34de83f03
MD5 48cbddb1882cc4a8b3ba8e88b8bd101d
BLAKE2b-256 a9c918a69e82f263d57d384cdf19556b799804c78e41ff791cbff98f628bf4d9

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page