HeaderBreakdown

converts a collection of HTTP headers into a dictionary structure for automated analysis; creates parsed and analyzed objects; accepts strings, lists, or dictionaries of header values

Installation

pip install headerbreakdown
or
pip3 install headerbreakdown
or
python3 -m pip install headerbreakdown

Example Output

{
    "_direction": "response",
    "_httpversion": "HTTP/1.1",
    "_responsecode": "302",
    "_responsephrase": "Found",
    "location": {
        "key": "Location",
        "value": [
            "https://www.google.com/?gws_rd=ssl"
        ]
    },
    "cachecontrol": {
        "key": "Cache-Control",
        "value": [
            "private"
        ]
    },
    "contenttype": {
        "key": "Content-Type",
        "value": [
            "text/html; charset=UTF-8"
        ],
        "subvalues": [
            "charset=UTF-8",
            "text/html"
        ],
        "microvalues": [
            {
                "microkey": "charset",
                "microvalue": "UTF-8"
            }
        ]
    },
    "date": {
    ...

Example Usage

from headerbreakdown import HeaderBreakdown as hb
import json
# header with multiple Host and User-Agent values
someheader = "GET /?gws_rd=ssl HTTP/1.1\r\nHost: www.google.com\r\nHost: www.bing.com\r\nHost: www.yahoo.com\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/99.0\r\nCookie: 1P_JAR=2021-03-13-04"
# create the object
h = hb(someheader)
# print only the parsed version of the header
print(h.parsed)
# print the analyzed version with subvalues, minivalues, and microkeys and microvalues
print(h.analyzed)
# print as json
print(json.dumps(h.analyzed, indent=4))
# etc

Available Attributes

• Note that method and path are also presented as normalized keys alongside other request header keys (host will also be present)

Releases and Updates

• 2022-08-11
  • minor edits and comments for readability
  • made hb_unittest.py and commented out unit_tests() in headerbreakdown.py
• 2022-08-10
  • converted method and path to lists to accomodate illegal headers such as multiple methods and paths
  • fixed attributes to only set if not already present
    • prevents "requestrequest" or "responseresponse" for _direction if illegal headers are set
• 2022-03-25
  • complete overhaul
  • accepts strings, lists, and dictionaries as input
  • simplified output dict structure
  • standardized keys
  • removed json outputs
  • TO DO - convert metadata string attributes to lists (smuggling)
• 2021-04-06
  • added nested_direction_json/output, ex. {"headers":{"request":{...}}}
  • so direction gets captured and headers do not get overwritten if processing a capture with both sides of the communication
  • the nested_direction_* attributes will be type None when processing a single, direction-ambiguous header (ex. "Set-Cookie: k1=v1;k2=v2")
• 2021-04-01
  • minor fix for HTTP/ detection
• 2021-03-23
  • minor edits, added summary and nested_output/nested_json attributes, ex. {"headers":{...}}
• 2021-03-13
  • first release