crtstream 0.1.1

Certificate Transparency log streamer

crtstream

crtstream is a lightweight Certificate Transparency (CT) log streamer written in Python. It continuously monitors CT logs and outputs newly issued certificates in real time.

It can output:

• domain names only
• human-readable summaries
• full structured JSON suitable for pipelines and storage

---

✨ Features

• 📡 Real-time streaming from multiple CT logs
• 📜 Supports X.509 and precert entries (RFC 6962)
• 🧵 Multi-threaded (one thread per log)
• 📦 Installable via pip
• 🔌 Clean CLI interface
• 🧾 JSON output for SIEM / data pipelines
• ⚙️ Simple configuration via logs.json

---

📦 Installation

From PyPI (recommended)

pip install crtstream

Or with pipx:

pipx install crtstream

From source (development)

git clone https://github.com/yourname/crtstream.git
cd crtstream
pip install -e .

---

⚙️ Configuration

crtstream reads CT log endpoints from a local JSON file.

logs.json (default)

Create a file named logs.json in the directory where you run crtstream:

{
  "google_us": "https://ct.googleapis.com/logs/us1/argon2025h2",
  "google_eu": "https://ct.googleapis.com/logs/eu1/xenon2025h2",
  "cloudflare": "https://ct.cloudflare.com/logs/nimbus2025",
  "digicert_yeti": "https://yeti2025.ct.digicert.com/log"
}

You may also specify a custom file:

crtstream --logs /path/to/logs.json

---

🚀 Usage

Basic usage

crtstream

Print only domains

crtstream --domains-only

Output full JSON (one object per line)

crtstream --json

Custom logs file

crtstream --logs logs.json --json

---

🧾 JSON Output Format

When using --json, each line is a JSON object:

{
  "log_name": "google_us",
  "log_url": "https://ct.googleapis.com/logs/us1/argon2025h2",
  "entry_index": 123456,
  "timestamp": "2025-01-01T12:00:00Z",
  "certificate": {
    "domains": ["example.com", "www.example.com"],
    "subject": { "commonName": "example.com" },
    "issuer": { "commonName": "Google Trust Services" },
    "validity": {
      "not_before": "2025-01-01T00:00:00",
      "not_after": "2025-04-01T23:59:59"
    },
    "serial_number": "0x123abc",
    "public_key": {
      "type": "RSAPublicKey",
      "key_size": 2048
    },
    "version": "v3"
  },
  "raw_entry": { "...": "..." }
}

This format is ideal for:

• log ingestion
• data analysis
• security monitoring
• threat intelligence pipelines

---

🧠 How it works

• Polls each CT log every few seconds
• Fetches new entries using /ct/v1/get-entries
• Parses X.509 or precert entries
• Extracts SAN domains and metadata
• Streams output continuously

---

⚠️ Notes & Limitations

• This tool does not persist state (restarts from near-tip)
• CT logs may rate-limit or temporarily fail
• No filtering is applied by default

---

🤝 Contributing

Contributions are welcome.

1. Fork the repository
2. Create a feature branch
3. Submit a pull request

Please keep changes focused and well-documented.

---

📜 License

This project is licensed under the MIT License. See the LICENSE file for details.