bigdata-research-tools 0.8.0

Bigdata.com API High-Efficiency Tools at Scale

Bigdata Research Tools

Bigdata.com API High-Efficiency Tools at Scale

This repository provides efficient toolset to use the Bigdata.com SDK.

---

Installation

Install the package from PyPI using pip:

pip install bigdata-research-tools

---

Usage

The following example demonstrates the convenient way to run multiple searches in a concurrent and rate-limited manner:

from bigdata_research_tools import run_search
from bigdata_client import Bigdata

bigdata = Bigdata()

results = run_search(bigdata=bigdata,
                     queries=YOUR_LIST_OF_QUERIES,
                     limit=1000)

---

Key Features

• Rate Limiting: Enforces a configurable query-per-minute (RPM) limit using a token bucket algorithm.
• Concurrency Support: Executes multiple search queries simultaneously with a user-defined maximum number of threads.
• Thread-Safe: Ensures safe concurrent access to shared resources with built-in thread locks.
• Flexible Configuration:
  • Set custom RPM limits and token bucket sizes.
  • Configure search parameters such as date ranges, sorting, and result limits.
• Minimum Dependencies: Requires only the bigdata_client SDK.
• Ease of Use: Includes a convenience function for running multiple searches with minimal setup.

---

RavenPack | Bigdata.com
All rights reserved © 2025