ParallelRunner

Run a loop inside a single Robot Framework test case in parallel — with one clean log.html, not a merge of many.

ParallelRunner is a small Robot Framework library that lets you fan a keyword out across a thread pool from within a test case — e.g. hit 100 API endpoints, or repeat one check N times — and get back a single, correctly ordered log.html plus a structured list of per-item results.

Keyword documentation: https://cristiangarciavd.github.io/robotframework-parallelrunner/ParallelRunner.html

Installation

pip install robotframework-parallelrunner

Requires Python 3.9+ and Robot Framework 5.0+ (installed automatically). See docs/INSTALLATION.md for development installs and upgrading from the pre-release import path.

Quickstart

The keyword you want to parallelize is a method of a Python library. It receives the current item first, and an injected _logger it should log through (so logs from different threads never interleave):

# my_library.py
class MyLibrary:
    def verify_agent_data(self, agent_id, _logger=None, **kwargs):
        _logger(f"Checking agent {agent_id}")
        ...
        return result

*** Settings ***
Library    ParallelRunner
Library    my_library.MyLibrary

*** Test Cases ***
Verify Agents In Parallel
    ${agents}=    Create List    1    2    3    4    5
    ${results}=    Run Parallel Scenarios
    ...    keyword=Verify Agent Data
    ...    library=my_library.MyLibrary
    ...    for_loop_iterable=${agents}

robot --pythonpath . my_suite.robot

See docs/QUICKSTART.md for a full walkthrough and docs/API_REFERENCE.md for every parameter of Run Parallel Scenarios. For the internal design (log buffering, thread safety), see ARCHITECTURE.md.

Why This Works

Thread safety. We avoid BuiltIn().run_keyword() inside threads, which is the main cause of crashes in multi-threaded Robot Framework. We call the underlying Python methods directly instead.

No log interleaving. Each worker thread buffers its own log messages in memory. Once every task finishes, the main thread replays all buffered logs sequentially into Robot Framework's real logger (_replay_logs). The result: log.html shows logs grouped cleanly by item, even though the work happened concurrently.

Encapsulation. Callers only ever see Run Parallel Scenarios — the threading, buffering, and replay logic stay out of your .robot files.

Summary of Benefits

• Speed. Waiting time is overlapped instead of added up. In this repo's examples, 9 HTTP checks with a 1 s delay each drop from 13 s to 2 s with 9 workers, and 4 suites × 8 items × 1 s drop from 32.7 s to 4.6 s with 8 workers (see Using It Together With pabot).
• Integrity. Your log.html remains the single source of truth — no broken XML tags from concurrent writes.
• No hidden failures. If any call fails, the keyword fails too (after all calls finish and their logs are replayed), with a message listing every failed item. Pass fail_on_error=False to get a PASS/FAIL status per item and check them yourself.
• Flexibility. Pass repeat=10 to stress-test a single endpoint, or for_loop_iterable=${items} for batch validation of a whole list.
• Ergonomics. Results come back in call order (not completion order), so ${results}[0] is always the first call. Add return_values_only=True to skip the status/logs envelope entirely and unpack each call's return value straight into its own variable — e.g. ${id1} ${id2} ${id3}= Run Parallel Scenarios ... repeat=3 return_values_only=True when seeding N independent fixture rows.

How Is This Different From pabot?

pabot is the standard tool for parallel execution in the Robot Framework ecosystem, and this project is not a replacement for it — the two solve different problems and compose well together.

Concretely:

• pabot takes your existing suites/tests, runs several of them at once as separate OS processes (so they don't share memory or a GIL), and then stitches the independent output.xml results back into one report. It answers: "I have many independent tests, how do I run them all faster?"
• ParallelRunner answers a different question: "I have one test case that needs to do the same kind of work many times (loop over a list of IDs, or repeat something N times) — how do I parallelize the body of that loop without corrupting the log or needing to merge anything?" It uses a pool of threads inside a single process, and produces one already-merged, already-ordered log for that test case.
• Because ParallelRunner uses threads, not processes, it shares memory and is subject to the GIL — it is a good fit for I/O-bound work (network calls, waiting on APIs/databases) where threads spend most of their time blocked on I/O, not a good fit for CPU-bound number crunching.
• The two are complementary: you can use pabot to run many suites in parallel across processes, where individual test cases within those suites use ParallelRunner to parallelize their own inner loops across threads.

If you need to speed up "run these 50 independent test files faster," reach for pabot. If you need to speed up "this one test case loops over 100 items and I want a single readable log instead of 100 sequential HTTP round trips (or 100 merged output.xml files)," that's what ParallelRunner is for.

Using It Together With pabot

The two combine without any extra configuration: pabot splits suites (or tests) across processes, and inside each process ParallelRunner splits a loop across threads. Every pabot process gets its own ParallelRunner instance, its own thread pool and its own output.xml, so nothing is shared between processes, and pabot merges the results as usual.

pabot --processes 4 --pythonpath . atest/
pabot --processes 4 --testlevelsplit --pythonpath . atest/

Both commands pass the full acceptance suite of this repository (52 tests; checked with pabot 5.2.2).

How much time each one saves, from the offline pabot timing demo (4 suites × 8 items × 1 s of simulated I/O wait; poetry run invoke demo-pabot):

ParallelRunner removes the waiting inside each test, and pabot removes it between suites. With suites this short, pabot's process start-up (about 2.5 s here) is most of what's left, so the combination pays off more as suites get longer.

Things to keep in mind:

• Run pabot from the virtual environment where the library is installed (or via poetry run pabot ...). pabot starts whichever robot command is first on the PATH. If that belongs to another Python installation, every test that uses the library fails with No keyword with name 'Run Parallel Scenarios' found.
• Concurrency multiplies. The number of simultaneous calls is roughly pabot processes × ROBOT_THREAD_WORKERS: 4 processes × 4 workers means up to 16 concurrent requests against the same API or database. Lower one of the two if you hit rate limits, connection-pool limits or lock timeouts.
• Shared test resources can collide across processes. Data, files, user accounts, or a table that a Suite Setup resets are visible to every process. With --testlevelsplit, each test of a suite runs its own Suite Setup in its own process. For example, the examples/db_seed suite drops and recreates its table in Suite Setup, so two of its tests could interfere with each other. Keep shared state per process, or split by suite.

When Not To Use This

• CPU-bound work. Threads share Python's GIL — number crunching won't get faster this way. Use multiprocessing, or pabot (separate processes), instead.
• Your keyword mutates shared state without synchronization. Logging is made thread-safe for you; your own keyword's side effects are not. If it writes to a shared variable, file, or object without a lock, running it concurrently can race the same way any multi-threaded code can.
• You need per-item retries or a per-item timeout. Not implemented yet (see ROADMAP.md). A Robot [Timeout] on the test or keyword does interrupt the whole batch when it expires and replays the logs collected so far, but calls that are already running can't be stopped: they keep running in the background until they return.
• You need process-level isolation (a crash in one call shouldn't be able to affect another) or cross-machine parallelism — that's pabot's domain, not this library's.

Project Structure

robotframework-parallelrunner/
├── src/ParallelRunner/     # The installable library (do not depend on internals prefixed with `_`)
├── examples/               # Example libraries used by the tests, plus the pabot and log levels demos
├── atest/                  # Robot Framework acceptance suites
├── utest/                  # pytest unit tests
├── docs/                   # INSTALLATION, QUICKSTART, API_REFERENCE, generated keyword docs
├── tasks.py                # invoke tasks: tests, libdoc, demo-pabot, build, ...
└── ARCHITECTURE.md         # Technical deep dive

See ARCHITECTURE.md for the full breakdown and design principles.

Optional: UI automation with Playwright

The same thread-pool approach applies to browser automation, not just HTTP calls — see examples/playwright_ui/ for a worked example using Playwright's official sync_api. It's kept out of the default install and CI (heavy dependency, real browser download), so it's opt-in: read that folder's README before installing anything.

Contributing

See CONTRIBUTING.md and CODE_OF_CONDUCT.md. For what's done and what's planned, see ROADMAP.md and CHANGELOG.md.

License

MIT — see LICENSE.