bulk-post

A near-stdlib Python CLI that fires templated HTTP requests driven by CSV data. You define the request — URL, method, body, headers — with {{placeholder}} slots, and each CSV row supplies the values that fill them: one request per row, or a multi-step request workflow per row in --workflow mode. Supports bearer or basic auth (default: no auth) with automatic 401 re-prompt, a live terminal UI with pause/resume, parallel execution, and a retry file for failed rows. Third-party dependencies: PyYAML and jsonpath-ng — both always installed, but lazily imported (PyYAML only on the --workflow code path, jsonpath-ng only on the workflow-variables code path).

Requirements

• Python 3.12+
• PyYAML — runtime dependency; lazily imported, only exercised in --workflow mode
• jsonpath-ng — runtime dependency; lazily imported, only exercised when a workflow declares variables

Installation

From PyPI

Install from PyPI:

uv tool install bulk-post   # or: pipx install bulk-post  /  pip install bulk-post
bulk-post --help

With Homebrew

Install from the tap:

brew tap true-monte-kristo/tap
brew trust true-monte-kristo/tap   # Homebrew 6.x requires trusting third-party taps once
brew install bulk-post
bulk-post --help

From source

Install globally from a checkout with uv:

uv tool install .
uv tool install . --reinstall   # re-install after changing the code

Or run directly without installing (from the repo root):

python -m bulk_post --help          # ensure pyyaml and jsonpath-ng are installed (uv run handles this)

Usage

bulk-post -u <url-template> -c <csv-file> [options]

Examples

Cancel every invoice in a CSV using DELETE:

bulk-post \
  -u "https://api.example.com/invoices/{{id}}/cancel" \
  -c invoices.csv \
  -m DELETE

PATCH with a JSON body, 200 ms between requests, verbose output:

bulk-post \
  -u "https://api.example.com/invoices/{{id}}/status" \
  -c invoices.csv \
  -m PATCH \
  -b '{"status": "cancelled", "reason": "{{reason}}"}' \
  -d 200 \
  -v

POST form-encoded data:

bulk-post \
  -u "https://api.example.com/items" \
  -c items.csv \
  -m POST \
  -b "id={{id}}&status={{status}}" \
  -C "application/x-www-form-urlencoded"

Resume after a failure at row 47:

bulk-post -u "https://api.example.com/items/{{id}}" -c items.csv -o 47

CLI flags

CSV format

The CSV must have a header row. Column names are used as placeholder names in --url, --body, and --header values. Every {{placeholder}} in the URL, body, or header values must match a column name; the script exits with an error if any are missing.

The input delimiter is detected automatically (comma, semicolon, tab, or pipe), falling back to comma when it can't be determined. The failed-rows retry CSV is written with the same delimiter as the input. There is no delimiter flag.

id,reason
1001,duplicate
1002,customer_request

Auth

Select the auth method with --auth-type / -a (default: none):

Bearer token

Pass --auth-type bearer (or -a bearer). Token resolution order: --token / -t flag → BULK_TOKEN env var → interactive prompt at startup.

If the server returns 401 mid-run, the script pauses, prompts for a fresh token, and retries the failed row automatically. This is handy when tokens are short-lived SSO bearer tokens that must be copied manually (e.g. from browser DevTools) and cannot be fetched programmatically.

Basic auth

Pass --auth-type basic (or -a basic). Credentials (user:pass) are resolved in the same order: --user / -U flag → BULK_USER env var → interactive prompt. On 401, the script prompts for new credentials and retries.

No auth (default)

The default when --auth-type is omitted (or pass --auth-type none / -a none explicitly). No Authorization header is sent.

Terminal UI

When running in an interactive terminal, a live bottom bar shows:

• Progress bar — current / total rows with a visual fill bar
• Command input — type a command and press Enter; Tab autocompletes

Available commands:

In non-TTY mode (piped input, CI, test environments) the bottom bar is skipped and no interactive commands are available.

Retry file

Rows that fail (network error, non-2xx response, or substitution error) are written to the retry file. By default this is <csv-stem>_failed.csv next to the input file. Re-run with -c <stem>_failed.csv to retry only those rows.

If no rows fail, the retry file is deleted automatically.

Workflow mode

Instead of a single --url template, you can define a multi-step workflow in a YAML file and run it with --workflow / -w.

Each CSV row fires all steps in document order. Steps within a row are always sequential; --parallel controls per-row concurrency across rows.

Workflow YAML format

workflow:
  description: Optional human-readable description  # skipped at runtime

  groupA:                          # logical grouping for shared auth
    auth:
      type: bearer                 # bearer | basic | none
      token: some_token            # optional — prompted if omitted
    endpoints:
      - step-name:                 # user-chosen name; unique within the group
          url: https://api.example.com/{{id}}
          method: POST             # default POST
          headers:
            Content-Type: application/json
            X-Custom: value
          body: '{"key": "{{col}}"}'
          on_error: stop           # stop (default) | continue
          auth:                    # step-level auth overrides group auth
            type: bearer
            token: override_token

  groupB:
    auth:
      type: basic
      user: alice
      password: secret
    endpoints:
      - another-step:
          url: https://other.example.com/{{id}}
          method: DELETE

  groupC:                          # no auth
    endpoints:
      - no-auth-step:
          url: https://public.example.com/{{id}}
          method: GET

Key rules:

• Execution order — steps fire in the order they appear in the document (top to bottom across all groups).
• Group auth — all steps in a group inherit the group's auth unless they declare their own.
• on_error — stop (default) halts remaining steps for that row and writes it to the retry file; continue logs the failure, writes the row, and proceeds to the next step.
• Placeholders — {{col}} in url, body, and header values is replaced with the matching CSV column value, same as in single-URL mode.

Retry and resume

When a step fails, the row is written to the retry file with an extra column _bulk_post_step set to the path of the first failed step (e.g. groupA/step-name). Re-running with that retry CSV skips all steps before the failed one, resuming mid-workflow automatically.

Workflow variables

A step can capture a value from an earlier step's JSON response and pass it as a {{$name}} placeholder into that step's url, headers, or body. This lets you chain steps — for example, create a resource in step 1 and use the returned id in the URL of step 2.

Variables are declared under a variables: key at group level (inherited by all endpoints in the group) and/or at endpoint level (overrides the group on name conflict):

workflow:
  groupA:
    auth:
      type: bearer
    endpoints:
      - create-item:
          url: https://api.example.com/items
          method: POST
          body: '{"name": "{{name}}"}'

  groupB:
    endpoints:
      - delete-item:
          # Capture the id from create-item's response at endpoint level
          variables:
            $id:
              source: .workflow.groupA.create-item  # leading dot and "workflow." prefix are optional
              jsonPath: $.id                        # JSONPath; first match is used
              nullable: false                       # fail this step if id is missing
          url: https://api.example.com/items/{{$id}}
          method: DELETE

Variable rules:

• Names must start with $ (e.g. $id) and are referenced as {{$id}} in URL, headers, and body.
• source is written as .workflow.<group>.<endpoint> (or just <group>/<endpoint>). It must refer to an endpoint that runs before the current step — forward and self references are rejected at startup.
• jsonPath uses full JSONPath syntax (powered by jsonpath-ng). Only the first match is used. A match that is an object or array (non-scalar) fails the step.
• nullable defaults to true. When false, a null value or no-match fails the step (row written to retry file); when true, it resolves to an empty string.
• Variable values are scoped to a single CSV row and are never shared across rows.
• All variable declarations are validated at startup — undefined references, bad names, unreachable sources, and invalid JSONPath expressions all cause an immediate exit with a clear error.

Resume/retry with variables:

When a row fails, any resolved variable values are persisted into reserved retry-CSV columns named _bulk_post_var/<source_path>/<name>. Re-running the retry CSV skips completed steps and reads these persisted values for variables whose source step was skipped.

Security note: retry CSVs may contain response-derived data (potentially sensitive) in plaintext. Do not share or commit retry CSVs that were produced from workflows using variables.

Example

bulk-post -w workflow.yaml -c rows.csv

Running tests

uv run python -m unittest discover tests/

Tests use stdlib unittest. PyYAML and jsonpath-ng are runtime dependencies (always installed); the workflow-parsing cases use PyYAML and the workflow-variable cases use jsonpath-ng. uv run provides both from uv.lock automatically; if you run python -m unittest directly, do so inside a virtualenv that has both packages.