rgapi

rgapi is a Python API for ripgrep-style walking and search. It is meant for Python code that wants fd-style file discovery or rg-style searching without shelling out.

It uses the same ignore, grep-regex, and grep-searcher crates that ripgrep uses for walking, regex matching, and file scanning. Walking and searching run in parallel by default. Most expensive work stays in Rust.

Overview

For common file discovery and search:

from rgapi import fd, ls, rg, rg_iter

fd(".", ext="py", exclude="test_*.py")
ls("src")
for row in rg_iter("TODO", ".", include="*.py", context=2): print(row.asdict())
rg("TODO", ".", ext="py", skip_dir=".venv", paths=True)

For cell-aware search of Jupyter notebooks (see Notebooks):

from rgapi import nbrg

nbrg("read_csv", ".", cell_context=1)

Every walk and search has an async twin, plus streaming forms that yield results as they are found (see Async):

from rgapi import fda, rga, rga_iter, nbrga, nbrga_iter

await rga("TODO", ".", ext="py", timeout_ms=200)
async for row in rga_iter("TODO", "."): print(row)

For direct access to the regex, search, and walk pieces:

from rgapi import compile, search_path, search_text, walk

matcher = compile("TODO")
matcher.is_match("TODO")
matcher.finditer("TODO TODO")

walk(".")
search_text(matcher, "alpha\nTODO\nomega\n", path="memory.txt", context=1)
search_path(matcher, "src/lib.rs", display_path="src/lib.rs")

Install

pip install rgapi

Semantics

fd and walk return slash-separated paths relative to root. They use the ignore crate, so .gitignore, .ignore, and the usual ripgrep filters apply by default. .rgignore files are also honored and take precedence over .gitignore. Hidden files are skipped unless hidden=True. Pass ignore=False to disable all ignore filtering (including .rgignore). Symlinks are not followed unless follow_links=True; same_file_system=True avoids crossing filesystem boundaries. Traversal is parallel, and result order is not guaranteed; use sorted(...) if order matters. root arguments accept str or pathlib.Path and expand ~; ., ./, and paths containing .. work across the sync and async APIs.

fd adds fd-like filtering on top of walk: pattern is a smart-case regex matched against each basename, and include/exclude use glob syntax. Lowercase patterns match case-insensitively; a pattern containing uppercase letters is case-sensitive. Use path_re when matching the slash-separated relative path instead. glob= is accepted as an alias for include=. A basename glob such as *.py also matches recursively, so it finds src/app.py. Use ext="py" or ext=["py", "rs"] for extension filters, which compose as AND with include/glob (so include="src/*", ext="py" means src/* and *.py, like combining rg -g with -t); use min_depth=/max_depth= to bound recursion, and max_filesize= to skip files above a byte limit.

ls lists like the shell command: it is fd with defaults flipped to one level (max_depth=1), directories included, ignore rules off, and results sorted by name. hidden=True is ls -a, and every fd filter still applies.

fd_iter is the lazy form of fd, yielding FileEntry paths as the walk finds them, and takes every fd filter. It has no timeout_ms, since a consumer that stops asking for paths ends the walk itself.

path_re and skip_path_re are regex filters on slash-separated relative paths. They filter returned paths or searched files, but do not control traversal. skip_dir uses glob syntax to prune matching directory subtrees, and skip_dir_re does the same with regex.

rg and rg_iter return structured rows rather than raw CLI text. They accept the same include, exclude, glob, ext, path_re, skip_path_re, skip_dir, skip_dir_re, min_depth, max_depth, max_filesize, follow_links, and same_file_system filters as fd. Each row is a SearchLine with:

kind         'match', 'before', 'after', or 'context'
path         path relative to root
line_number  1-based line number
lnhash       exhash-style `lineno|hash|` address for the line
line         line text without the trailing newline
matches      list of (start, end) byte offsets for match rows

rg, search_text, and search_path return SearchResults by default, a list subclass whose str() and notebook pretty display are rg-style multiline text. rg_iter yields rows lazily.

SearchLine has a structured repr, an rg-style str (the line is truncated to 120 chars with a trailing … for display; repr and asdict() keep the full line), and SearchLine.asdict() returns row fields as a plain Python dict. Pass rg(..., lnhashs=True) or rg_iter(..., lnhashs=True) to show lnhash addresses instead of line numbers in row display while keeping line_number available. rg(..., paths=True) returns unique matched paths, and rg(..., count=True) returns the total number of match spans. paths and count cannot both be set.

fd, walk, ls, and rg/nbrg with paths=True return PathResults, a list of FileEntry rows. A FileEntry is a str subclass holding the relative path, so all string uses keep working, and it stats itself lazily on first access: stat is a cached os.lstat result (None if the path has vanished), with size, mtime, and is_dir derived from it. A PathResults displays as an ls -l-style long listing, capped at rgapi.MAX_REPR rows with a final … N more line, so stats are read only for displayed rows; str() is still one plain path per line, and list(res) shows plain paths. rg(..., timeout_ms=200) and fd(..., timeout_ms=200) stop at the deadline and return whatever was collected by then; walk, ls, and the async forms take it too. Results record how they ended: stop_reason is None for a complete result, "max_results" when truncated by max_results, or "timeout" when a deadline hit, and complete is true when stop_reason is None. count=True returns a plain int, which cannot carry the flag, so it rejects timeout_ms.

before_context, after_context, and context are like rg -B, rg -A, and rg -C. Files containing NUL bytes or invalid UTF-8 are skipped.

Block summaries

rg(..., summary=True) returns one row per blank-line-delimited block instead of one row per matching line. Empty and whitespace-only lines delimit blocks. A block containing several matching lines appears once and keeps every matching SearchLine in matches.

rg("TODO", ".", summary=True, context=1, maxlen=120)

The result is BlockResults, a list of SearchBlock objects. Each block has path, block_index, start_line, end_line, start_lnhash, end_lnhash, kind, full source, and matches. Its display is path:start-end:source for matches and path:start-end-source for context. With lnhashs=True, the numeric range becomes copyable boundary addresses such as path:4|a3f2|,6|b1c3|:source. Embedded newlines are shown as \n; maxlen limits displayed source without changing source or asdict().

In summary mode, before_context, after_context, and context count neighbouring blocks. max_results counts matching blocks and retains their block context. summary=True cannot be combined with paths or count; it can be combined with lnhash when copyable block boundaries are useful.

Search is case-sensitive by default, matching rg. Use smart_case=True for rg --smart-case behavior, or case_sensitive=False to force case-insensitive matching.

Notebooks

nbrg searches Jupyter .ipynb files cell-by-cell, so results are cells rather than raw JSON lines, and each match is identified by its cell id (the nbformat cell/message id) rather than a line number. Searching a notebook with plain rg matches the escaped JSON text (including outputs and metadata) and reports meaningless JSON line numbers; nbrg instead searches each cell's reconstructed source and reports the cell id, which is stable across edits and points at the actual unit you work with.

from rgapi import nbrg

nbrg("read_csv", ".")                  # cells whose source matches, across all notebooks under "."
nbrg("read_csv", ".", cell_context=1)  # also include neighbouring cells as context

Notebooks are walked, parsed, and matched together in one parallel Rust pass, using the same regex engine as rg, so regex behaviour and the case_sensitive/smart_case flags match rg. Only cell source is searched, not outputs or metadata. nbrg accepts the same discovery filters as fd/rg (include, exclude, glob, hidden, max_depth, skip_dir, …).

nbrg returns NbResults, a list of NbCell. Each NbCell has:

path         notebook path relative to root
cell_index   0-based position of the cell in the notebook
cell_id      nbformat cell id (falls back to the cell index for notebooks without ids)
cell_type    'code', 'markdown', or 'raw'
kind         'match' or 'context'
source       full cell source
matches      list of SearchLine rows for the matched lines within the cell

NbCell.asdict() returns those fields as a plain dict (with matches as SearchLine dicts). str() and pretty display show one newline-escaped line per cell, keyed by cell_id: path:cell_id:source for matches and path:cell_id-source for context. A match row starts at its first matched line: earlier lines display as …[Ln] (n the matched line's 1-based number in the cell), except that a leading #| directive line is kept, e.g. #| export…[L4]needle here. maxlen controls the displayed source length and defaults to 120; the full source remains in source and asdict(). A cell with several matches appears once, with every hit collected in matches.

cell_context=N includes the N cells before and after each matching cell as kind="context" rows (deduplicated per notebook).

nbrg_iter yields NbCell rows lazily as notebooks are parsed. nbrg also accepts max_results (at most that many cells, after sorting by path and cell index), count=True (number of matching cells), and timeout_ms= with the same stop_reason semantics as rg.

Notebook walking, parsing, and matching all happen in parallel in Rust, in the same pass as the file walk. Parsing uses a lean model that reads only each cell's id, cell_type, and source and skips outputs and metadata, so large embedded outputs (images, plots) are never materialized. search_nb(pattern, path, ...) searches a single notebook file the same way.

Async

fda, rga, and nbrga are awaitable twins of fd, rg, and nbrg, and fda_iter, rga_iter and nbrga_iter are async generators that yield rows as the search finds them. All take the same arguments and return the same types as their sync counterparts.

from rgapi import fda, rga, rga_iter

await fda(".", ext="py")
res = await rga("TODO", ".", timeout_ms=200)
if not res.complete: print(f"partial results: {res.stop_reason}")
async for row in rga_iter("TODO", "."): ...

None of this uses asyncio.to_thread or the loop's executor. The walk and search run on Rust threads, and a single callback settles the awaited future (or feeds the generator's queue) through loop.call_soon_threadsafe, so the event loop never blocks and contextvars behave normally.

Cancellation cleans up the Rust workers automatically. Wrapping a call in asyncio.wait_for or asyncio.timeout, cancelling the task (as starlette does when a client disconnects), or leaving an async for early all stop the search within about one row. One caveat comes from the language rather than the library: break inside async for only finalizes the generator at GC time, so for prompt cleanup wrap the iterator in contextlib.aclosing:

async with aclosing(rga_iter("TODO", ".")) as it:
    async for row in it:
        if enough(row): break

The streaming forms suit incremental display, such as pushing each batch of results to a browser as it arrives. The collected forms with timeout_ms give the best results available within a budget, and asyncio.wait_for gives timeout-as-failure. Pick per call site.

Benchmarks

tools/bench.py compares the rg CLI with in-process rgapi. Run it against a release build. One run on this machine, using best time from seven repeats:

fixture rg rgapi
6 x 2 MB files, 2 matches 6.54 ms 1.44 ms
800 x 1.5 KB files, 2 matches 13.90 ms 10.94 ms
tiny dir, repeated 30x 5.92 ms 2.14 ms

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for fastai from gravatar.com fastai

Credits

Author: Jeremy Howard

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.10

Provides Extra

dev

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rgapi-0.1.19.tar.gz (50.0 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

rgapi-0.1.19-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

rgapi-0.1.19-cp313-cp313-macosx_11_0_arm64.whl (1.6 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

rgapi-0.1.19-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

rgapi-0.1.19-cp312-cp312-macosx_11_0_arm64.whl (1.6 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

rgapi-0.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

rgapi-0.1.19-cp311-cp311-macosx_11_0_arm64.whl (1.6 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

rgapi-0.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

rgapi-0.1.19-cp310-cp310-macosx_11_0_arm64.whl (1.6 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

File details

Details for the file rgapi-0.1.19.tar.gz.

File metadata

  • Download URL: rgapi-0.1.19.tar.gz
  • Upload date:
  • Size: 50.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rgapi-0.1.19.tar.gz
Algorithm Hash digest
SHA256 273dd3d241e19b6450e39d96b157394eff206ef44aa18ae05daebeaa30dd2e85
MD5 cd6150c5fe8d89f05c89b33f6dbb1f8e
BLAKE2b-256 1abef499c1ecb7a8adfa8d86a3a4186cc944d1883e4adeb42c32651a53940979

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19.tar.gz:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 41c0f72fa9429aab07ab2f38831d9912269c07e1cffbc2f7f74e77933eeb6ef1
MD5 a089dcc2e2d672ec95bd5cf81230a148
BLAKE2b-256 159c7a4d9b7c1aedb9748beda9693080fbb0e59dc95e7720e25ff8d66a71c6b7

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b6f45f589893b258c21d6bba130967fca6512f39ae36a8cc610b9627b746bd2b
MD5 db716e4914b068ceadb6eb6dc2afe165
BLAKE2b-256 4cf8cc4376bca5da3760aac327009b51167ebe97731eeea8f39a82b2981cd0ca

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f5d8bd46aea991f2cb8b7f460720cef85c710e49e767c30bfb0318bfac969c3a
MD5 02db674648d9d67661263670141ec6be
BLAKE2b-256 ecb0f8bcaebd8a415be724d98eb2c0ca3a4f207d5d4b028cf77069f478a90436

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 45b6f00b4da36e8f5ea454e88d586fda4301829df7e40a0ef276de4191977e7a
MD5 2a4d67a4d297e3a837ba01488d492779
BLAKE2b-256 dd21ed1d216b03dc667a1cf63cfc0eaf67120415d6d7aa1ba1562cbe0c856c3c

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f2b0a0986c1def6e640db26c18fd10bfd424135bdfae2a76d08ec3448a68ca9b
MD5 fd795756ba309199f19181c79626710a
BLAKE2b-256 b917d454eb56e244161745a1dfa7b1eb19e7dcabc185a3a883746743f2166166

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 61c8f8c804c6967666ec6c95970a76237bb35b2eeaf3e8cfb190d9e84553c5cb
MD5 ae0d3f87946966cd890940ece5b2a561
BLAKE2b-256 a4b632d85fabbfa25bcbe47096fc7332b73d373e2c4e7149c83e5773083997ad

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f5f8b8d00dd7d2b183219253239f089e395c9c08582771ac004c62f3e407d288
MD5 ed79e7e0d9a1bbeb9b7c9ae555dcd767
BLAKE2b-256 4bad4ee16661ed14c8f1fd566926bd63c648cc6fdd9ff73168c5527675e96cfe

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.19-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.19-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d8fb1ebd5661b5c23e0bb6ed4af72dca3275f085f8f2b5946f20fffc75af340f
MD5 dfd0a1de8730162de29dc815abf67285
BLAKE2b-256 2bb49c76efc5ef6ce79ce2a18bdf90f5a3de3ed5280ac0d9406b350224089722

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.19-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page