rgapi

rgapi is a Python API for ripgrep-style walking and search. It is meant for Python code that wants fd-style file discovery or rg-style searching without shelling out.

It uses the same ignore, grep-regex, and grep-searcher crates that ripgrep uses for walking, regex matching, and file scanning. Walking and searching run in parallel by default. Most expensive work stays in Rust.

Overview

For common file discovery and search:

from rgapi import fd, rg, rg_iter

fd(".", ext="py", exclude="test_*.py")
for row in rg_iter("TODO", ".", include="*.py", context=2): print(row.asdict())
rg("TODO", ".", ext="py", skip_dir=".venv", paths=True)

For cell-aware search of Jupyter notebooks (see Notebooks):

from rgapi import nbrg

nbrg("read_csv", ".", cell_context=1)

Every walk and search has an async twin, plus streaming forms that yield results as they are found (see Async):

from rgapi import fda, rga, rga_iter, nbrga, nbrga_iter

await rga("TODO", ".", ext="py", timeout_ms=200)
async for row in rga_iter("TODO", "."): print(row)

For direct access to the regex, search, and walk pieces:

from rgapi import compile, search_path, search_text, walk

matcher = compile("TODO")
matcher.is_match("TODO")
matcher.finditer("TODO TODO")

walk(".")
search_text(matcher, "alpha\nTODO\nomega\n", path="memory.txt", context=1)
search_path(matcher, "src/lib.rs", display_path="src/lib.rs")

Install

pip install rgapi

Semantics

fd and walk return slash-separated paths relative to root. They use the ignore crate, so .gitignore, .ignore, and the usual ripgrep filters apply by default. .rgignore files are also honored and take precedence over .gitignore. Hidden files are skipped unless hidden=True. Pass ignore=False to disable all ignore filtering (including .rgignore). Symlinks are not followed unless follow_links=True; same_file_system=True avoids crossing filesystem boundaries. Traversal is parallel, and result order is not guaranteed; use sorted(...) if order matters. root arguments accept str or pathlib.Path and expand ~; search_path also accepts path-like file paths. Display labels such as display_path are stringified without expansion.

fd adds fd-like filtering on top of walk: pattern is a substring match on the relative path, and include/exclude use glob syntax. glob= is accepted as an alias for include=. A basename glob such as *.py also matches recursively, so it finds src/app.py. Use ext="py" or ext=["py", "rs"] for extension filters, min_depth=/max_depth= to bound recursion, and max_filesize= to skip files above a byte limit.

path_re and skip_path_re are regex filters on slash-separated relative paths. They filter returned paths or searched files, but do not control traversal. skip_dir uses glob syntax to prune matching directory subtrees, and skip_dir_re does the same with regex.

rg and rg_iter return structured rows rather than raw CLI text. They accept the same include, exclude, glob, ext, path_re, skip_path_re, skip_dir, skip_dir_re, min_depth, max_depth, max_filesize, follow_links, and same_file_system filters as fd. Each row is a SearchLine with:

kind         'match', 'before', 'after', or 'context'
path         path relative to root
line_number  1-based line number
lnhash       exhash-style `lineno|hash|` address for the line
line         line text without the trailing newline
matches      list of (start, end) byte offsets for match rows

rg, search_text, and search_path return SearchResults by default, a list subclass whose str() and notebook pretty display are rg-style multiline text. rg_iter yields rows lazily.

SearchLine has a structured repr, an rg-style str (the line is truncated to 120 chars with a trailing … for display; repr and asdict() keep the full line), and SearchLine.asdict() returns row fields as a plain Python dict. Pass rg(..., lnhash=True) or rg_iter(..., lnhash=True) to show lnhash addresses instead of line numbers in row display while keeping line_number available. rg(..., paths=True) returns unique matched paths, and rg(..., count=True) returns the total number of match spans. paths and count cannot both be set.

fd, walk, and rg(..., paths=True) return PathResults, a list subclass displayed one path per line. rg(..., timeout_ms=200) stops the search at the deadline and returns whatever was collected by then. Results record how they ended: stop_reason is None for a complete result, "max_results" when truncated by max_results, or "timeout" when a deadline hit, and complete is true when stop_reason is None. count=True returns a plain int, which cannot carry the flag, so it rejects timeout_ms.

before_context, after_context, and context are like rg -B, rg -A, and rg -C. Files containing NUL bytes or invalid UTF-8 are skipped.

Search is case-sensitive by default, matching rg. Use smart_case=True for rg --smart-case behavior, or case_sensitive=False to force case-insensitive matching.

Notebooks

nbrg searches Jupyter .ipynb files cell-by-cell, so results are cells rather than raw JSON lines, and each match is identified by its cell id (the nbformat cell/message id) rather than a line number. Searching a notebook with plain rg matches the escaped JSON text (including outputs and metadata) and reports meaningless JSON line numbers; nbrg instead searches each cell's reconstructed source and reports the cell id, which is stable across edits and points at the actual unit you work with.

from rgapi import nbrg

nbrg("read_csv", ".")                  # cells whose source matches, across all notebooks under "."
nbrg("read_csv", ".", cell_context=1)  # also include neighbouring cells as context

Notebooks are walked, parsed, and matched together in one parallel Rust pass, using the same regex engine as rg, so regex behaviour and the case_sensitive/smart_case flags match rg. Only cell source is searched, not outputs or metadata. nbrg accepts the same discovery filters as fd/rg (include, exclude, glob, hidden, max_depth, skip_dir, …).

nbrg returns NbResults, a list of NbCell. Each NbCell has:

path         notebook path relative to root
cell_index   0-based position of the cell in the notebook
cell_id      nbformat cell id (falls back to the cell index for notebooks without ids)
cell_type    'code', 'markdown', or 'raw'
kind         'match' or 'context'
source       full cell source
matches      list of SearchLine rows for the matched lines within the cell

NbCell.asdict() returns those fields as a plain dict (with matches as SearchLine dicts). str()/pretty display is one truncated, newline-escaped line per cell, keyed by cell_id rather than a line number: path:cell_id:source for matches and path:cell_id-source for context cells. A cell with several matches appears once, with every hit collected in matches.

cell_context=N includes the N cells before and after each matching cell as kind="context" rows (deduplicated per notebook).

nbrg_iter yields NbCell rows lazily as notebooks are parsed. nbrg also accepts max_results (at most that many cells, after sorting by path and cell index), count=True (number of matching cells), and timeout_ms= with the same stop_reason semantics as rg.

Notebook walking, parsing, and matching all happen in parallel in Rust, in the same pass as the file walk. Parsing uses a lean model that reads only each cell's id, cell_type, and source and skips outputs and metadata, so large embedded outputs (images, plots) are never materialized. search_nb(pattern, path, ...) searches a single notebook file the same way.

Async

fda, rga, and nbrga are awaitable twins of fd, rg, and nbrg, and rga_iter and nbrga_iter are async generators that yield rows as the search finds them. All take the same arguments and return the same types as their sync counterparts.

from rgapi import fda, rga, rga_iter

await fda(".", ext="py")
res = await rga("TODO", ".", timeout_ms=200)
if not res.complete: print(f"partial results: {res.stop_reason}")
async for row in rga_iter("TODO", "."): ...

None of this uses asyncio.to_thread or the loop's executor. The walk and search run on Rust threads, and a single callback settles the awaited future (or feeds the generator's queue) through loop.call_soon_threadsafe, so the event loop never blocks and contextvars behave normally.

Cancellation cleans up the Rust workers automatically. Wrapping a call in asyncio.wait_for or asyncio.timeout, cancelling the task (as starlette does when a client disconnects), or leaving an async for early all stop the search within about one row. One caveat comes from the language rather than the library: break inside async for only finalizes the generator at GC time, so for prompt cleanup wrap the iterator in contextlib.aclosing:

async with aclosing(rga_iter("TODO", ".")) as it:
    async for row in it:
        if enough(row): break

The streaming forms suit incremental display, such as pushing each batch of results to a browser as it arrives. The collected forms with timeout_ms give the best results available within a budget, and asyncio.wait_for gives timeout-as-failure. Pick per call site.

Benchmarks

tools/bench.py compares the rg CLI with in-process rgapi. Run it against a release build. One run on this machine, using best time from seven repeats:

fixture rg rgapi
6 x 2 MB files, 2 matches 6.54 ms 1.44 ms
800 x 1.5 KB files, 2 matches 13.90 ms 10.94 ms
tiny dir, repeated 30x 5.92 ms 2.14 ms

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.1.19)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for fastai from gravatar.com fastai

Credits

Author: Jeremy Howard

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.10

Provides Extra

dev

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rgapi-0.1.13.tar.gz (38.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

rgapi-0.1.13-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

rgapi-0.1.13-cp313-cp313-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

rgapi-0.1.13-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

rgapi-0.1.13-cp312-cp312-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

rgapi-0.1.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

rgapi-0.1.13-cp311-cp311-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

rgapi-0.1.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

rgapi-0.1.13-cp310-cp310-macosx_11_0_arm64.whl (1.5 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

File details

Details for the file rgapi-0.1.13.tar.gz.

File metadata

  • Download URL: rgapi-0.1.13.tar.gz
  • Upload date:
  • Size: 38.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for rgapi-0.1.13.tar.gz
Algorithm Hash digest
SHA256 9c2d2aa6e0c2e0a137540a249a9dd3f7594d04d8302a7110d10ceb1694fa273c
MD5 e0557be85de26be0dffb396f99bf4156
BLAKE2b-256 1c8a73e560a3856122b471467851b1886638e902f134857cf3cb3863a5aa9589

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13.tar.gz:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 99b2853bd7fdcc6f7117108d5bf76612b3dfb71b54e818fd155a44c4c91c9dab
MD5 f4745df5f6f01ac8826e8c53919d329b
BLAKE2b-256 18a9e8f3543f173305da36823ec1b5b9d07ef611e416e2a6a688a96efe5c869d

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 33a9439273affe5d4b9a9ee979bf2059ac51328065808d4feee4821976365d88
MD5 40a54af10182889b3c527fc24af1522f
BLAKE2b-256 82836b60ace744d948c33c4f78b9649762759dcbf02de88e06182d9e6afea813

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3c0396378e15bd606911ba38d101457841edaafe236ceda8e982a57411dc1eb7
MD5 51cbb888f657a0c4fd50279879ef7d47
BLAKE2b-256 9eb366dafbca8719fef790c7c5a531932f97934258f43886bff5dea05dcf02a0

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ccf3ba00cde30419a3bd883802d149566d2afefeb91e6e0a63d1d49f1c09227d
MD5 e5c2902585baac8f4fc316f65dd69db9
BLAKE2b-256 304c8ef6e4bcff583d3f9659d8a576aa215f72242dc39a9dec6ec8bc19fabe8b

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 66359f363f3a0d06993a1d1d8e280ff9ca18d1ba187b6c2265fcc3b44d05f424
MD5 e00653c849743b4bc87869412e1282a7
BLAKE2b-256 3e34712fef44b46d21ead8e4955b4d577756a0933e71db7d0f4e416b1aefd7b6

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 11fd3a87432bcc7600f2707af3458df05e6ead48c7b347a1f5f35dfde82676d2
MD5 a579447f680b63a5befca7e42e242312
BLAKE2b-256 3795c170e9e7b91b3f5a6c1b36663ebfdfe61fd3dabcb357d835a7482a4ef89d

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4fae2666197c1bdb1560c5671beb8efc9672f3d25e6775838e0f0e74e7f842b0
MD5 a0fdc25390f4bb4fb34eef4d9d7ee188
BLAKE2b-256 f28f6e6b4c55a1a97d78e5b493db6424c2c339869112b036fbbf3d113f476208

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rgapi-0.1.13-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rgapi-0.1.13-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f0d21099a9d46e00c0bce0262e4c8a4ef6468ec562ee08d76bd7fea157844f8d
MD5 4efda6bae410a7c644cb6e5c7bbb5b70
BLAKE2b-256 1e047c16b9f0be8fb5489335483b8fe7b598200c444f1f78fd0446ea927daea6

See more details on using hashes here.

Provenance

The following attestation bundles were made for rgapi-0.1.13-cp310-cp310-macosx_11_0_arm64.whl:

Publisher: ci.yml on AnswerDotAI/rgapi

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page