fastpose

A fast Python backend for robust camera pose estimation inspired by PoseLib. This package was mostly vibe-coded using Claude Code. Fundamental/essential matrix, homography, calibrated and uncalibrated relative pose (including shared- and varying-focal variants), absolute pose (P3P/P4Pf), and monocular-depth-assisted relative pose — all built on one numba-compiled LO-RANSAC engine and benchmarked against PoseLib (C++). Every problem also has a CUDA backend that runs the whole estimate batched on the GPU.

See TESTS.md for the tests and for the synthetic benchmarks that compare against PoseLib. Real-data numbers are coming later.

Install

From the repository root:

pip install .

Dependencies are just numpy and numba.

Numba compiles kernels lazily on first use, which makes the very first call to each estimate_* function slow (a few seconds). To compile everything up front, run:

fastpose-warmup

Use fastpose-warmup --problem fundamental / homography / essential / absolute / absolute-focal / varying-focal / shared-focal / monodepth to warm up only one backend (monodepth covers all four monodepth variants). Note that the warmup may take several minutes.

Where the compiled kernels go

The cache lands in __pycache__ directories next to the installed package and is large — a full warmup writes hundreds of MB of .nbc/.nbi files. Set NUMBA_CACHE_DIR to put it somewhere you can manage or clear instead. If the install directory is not writable (system installs, containers), numba falls back to a user-wide cache directory on its own.

Entries are keyed on the host CPU, the numba version and each source file's timestamp and size, so an upgrade recompiles rather than reusing stale code — but pip does not remove the superseded files and they accumulate. To reclaim the space:

fastpose-clean-cache --dry-run   # report what would go
fastpose-clean-cache             # delete it

It searches all three locations numba might have used and removes only .nbi/.nbc files, leaving .pyc and sources alone. The kernels recompile on next use, or run fastpose-warmup to rebuild them up front.

Multiprocessing

Warm the cache before you fork. Numba's on-disk kernel cache is not safe against concurrent writers, and two worker processes JIT-compiling the same kernel at once can corrupt it. Always compile in the parent first:

from fastpose.estimators.warmup import warmup

warmup()                                # or run `fastpose-warmup` beforehand
with multiprocessing.Pool(n) as pool:   # fork children inherit the compiled
    ...                                 # kernels and never touch the cache

With the default fork start method the children inherit the compiled dispatchers in memory and never read or write the cache at all, so this is usually all you need. Under spawn, workers re-import and load from the cache, which is safe as long as the warmup covered everything they use. If you cannot warm up first, give each worker its own NUMBA_CACHE_DIR instead.

Usage

Every estimate_* function takes correspondence arrays plus RANSAC options (iterations, max_error, min_iterations for adaptive termination, lo_iterations for the local-optimization budget, seed, and num_threads for threading) and returns (model, info):

• model is a dict holding the estimated quantities — R/t for pose problems, F for the fundamental matrix, plus f/f1/f2 for the unknown-focal variants and scale/shift1/shift2 for the monodepth ones. On total failure (info['num_inliers'] == 0) model holds a generic identity/placeholder result instead of a fit.
• info is a dict with inliers (boolean mask), num_inliers, model_score (the scorer's truncated score), iterations (RANSAC samples drawn) and refinements (whether the post-RANSAC polish pass ran and was adopted).

import numpy as np
from fastpose.estimators import (estimate_fundamental, estimate_homography,
                                 estimate_relative_pose, estimate_absolute_pose)

# fundamental matrix (7-point): x1, x2 are pixel coordinates
model, info = estimate_fundamental(x1, x2, iterations=1000, max_error=2.0)
F = model['F']

# calibrated relative pose (5-point): x1, x2 are normalized (x - c) / f,
# or pass camera1/camera2 (3x3 K, or a poselib.Camera) to pass pixel points
# directly and have them unprojected for you
model, info = estimate_relative_pose(x1, x2, camera1=K1, camera2=K2,
                                      iterations=1000, max_error=2.0)
R, t = model['R'], model['t']

# absolute pose (P3P): x is normalized image points, X the 3D points
model, info = estimate_absolute_pose(x, X, iterations=1000, max_error=2.0 / focal)
R, t = model['R'], model['t']

# homography (4-point DLT): x1, x2 are pixel coordinates, x2 ~ H x1
model, info = estimate_homography(x1, x2, iterations=1000, max_error=3.0)
H = model['H']

API reference

All functions live under fastpose.estimators and are re-exported from fastpose. min pts is the minimal-sample size — calling with fewer correspondences raises ValueError.

The # params: comment at the top of each function in src/fastpose/estimators/ documents every argument and exact input units (several take normalized/calibrated points rather than pixels — see the docstrings if you're not passing a camera). Every relative/absolute-pose function also accepts a final_refinement_iterations argument: an LM step budget for a final robust-loss polish pass over the RANSAC inliers (defaults to 100 iterations; 0 disables it), independent of lo_iterations. The uncalibrated solvers assume principals at origina if they are not provided as optional params.

Every function also takes num_threads/batch_per_thread for the parallel CPU driver (Threading) and device/batch for the GPU one (GPU (CUDA)).

Threading

Every estimate_* function also takes num_threads and batch_per_thread. The default (num_threads=None, or 1) runs the single-threaded RANSAC driver. Anything higher switches to a batched parallel driver: hypotheses are drawn num_threads * batch_per_thread at a time and solved and scored across that many numba threads. batch_per_thread defaults to 32.

model, info = estimate_relative_pose(x1, x2, camera1=K1, camera2=K2,
                                     iterations=1000, max_error=2.0,
                                     num_threads=8)

Measured on 8 physical cores at a 1000-iteration budget: 1.7–2.8× end to end, the smaller figure at large point counts. The batched loop itself scales about 3.5–4×; local optimization and the final refinement stay single-threaded and cap the rest, and they cost more as the inlier count grows.

Two caveats. This buys latency on one call, not throughput — if you already run one process per core (see Multiprocessing), threading inside each call only oversubscribes, so leave it off. And the parallel result is close to but not bit-identical to the serial one: every hypothesis in a batch is scored against the incumbent as it stood when the batch started, so models the serial driver would bail out of early can get scored in full. A run is reproducible from (seed, num_threads * batch_per_thread) regardless of how many threads actually run, and adaptive termination overshoots by at most one batch.

GPU (CUDA)

Every estimate_* function takes device='cuda', which runs a batch-parallel LO-RANSAC on the GPU. Solving, scoring, local optimization and the final polish pass all run on device.

model, info = estimate_relative_pose(x1, x2, camera1=K1, camera2=K2,
                                     iterations=20000, max_error=2.0,
                                     device='cuda')

Needs a CUDA device and a working numba.cuda. fastpose.cuda.is_available() tells you whether there is one; fastpose.cuda.unavailable_reason() says why not.

Warm up first. A cold GPU estimate compiles the solver, the scorer and two LM kernels (local optimization and the final polish use different losses). That is seconds per problem, and it is easy to mistake for the steady-state cost:

fastpose-warmup --device cuda        # --device all also warms the CPU kernels

Low-occupancy warnings. Several launches are small by design (one block per local-optimization candidate, one thread per hypothesis when sampling and solving), so for the duration of an estimate the driver silences numba's NumbaPerformanceWarning about grids under 128 blocks. Nothing else is filtered.

When it helps

The GPU keeps thousands of hypotheses in flight, so it wins on many iterations and many matches, and loses on small problems where a round of kernel launches costs more than the whole CPU estimate. For estimate_relative_pose on an RTX A4000 Laptop, against the single-threaded CPU driver:

Reproduce with python -m benchmarks.estimators.essential cuda-scaling.

What differs from the CPU result

This is a batched LO-RANSAC, not the serial one made faster, so it does not reproduce the CPU result bit for bit:

• Different samples. A run is reproducible from (seed, batch) on a given device, but it is not the CPU driver's sample sequence.
• Local optimization sees one candidate per round, gated on the same criterion the CPU driver uses (a minimal model must improve the best minimal score or inlier count).
• Adaptive termination is evaluated per round, so it can overshoot by up to one batch.
• Mixed precision. Per-point scoring and LM arithmetic is float32; the minimal solvers, the accumulators and the returned model stay float64. So scores differ by ~1e-6 relative and an inlier count can differ by a point.

Accuracy is held to the CPU path rather than assumed equal to it: tests/test_cuda.py checks every kernel against its CPU counterpart and asserts model-selection and minimizer quality directly, not element-wise tolerances alone.

batch

batch (hypotheses per round, default 4096) is exposed for tuning, but the driver already picks the round schedule: full size when min_iterations >= iterations (the default), and ramping geometrically from 256 when adaptive termination is on, so an early stop wastes at most a small round.

Round size tracks the iteration budget, not the match count - 4096 beat 128 by ~20x equally at 2 000, 16 000 and 50 000 matches - so raise it only if you have measured a reason to. At iterations <= batch there is a single round and the GPU is mostly idle, which is the case to avoid.

motion_from_essential (also exported from fastpose.estimators) decomposes an externally estimated essential matrix into (R, t) candidates, for cases where you already have E/F from elsewhere.

Architecture

The engine (src/fastpose/estimators/ransac.py) is a problem-agnostic LO-RANSAC compiled with numba. A camera pose problem plugs in three components, each a thin class wrapping a numba kernel with a fixed signature:

• Solver (src/fastpose/solvers/) — minimal solver: solve(data, sample, models, workspace) -> num_models plus metadata (sample_size, num_params, max_models, workspace_size).
• Scorer (src/fastpose/scorers/) — truncated robust score: score(model, data, max_error_sq, best_score) -> (score, num_inliers).
• Refiner (src/fastpose/refiners/, optional) — non-minimal refit for local optimization: refine(data, model, refined, max_error_sq, num_iterations) -> success.

data is an opaque tuple of contiguous arrays chosen by the problem (for F: four coordinate columns, SIMD-friendly). The engine handles sampling, model selection, local optimization, final refinement and adaptive termination, and is specialized per problem by RansacEstimator(solver, scorer, refiner). Adding a new problem means writing the three kernels and their wrapper — the RANSAC loop is reused unchanged; the 5-point problem shares the Sampson scorer verbatim with the 7-point one and only adds a solver and a refiner.

The CUDA backend reuses that structure rather than duplicating it. Kernels that are portable between the two backends are written once against the jit(fastmath=, inline=) shim in src/fastpose/jit_backend.py and built twice — with njit for the CPU and cuda.jit(device=True) for the GPU. Every minimal solver is shared this way (the 5-point chain — nullspace, constraint expansion, Gauss-Jordan, Danilevsky, Sturm, essential decomposition — plus P3P, P4Pf, the 7- and 6-point chains and the four monodepth solvers), as are the per-point residuals, the cheirality test and the jacobians. A problem's GPU-specific code is one module under cuda/problems/: how its solve kernel allocates scratch, and how the shared per-point kernels are seen by a block.

Two things could not be shared, both because numba's runtime is host-only, so neither np.empty nor .reshape compiles in device code:

• Scratch is passed in pre-shaped. Each _solve_*_core takes every scratch array as an explicit argument; the CPU keeps the flat-workspace contract through a thin wrapper that does the slicing, and the CUDA kernel allocates the same pieces as cuda.local.array (which the hardware interleaves across threads, so the accesses coalesce for free).
• The scorer and the LM accumulate are reductions on the GPU, not serial loops, so they are written separately and checked against the CPU kernels point-for-point in the tests.

The shared per-point kernels take the numba type their float literals are cast to (build_sampson_point_kernels(jit, real=...)), which is what lets the GPU build them in float32 while the CPU keeps float64 — a bare 1.0 in numba is a float64 constant and would silently promote an otherwise-float32 expression back to double. kernel_cache.py knows how to hash a numba type so this extra closure cell does not disable the on-disk cache.

src/fastpose/refiners/lm.py contains a shared Levenberg-Marquardt loop (damping schedule, damped normal equations, accept/reject, convergence), compiled per problem via the same closure specialization. A refinement problem only defines four small kernels over a flat state vector: init_state, state_to_model, accumulate (residuals + jacobian -> normal equations) and apply_step (retraction).

Layout

src/fastpose/
    solvers/       minimal solvers: 7-point F, 4-point DLT homography, 5-point E,
                   7-point varying-focal and 6-point shared-focal relative pose,
                   P3P and P4Pf absolute pose, four 3-point monodepth relative
                   pose variants; shared helpers in solvers/utils.py
    scorers/       robust scorers: truncated Sampson (MSAC) for flat 3x3 models,
                   for pose models [R|t] (E = [t]_x R assembled on the fly), for
                   varying/shared-focal pose models [R|t|f1|f2] and for the
                   monodepth model layouts; truncated reprojection error for
                   absolute pose models [R|t] and unknown-focal models [R|t|f];
                   truncated symmetric transfer error for homographies
    refiners/      local optimization; refiners/lm.py is the generic LM engine,
                   refiners/utils.py the shared factorization/jacobian machinery,
                   refiners/{fundamental,homography,essential,varying_focal,
                   shared_focal,absolute,absolute_focal,monodepth}.py the
                   per-problem kernels
    kernel_cache.py  process-independent numba cache keys for the closure-
                   specialized kernels (what makes `fastpose-warmup` stick)
    clean_cache.py   `fastpose-clean-cache`: deletes the cached kernels from
                   whichever of numba's three locations they landed in
    jit_backend.py   the `jit(fastmath=, inline=)` decorator shim that lets one
                   kernel source compile for CPU (njit) or GPU (cuda.jit)
    cuda/          the CUDA backend, covering every problem above. cuda/ransac.py
                   is the batched driver and cuda/{scoring,lm,reductions}.py the
                   problem-agnostic kernels (one block per hypothesis for the
                   scorer, one per candidate for the whole in-kernel LM loop);
                   cuda/problem.py is the seam between them and the per-problem
                   device code in cuda/problems/, one module each
    estimators/    the RANSAC engine (estimators/ransac.py) and the full pipelines
                   listed in the API reference above; shared helpers in
                   estimators/utils.py
benchmarks/    benchmarks/estimators compares against poselib (mAA + runtime
               scaling); benchmarks/solvers evaluates solver accuracy on
               synthetic noise-free minimal samples; shared metrics, data
               generation and plotting in benchmarks/utils.py
tests/         finite-difference checks of the refiner jacobians (tests/jac)
               and end-to-end estimator tests on synthetic scenes

All four subpackages (solvers, scorers, refiners, estimators) are only reachable through fastpose (e.g. from fastpose.solvers import p3p) — import solvers / import estimators etc. directly is not supported.

Backend notes

Technical detail on each backend's solver/scorer/refiner, for anyone extending fastpose or curious how a specific number is computed.

fastmath

fastmath=True goes on the kernels where reassociation and FMA contraction measurably pay — the minimal solvers, the per-point scoring loops and the LM accumulate kernels — and stays off the O(1) work around them: the per-model and per-LM-iteration setup in refiners/utils.py, the state retractions and the RANSAC driver itself. Numba's fast-math flags are per-function and do not propagate into a separately compiled callee, so flagging a thin wrapper only affects its own handful of scalar operations. Kernels declared inline='always' need no flag either: numba inlines them at the Numba IR level, so they are lowered with their caller's setting.

Where it pays it pays well — the monodepth LM engine is ~40% slower without it, the 7-point solver ~11% — and where it does not, the flag is left off rather than applied for uniformity. Enabling it on the RANSAC driver, for instance, measures 0% and bit-identical results, because all of that loop's arithmetic lives in the solver, scorer and refiner it calls; it would only add nnan to the score comparisons, which every kernel here is written to avoid needing (1e300 is the failure sentinel throughout, never inf).

The one deliberate removal is the shared-focal solver, below.

Next steps

• Degeneracy handling (e.g. dominant-plane checks a la DEGENSAC).
• Benchmark the CUDA backend on a datacenter GPU. Every number on this page is from a laptop RTX A4000 at 1/64-rate float64, and only estimate_relative_pose has a cuda-scaling benchmark mode so far.

Citations

If you find this library useful please consider citing:

@misc{kocur2026fastpose,
  title  = {{fastpose}: a fast Python backend for robust camera pose estimation},
  author = {Kocur, Viktor},
  year   = {2026},
  url    = {https://github.com/kocurvik/fastpose}
}

To cite individual solvers see REFERENCES.md.