Skip to main content

fortran-nlp-kernels-cuda

Python Fortran CUDA Fortran Platform License Version

CUDA Fortran and cuBLAS companion distribution for fortran-nlp-kernels. This package installs the optional native fortran_nlp._cuda extension. The matching base package provides the Python API, CPU extension, lexical and vector-search modules, backend dispatch, preprocessing, normalization, calibration, and runtime utilities.

Release compatibility

fortran-nlp-kernels-cuda is versioned independently from the base package. The current compatible release pair is:

fortran-nlp-kernels==0.1.1
fortran-nlp-kernels-cuda==0.1.2

CUDA companion 0.1.2 is the corrected external-runtime binary release for base package 0.1.1. It replaces the yanked CUDA companion 0.1.1 wheel. The companion declares this dependency:

fortran-nlp-kernels==0.1.1

Installing the CUDA companion therefore installs the matching base package automatically.

Installation from PyPI

Create or activate a CPython 3.12 environment:

python3.12 -m venv env312
source env312/bin/activate
python -m pip install --upgrade pip

Install the CUDA companion:

python -m pip install fortran-nlp-kernels-cuda

For an explicitly reproducible installation:

python -m pip install \
fortran-nlp-kernels==0.1.1 \
fortran-nlp-kernels-cuda==0.1.2

The installation provides:

fortran_nlp/
├── _cpu.cpython-312-...so
├── _cuda.cpython-312-...so
├── lexical/
├── vector_search/
├── auto.py
├── runtime.py
└── calibration.py

External NVIDIA runtime requirements

The CUDA companion uses an external-runtime packaging model. The wheel does not bundle CUDA, cuBLAS, or NVIDIA HPC SDK runtime libraries. These libraries must be installed separately and discoverable by the Linux dynamic loader. The validated 0.1.2 environment is:

  • Linux x86-64 with glibc 2.34 or newer
  • CPython 3.12
  • NVIDIA GPU with compute capability 7.5
  • compatible NVIDIA driver
  • NVIDIA HPC SDK 26.9
  • CUDA Runtime 13.3
  • cuBLAS 13
  • The release wheel was built and validated on an NVIDIA GeForce MX450 with compute capability 7.5.
  • Install NVIDIA HPC SDK 26.9 on Ubuntu
  • Add the NVIDIA HPC SDK repository:
curl https://developer.download.nvidia.com/hpc-sdk/ubuntu/DEB-GPG-KEY-NVIDIA-HPC-SDK \
| sudo gpg --dearmor \
-o /usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/nvidia-hpcsdk-archive-keyring.gpg] https://developer.download.nvidia.com/hpc-sdk/ubuntu/amd64 /" \
| sudo tee /etc/apt/sources.list.d/nvhpc.list

Install NVIDIA HPC SDK 26.9:

sudo apt-get update
sudo apt-get install -y nvhpc-26-9

Verify the installation:

/opt/nvidia/hpc_sdk/Linux_x86_64/26.9/compilers/bin/nvfortran \
--version

Configure NVIDIA runtime paths Set the NVIDIA HPC SDK root:

export NVHPC_ROOT=/opt/nvidia/hpc_sdk/Linux_x86_64/26.9

Add the compiler tools to PATH:

export PATH="$NVHPC_ROOT/compilers/bin:$PATH"

Add the required CUDA, cuBLAS, and NVIDIA HPC SDK runtime directories:

export LD_LIBRARY_PATH="$NVHPC_ROOT/math_libs/13.3/targets/x86_64-linux/lib:$NVHPC_ROOT/cuda/13.3/targets/x86_64-linux/lib:$NVHPC_ROOT/compilers/lib:${LD_LIBRARY_PATH:-}"

To apply this configuration automatically in future shells, add the same lines to ~/.bashrc:

cat >> ~/.bashrc <<'EOF'
export NVHPC_ROOT=/opt/nvidia/hpc_sdk/Linux_x86_64/26.9
export PATH="$NVHPC_ROOT/compilers/bin:$PATH"
export LD_LIBRARY_PATH="$NVHPC_ROOT/math_libs/13.3/targets/x86_64-linux/lib:$NVHPC_ROOT/cuda/13.3/targets/x86_64-linux/lib:$NVHPC_ROOT/compilers/lib:${LD_LIBRARY_PATH:-}"
EOF
source ~/.bashrc

Installation verification

Check that the NVIDIA driver sees the GPU:

nvidia-smi

Display the GPU name, compute capability, driver version, and memory:

nvidia-smi \
--query-gpu=name,compute_cap,driver_version,memory.total \
--format=csv

Verify both native extensions:

python - <<'PY'
from fortran_nlp import _cpu, _cuda
print("_cpu:", _cpu.__file__)
print("_cuda:", _cuda.__file__)
print("CPU and CUDA extension imports: PASS")
PY

The low-level CUDA extension is imported from:

from fortran_nlp import _cuda

It is not imported from fortran_nlp.vector_search.

Inspect dynamic-library resolution Locate the installed CUDA extension:

cuda_extension="$(
python -c 'from fortran_nlp import _cuda; print(_cuda.__file__)'
)"

Inspect its shared-library dependencies:

ldd "$cuda_extension"

Check specifically for unresolved libraries:

ldd "$cuda_extension" | grep "not found"

Expected result:

no output

If libcublas.so.13, libcublasLt.so.13, libcudart.so.13, libnvomp.so, or libnvcpumath.so is not found, check that LD_LIBRARY_PATH contains all three required directories:

printf '%s
' "$LD_LIBRARY_PATH"

Available backends

Lexical search

cuda_fortran -> handwritten CUDA Fortran Jaccard scoring

Vector search

cuda_fortran -> handwritten CUDA Fortran IP and squared-L2 scoring
cuda_cublas -> cuBLAS SGEMV and SGEMM scoring

Both vector backends reuse the existing deterministic CPU top-k implementation. CUDA backends are explicit. They are not selected by backend="auto".

Lexical Jaccard usage

from fortran_nlp.lexical import JaccardIndex
documents = [
"reset the account password",
"update contact information",
"create a new insurance application",
]
index = JaccardIndex(
documents,
backend="cuda_fortran",
)
results = index.search(
"how to reset the account password",
k=2,
minimum_score=0.01,
)
for result in results:
print(result)

Batched CUDA lexical search uses the same index:

batch_results = index.search_batch(
[
"how to reset the account password",
"contact information update",
],
k=2,
minimum_score=0.01,
)

Vector-search usage

The public database layout is:

(vector_count, dimension)

The public batched-query layout is:

(query_count, dimension)

Inputs are converted to float32. Inner product with CUDA Fortran

import numpy as np
from fortran_nlp.vector_search import FlatIndexIP
database = np.array(
[
[1.0, 2.0, 3.0],
[4.0, 5.0, 6.0],
[-1.0, 2.0, 1.0],
],
dtype=np.float32,
)
query = np.array(
[1.0, 2.0, 3.0],
dtype=np.float32,
)
index = FlatIndexIP(
database,
backend="cuda_fortran",
)
scores, indices = index.search(query, k=2)
print(scores)
print(indices)

Squared-L2 with CUDA Fortran

from fortran_nlp.vector_search import FlatIndexL2
index = FlatIndexL2(
database,
backend="cuda_fortran",
)
distances, indices = index.search(query, k=2)
print(distances)
print(indices)

FlatIndexL2 returns squared Euclidean distances. Inner product with cuBLAS

index = FlatIndexIP(
database,
backend="cuda_cublas",
)
scores, indices = index.search(query, k=2)

Cosine similarity with cuBLAS

from fortran_nlp.vector_search import FlatIndexCosine
index = FlatIndexCosine(
database,
backend="cuda_cublas",
)
scores, indices = index.search(query, k=2)

FlatIndexCosine normalizes database vectors and queries before inner-product search. Zero vectors are invalid because cosine normalization is undefined for a zero norm.

Batched vector search

queries = np.array(
[
[1.0, 2.0, 3.0],
[0.0, 1.0, 0.0],
],
dtype=np.float32,
)
scores, indices = index.search_batch(
queries,
k=2,
)
print(scores.shape)
print(indices.shape)

The returned shapes are:

(query_count, k)

CUDA vector-search architecture

CUDA vector search in the current release follows this execution model:

GPU scoring
-> copy the complete score array or matrix to the host
-> deterministic CPU top-k
-> return values and zero-based indices

The handwritten backend uses one CUDA thread per database vector for single-query scoring and one CUDA thread per query-database pair for batched scoring. The cuBLAS backend uses:

SGEMV -> single-query inner product
SGEMM -> batched inner product
GPU squared-norm kernels -> L2 norm expansion
GPU L2-combination kernel -> squared-L2 scores
CPU top-k -> deterministic result selection

GPU top-k, GPU-resident indexes, CPU/GPU automatic dispatch, multi-GPU execution, and mixed-precision scoring are not included in this release. Building from source A source build requires:

  • Linux x86-64
  • CPython 3.12
  • NVIDIA HPC SDK 26.9
  • CUDA 13.3
  • compatible NVIDIA driver and GPU
  • C++17 compiler
  • Meson and meson-python
  • nanobind 3.1 or newer
  • NumPy 2.0 or newer

Clone the repository:

git clone https://github.com/suwesh/fortran-nlp-kernels.git
cd fortran-nlp-kernels

Build an editable installation with CUDA enabled:

python -m pip install -e . \
tings=setup-args="-Dfnlp_cuda=true" \
--config-settings=setup-args="-Dfnlp_cuda_arch=cc75" \
--config-settings=setup-args="-Dfnlp_cuda_toolkit=cuda13.3" \
--config-settings=setup-args="-Dfnlp_nvhpc_root=/opt/nvidia/hpc_sdk/Linux_x86_64/26.9"

The fnlp_cuda_arch value must match the intended GPU target. The published 0.1.2 wheel targets cc75. Verify the editable build:

python - <<'PY'
from fortran_nlp import _cpu, _cuda
print("_cpu:", _cpu.__file__)
print("_cuda:", _cuda.__file__)
PY

Troubleshooting _cuda is unavailable

Error:

backend='cuda_fortran' requires the optional fortran_nlp._cuda extension

Confirm that the companion is installed:

python -m pip show fortran-nlp-kernels-cuda

Then test the direct import:

python -c "from fortran_nlp import _cuda; print(_cuda.__file__)"

libcublas.so.13 is not found Ensure the cuBLAS directory is on LD_LIBRARY_PATH:

export NVHPC_ROOT=/opt/nvidia/hpc_sdk/Linux_x86_64/26.9
export LD_LIBRARY_PATH="$NVHPC_ROOT/math_libs/13.3/targets/x86_64-linux/lib:$NVHPC_ROOT/cuda/13.3/targets/x86_64-linux/lib:$NVHPC_ROOT/compilers/lib:${LD_LIBRARY_PATH:-}"

libcudart.so.13 is not found

Confirm that the CUDA runtime exists:

find "$NVHPC_ROOT/cuda/13.3" \
-name 'libcudart.so.13' \
-print

Ensure this directory is configured:

$NVHPC_ROOT/cuda/13.3/targets/x86_64-linux/lib

NVIDIA HPC SDK runtime is not found

Check these libraries:

find "$NVHPC_ROOT/compilers/lib" \
( -name 'libnvomp.so' \
-o -name 'libnvcpumath.so' ) \
-print

Ensure this directory is configured:

$NVHPC_ROOT/compilers/lib

Current limitations

  • Linux x86-64 only
  • CPython 3.12 only
  • glibc 2.34 or newer
  • NVIDIA compute capability 7.5 target
  • external NVIDIA runtime installation required
  • CUDA backends must be selected explicitly
  • backend="auto" remains CPU-only
  • deterministic top-k executes on the CPU
  • complete score arrays or matrices are copied back to the host
  • no GPU-resident database
  • no GPU top-k
  • no multi-GPU execution
  • no mixed-precision scoring

License

This project is licensed under GPL-3.0-only. See LICENSE.

Author

Suwesh Prasad Sah

Metadata

Release files for fortran-nlp-kernels-cuda 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for fortran-nlp-kernels-cuda 0.1.2
File Interpreter ABI Platform Actions
fortran_nlp_kernels_cuda-0.1.2-cp312-cp312-manylinux_2_34_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.34+ x86-64 Details for fortran_nlp_kernels_cuda-0.1.2-cp312-cp312-manylinux_2_34_x86_64.whl

Release files / fortran_nlp_kernels_cuda-0.1.2-cp312-cp312-manylinux_2_34_x86_64.whl

Download URL fortran_nlp_kernels_cuda-0.1.2-cp312-cp312-manylinux_2_34_x86_64.whl
Size 324.4 kB
Tags CPython 3.12 Linux glibc 2.34+ x86-64
SHA-256 checksum
How to use checksums
c196c361f422bc7c381520b587c285bd25cdbaf0fcb0d0e15bfc6b55ae5a0b0d
BLAKE2b-256 checksum
How to use checksums
df9d7b38cf46327ce23abcacc4dceb13fa8af2d8fc66ea2825bad24a0f5cbbbb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

Security for fortran-nlp-kernels-cuda 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

1 file in this release has been uploaded with provenance metadata.

Attestation for fortran_nlp_kernels_cuda-0.1.2-cp312-cp312-manylinux_2_34_x86_64.whl

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 4, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page