cos-comparison 0.3.2

Local similarity comparison for feature extraction – biologically inspired, zero‑training edge/pattern detection, multi-backend acceleration.

Cos Comparison

An AGI-oriented project based on local similarity comparison for feature extraction – biologically inspired, zero-training edge / pattern detection.

---

Core Idea

Information is produced by local comparison in raw data.
This module implements the centre-surround antagonism mechanism from neuroscience, extracting edges, textures, and keypoints using only sliding-window similarity.

The core formula (cosine-modulated similarity, recommended default):

$$ \text{cosmod} = \frac{2(A \cdot B)}{|A|^2 + |B|^2} $$

• A step toward biologically plausible AGI
• No training, no labels, no backpropagation
• Works on 1D, 2D, 3D, 4D data (audio, images, video, volumetric data)
• Supports passive (reflexive boundary detection) and active (template matching) modes
• Three high-performance backends with automatic fallback: Python C extension, ctypes pure C, pure Python
• Cross-platform support for Windows, Linux, and macOS

---

Core Principles

1. Information emerges from local comparison – edges, textures, and patterns arise by comparing neighboring regions.
2. Centre-surround antagonism – two sliding windows are compared with a fixed displacement vector d, mimicking the response of retinal ganglion cells.
3. Three complementary similarity measures – _cos (angular similarity), _mod (magnitude similarity), _cosmod (recommended balanced combination).
4. Dual operational modes – passive (boundary detection without templates) and active (template matching with a user-supplied kernel).
5. Multi-scale and multi-directionality – vary window size for scale selection, change d for orientation selectivity (vertical, horizontal, diagonal).
6. Dimension-agnostic – the same algorithm runs natively on 1D, 2D, 3D, 4D, and higher-dimensional data.
7. Zero training, zero labels – fully deterministic, ready to use out of the box.
8. Full determinism and interpretability – every output has a clear geometric meaning.
9. Modular, pluggable architecture – pure Python reference implementation with two optional high-performance C backends and automatic fallback.

---

Design Principles

The library adheres to two key design principles:

• Zero internal dependencies – the core is self-contained and does not rely on any third-party modules. It runs everywhere Python runs.
• Open integration – a generic hook/interface mechanism is provided, allowing users to seamlessly plug in any external module that conforms to the specified protocols (e.g., GPU/NPU accelerators, custom tensor types, or specialized hardware backends). This keeps the core lightweight while enabling unlimited extensibility.

---

🚀 What's New in Version 0.3.2

Version 0.3.2 brings full multi-backend support, critical bug fixes, and automatic C backend compilation.

Highlights

• Three Production Backends – fully supported with automatic fallback:
  • Python C Extension (cos_comparison_pydll) – maximum performance, tight Python integration, selective GIL release for multi-threaded workloads.
  • ctypes C Backend (cos_comparison_c) – pure C implementation with all callbacks executed natively in C, no Python C API dependency, cross-platform compatible.
  • Pure Python (cos_comparison) – zero-dependency reference implementation, guaranteed to work everywhere.
• Automatic C Compilation – setup.py now automatically compiles both C backends during installation, with graceful fallback to pure Python if compilation fails.
• Critical Bug Fixes – resolved multiple historical issues in the pure Python implementation:
  • Fixed vector_map_as_tensor indexing system (tuple subtraction error, incorrect multi-dimensional access)
  • Fixed _build_ones kernel generation logic (incorrect recursion producing wrong-shaped kernels)
  • Fixed parameter checking for default_contain infinite-dimension containers
  • Fixed create_void_list generator parameter handling
• Complete Callback System – all backends now support the full callback lifecycle: start_callback, end_callback, local_error_callback, global_error_callback, return_callback
• Numerical Consistency – all three backends produce bit-identical results (within floating-point precision) for all operations
• Full Local Statistics Support – mean_local and local_variance are now implemented natively in both C backends

Backend Performance Comparison

Backend	Relative Speed	Peak Memory	Status
Python C Extension	100–200×	~5–8 MB	✅ Stable
ctypes C Backend	50–100×	~8–12 MB	✅ Stable
Pure Python	1×	~22 MB	✅ Stable

Note: Memory measurements are based on a 424×322×3 test image with a 3×3 window. C backend memory is estimated from process working set measurements, as tracemalloc cannot track C-level allocations. All C backends automatically fall back to pure Python if compilation or loading fails,and are all added free-thread support.

⚠️ Breaking Changes

1. NumPy backend removed
Introducing a dedicated NumPy backend added unnecessary complexity and potential supply-chain risk. The core now natively supports NumPy arrays (and any type implementing __index__ and __len__, such as PyTorch tensors) via the duck-typing protocol, without requiring a separate backend.

2. Spelling errors corrected
All historical misspellings (e.g., genrate → generate) have been fixed throughout the codebase and documentation.

---

⚠️ Migration Guide from 0.2.x

Starting from version 0.2.0, the package has been restructured.
All core functions now reside in the core submodule.

Correct import statement:

from cos_comparison import core

Legacy code using import cos_comparison as cc will not work with 0.3.0. Please update your imports accordingly.

---

Installation

pip install cos-comparison

The installer will automatically attempt to compile both C backends during installation. If a C compiler is not available on your system, installation will complete successfully with the pure Python backend only.

To install with optional test dependencies:

pip install cos-comparison[test]

Manual Compilation

If you need to recompile the C backends after installation (e.g. after changing compiler settings):

# From the project root directory
python setup.py build_ext --inplace

This will build both the Python C extension and the ctypes shared library.

---

Quick Start

1D – detect discontinuities (passive mode)

from cos_comparison import core

data = [1.0, 2.0, 3.0, 4.0, 5.0]
result = core.cos_comparison_passive_1d(
    data,
    window_size=(2,),
    step=(1,),
    d=(1,)
)
print(result)

2D – edge detection (passive mode)

from cos_comparison import core

image = [
    [1, 1, 0, 0],
    [1, 1, 0, 0],
    [0, 0, 1, 1],
    [0, 0, 1, 1]
]
edges = core.cos_comparison_passive_2d(
    image,
    window_size=(2, 2),
    step=(1, 1),
    d=(1, 0)
)
print(edges)

Active mode – template matching

from cos_comparison import core

data = [1.0, 2.0, 3.0, 4.0, 5.0]
kernel = [1.0, 0.0]
result = core.cos_comparison_active_1d(
    data,
    kernel=kernel,
    step=(1,)
)
print(result)

Whole-tensor cosine similarity

from cos_comparison import core

a = [1, 2, 3]
b = [2, 3, 4]
sim = core.cos_1d(a, b)
print(sim)

Using the recommended _cosmod similarity measure

from cos_comparison import core

result = core.cos_comparison_passive_1d(
    data,
    window_size=(2,),
    step=(1,),
    d=(1,),
    algorithm=core._cosmod
)

---

Multi-Backend System

The package provides a sophisticated multi-backend manager that automatically selects the fastest available backend while maintaining a unified API.

Default Backend Priority

Priority	Backend	Implementation	Performance	Status
1	cos_comparison_pydll	Python C Extension	100–200×	✅ Stable
2	cos_comparison_c	ctypes Pure C Backend	50–100×	✅ Stable
3	cos_comparison	Pure Python	1×	✅ Stable

Key Features

• Automatic fallback – if a higher-priority backend is unavailable, the system automatically falls back to the next option.
• Runtime switching – manually switch backends at any time during execution.
• Unified API – all backends expose exactly the same interface.
• Configuration flexibility – control backend priority via a config file or environment variable.
• Zero-dependency guarantee – the pure Python backend always works, no compilation required.

Switching Backends

from cos_comparison import core

# Check current backend configuration
backends = core.get_available_backends()
print(backends)

# Force pure Python mode (useful for debugging)
core.set_mode("cos_comparison")

# Multiple backends (tried in specified order)
core.set_mode(["cos_comparison_pydll", "cos_comparison_c", "cos_comparison"])

# Enable ctypes backend only
core.set_mode(["cos_comparison_c", "cos_comparison"])

Environment Variable

# Unix
export COS_BACKEND=cos_comparison_pydll,cos_comparison_c,cos_comparison

# Windows
set COS_BACKEND=cos_comparison_pydll,cos_comparison_c,cos_comparison

---

API Overview

Passive mode (self-similarity / edge detection)

• cos_comparison_passive_1d / _2d / _3d / _4d + generic N-dimensional cos_comparison_passive

Active mode (template matching)

• cos_comparison_active_1d / _2d / _3d / _4d + generic N-dimensional cos_comparison_active

Whole-tensor similarity measures

• cos_1d / _2d / _3d / _4d (cosine similarity)
• mod_1d / _2d / _3d / _4d (magnitude similarity)
• cosmod_1d / _2d / _3d / _4d (cosine-modulated similarity, recommended)

Local statistics (sliding window)

• mean_local_1d / _2d / _3d / _4d
• local_variance_1d / _2d / _3d / _4d

Generic helpers

• cos(A, B, algorithm=core._cos) – works on arbitrarily nested lists and array-like objects.
• execute_many(func, arg_iter, kwarg_iter) – batch execution (returns a list).
• execute_many_iter(func, arg_iter, kwarg_iter) – batch execution (lazy generator).

---

Performance & Memory Benchmarks

Benchmark results for all three backends using a 322×424×3 RGB test image.
All tests were run with the same parameter grid: 3 window sizes, 3 offsets, and 3 similarity algorithms (27 combinations total).
Test environment: Windows 11 x64, 18-thread CPU, Python 3.14.6, JIT enabled, MSVC -O2 optimization.

Note: Memory figures for C backends are estimated because tracemalloc cannot track C-level allocations. Figures are derived from process-level working set measurements.

---

⏱️ Execution Time (27-run average)

Backend	3×3 window (s)	5×5 window (s)	7×7 window (s)	Total (27 runs)	Speedup vs Python
Pure Python	7.31	18.80	28.00	486.8 s	1×
ctypes C Backend	0.142	0.358	0.682	10.9 s	~45×
Python C Extension	0.088	0.213	0.407	6.35 s	~77×

• Algorithm choice (cos / mod / cosmod) has negligible impact (<5%) on execution time.
• Window size is the dominant factor – larger windows increase execution time proportionally.
• The C extension provides approximately 77× speedup over pure Python for typical workloads.

---

🧠 Memory Usage (peak working set)

Backend	Average peak memory	Notes
Pure Python	~21.5 MB	Stable across all parameter combinations
ctypes C Backend	~8–12 MB (estimated)	C-allocated memory, minimal Python overhead
Python C Extension	~5–8 MB (estimated)	Tightest integration, lowest memory usage

• Memory usage is largely independent of window size, offset, or algorithm choice.
• The pure Python backend uses a fixed ~21–22 MB for the standard test image.
• The C extension is extremely memory-efficient, allocating only the minimal required output tensor.

---

✅ Output Consistency

All three backends produce numerically identical output values (within floating-point precision) for every parameter combination – verifying that the core algorithm is correctly implemented across all backends.

---

📊 Full Raw Data

Complete CSV results (time and memory for all 27 combinations) are available in the tests/ directory:

• test_python.txt, test_pydll.txt, test_ctypes.txt – performance timing data
• memory_python.txt, memory_pydll.txt, memory_ctypes.txt – memory usage data
• out_*.txt – execution logs with saved heatmap output paths

---

🧪 How to Reproduce

# Performance test (all available backends)
python tests/test_performance.py --image tests/testdata/image_20260408.png --repeat 3 --output results.csv

# Memory test
python tests/test_memory.py --image tests/testdata/image_20260408.png --output memory.csv

Benchmarks run on a standard x86-64 CPU. Your results may vary depending on hardware, compiler optimizations, and operating system.

---

Author

I was born on May 31, 2008, and I feel fortunate to grow up in an era of rapid progress in artificial intelligence.

I have run extensive tests and observed many surprising emergent properties in the outputs. Earlier versions of this project contained numerous issues, as examination-oriented education left me limited time for thorough testing. I now have the opportunity to properly test, refine, and polish this work.

There remains a long road ahead to achieve true general artificial intelligence. I may be forced to set aside this research due to personal circumstances, but I do not want these ideas to fade away unnoticed. The purpose of open-sourcing this project is to share my thoughts, in the hope that others may build upon them and continue this line of inquiry.

---

Next Steps

1. Introduce a formal extension interface in the core, allowing users to plug in GPU, NPU, or other custom accelerators.
2. Complete the remaining cognitive submodules, working toward a simple but functional embodied AI agent.

---

License

MIT © 2026 Li Jinxin. See LICENSE for details.