Fast numerical expression evaluator for NumPy

## Project description

Author: David M. Cooke, Francesc Alted, and others. Robert A. McLeod robbmcleod@gmail.com https://github.com/pydata/numexpr http://numexpr.readthedocs.io/en/latest/

## What is NumExpr?

NumExpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like `'3*a+4*b'`) are accelerated and use less memory than doing the same calculation in Python.

In addition, its multi-threaded capabilities can make use of all your cores – which generally results in substantial performance scaling compared to NumPy.

Last but not least, numexpr can make use of Intel’s VML (Vector Math Library, normally integrated in its Math Kernel Library, or MKL). This allows further acceleration of transcendent expressions.

## How NumExpr achieves high performance

The main reason why NumExpr achieves better performance than NumPy is that it avoids allocating memory for intermediate results. This results in better cache utilization and reduces memory access in general. Due to this, NumExpr works best with large arrays.

NumExpr parses expressions into its own op-codes that are then used by an integrated computing virtual machine. The array operands are split into small chunks that easily fit in the cache of the CPU and passed to the virtual machine. The virtual machine then applies the operations on each chunk. It’s worth noting that all temporaries and constants in the expression are also chunked. Chunks are distributed among the available cores of the CPU, resulting in highly parallelized code execution.

The result is that NumExpr can get the most of your machine computing capabilities for array-wise computations. Common speed-ups with regard to NumPy are usually between 0.95x (for very simple expressions like `'a + 1'`) and 4x (for relatively complex ones like ```'a*b-4.1*a > 2.5*b'```), although much higher speed-ups can be achieved for some functions and complex math operations (up to 15x in some cases).

NumExpr performs best on matrices that are too large to fit in L1 CPU cache. In order to get a better idea on the different speed-ups that can be achieved on your platform, run the provided benchmarks.

## Usage

```>>> import numpy as np
>>> import numexpr as ne

>>> a = np.arange(1e6)   # Choose large arrays for better speedups
>>> b = np.arange(1e6)

>>> ne.evaluate("a + 1")   # a simple expression
array([  1.00000000e+00,   2.00000000e+00,   3.00000000e+00, ...,
9.99998000e+05,   9.99999000e+05,   1.00000000e+06])

>>> ne.evaluate('a*b-4.1*a > 2.5*b')   # a more complex one
array([False, False, False, ...,  True,  True,  True], dtype=bool)

>>> ne.evaluate("sin(a) + arcsinh(a/b)")   # you can also use functions
array([        NaN,  1.72284457,  1.79067101, ...,  1.09567006,
0.17523598, -0.09597844])

>>> s = np.array([b'abba', b'abbb', b'abbcdef'])
>>> ne.evaluate("b'abba' == s")   # string arrays are supported too
array([ True, False, False], dtype=bool)
```

## Documentation

Please see the official documentation at numexpr.readthedocs.io. Included is a user guide, benchmark results, and the reference API.

## Release history Release notifications | RSS feed

Files for numexpr, version 2.7.2
Filename, size File type Python version Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27m-macosx_10_9_x86_64.whl (101.1 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27m-manylinux1_i686.whl (431.0 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27m-manylinux1_x86_64.whl (473.1 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27m-manylinux2010_i686.whl (431.0 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27m-manylinux2010_x86_64.whl (473.1 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27mu-manylinux1_i686.whl (431.0 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27mu-manylinux1_x86_64.whl (473.1 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27mu-manylinux2010_i686.whl (431.0 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27mu-manylinux2010_x86_64.whl (473.1 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27m-win32.whl (113.4 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp27-cp27m-win_amd64.whl (106.1 kB) File type Wheel Python version cp27 Upload date Hashes
Filename, size numexpr-2.7.2-cp35-cp35m-macosx_10_9_x86_64.whl (101.1 kB) File type Wheel Python version cp35 Upload date Hashes
Filename, size numexpr-2.7.2-cp35-cp35m-manylinux1_i686.whl (440.1 kB) File type Wheel Python version cp35 Upload date Hashes
Filename, size numexpr-2.7.2-cp35-cp35m-manylinux1_x86_64.whl (468.3 kB) File type Wheel Python version cp35 Upload date Hashes
Filename, size numexpr-2.7.2-cp35-cp35m-manylinux2010_i686.whl (440.1 kB) File type Wheel Python version cp35 Upload date Hashes
Filename, size numexpr-2.7.2-cp35-cp35m-manylinux2010_x86_64.whl (468.3 kB) File type Wheel Python version cp35 Upload date Hashes
Filename, size numexpr-2.7.2-cp35-cp35m-win32.whl (89.8 kB) File type Wheel Python version cp35 Upload date Hashes
Filename, size numexpr-2.7.2-cp35-cp35m-win_amd64.whl (92.3 kB) File type Wheel Python version cp35 Upload date Hashes
Filename, size numexpr-2.7.2-cp36-cp36m-macosx_10_9_x86_64.whl (101.5 kB) File type Wheel Python version cp36 Upload date Hashes
Filename, size numexpr-2.7.2-cp36-cp36m-manylinux1_i686.whl (441.3 kB) File type Wheel Python version cp36 Upload date Hashes
Filename, size numexpr-2.7.2-cp36-cp36m-manylinux1_x86_64.whl (469.5 kB) File type Wheel Python version cp36 Upload date Hashes
Filename, size numexpr-2.7.2-cp36-cp36m-manylinux2010_i686.whl (441.3 kB) File type Wheel Python version cp36 Upload date Hashes
Filename, size numexpr-2.7.2-cp36-cp36m-manylinux2010_x86_64.whl (469.5 kB) File type Wheel Python version cp36 Upload date Hashes
Filename, size numexpr-2.7.2-cp36-cp36m-win32.whl (90.3 kB) File type Wheel Python version cp36 Upload date Hashes
Filename, size numexpr-2.7.2-cp36-cp36m-win_amd64.whl (92.8 kB) File type Wheel Python version cp36 Upload date Hashes
Filename, size numexpr-2.7.2-cp37-cp37m-macosx_10_9_x86_64.whl (101.5 kB) File type Wheel Python version cp37 Upload date Hashes
Filename, size numexpr-2.7.2-cp37-cp37m-manylinux1_i686.whl (443.6 kB) File type Wheel Python version cp37 Upload date Hashes
Filename, size numexpr-2.7.2-cp37-cp37m-manylinux1_x86_64.whl (471.3 kB) File type Wheel Python version cp37 Upload date Hashes
Filename, size numexpr-2.7.2-cp37-cp37m-manylinux2010_i686.whl (443.6 kB) File type Wheel Python version cp37 Upload date Hashes
Filename, size numexpr-2.7.2-cp37-cp37m-manylinux2010_x86_64.whl (471.3 kB) File type Wheel Python version cp37 Upload date Hashes
Filename, size numexpr-2.7.2-cp37-cp37m-win32.whl (90.3 kB) File type Wheel Python version cp37 Upload date Hashes
Filename, size numexpr-2.7.2-cp37-cp37m-win_amd64.whl (92.8 kB) File type Wheel Python version cp37 Upload date Hashes
Filename, size numexpr-2.7.2-cp38-cp38-macosx_10_9_x86_64.whl (98.5 kB) File type Wheel Python version cp38 Upload date Hashes
Filename, size numexpr-2.7.2-cp38-cp38-manylinux1_i686.whl (444.2 kB) File type Wheel Python version cp38 Upload date Hashes
Filename, size numexpr-2.7.2-cp38-cp38-manylinux1_x86_64.whl (472.9 kB) File type Wheel Python version cp38 Upload date Hashes
Filename, size numexpr-2.7.2-cp38-cp38-manylinux2010_i686.whl (444.2 kB) File type Wheel Python version cp38 Upload date Hashes
Filename, size numexpr-2.7.2-cp38-cp38-manylinux2010_x86_64.whl (472.9 kB) File type Wheel Python version cp38 Upload date Hashes
Filename, size numexpr-2.7.2-cp38-cp38-win32.whl (90.5 kB) File type Wheel Python version cp38 Upload date Hashes
Filename, size numexpr-2.7.2-cp38-cp38-win_amd64.whl (92.9 kB) File type Wheel Python version cp38 Upload date Hashes
Filename, size numexpr-2.7.2-cp39-cp39-macosx_10_9_x86_64.whl (98.5 kB) File type Wheel Python version cp39 Upload date Hashes
Filename, size numexpr-2.7.2-cp39-cp39-manylinux1_i686.whl (442.0 kB) File type Wheel Python version cp39 Upload date Hashes
Filename, size numexpr-2.7.2-cp39-cp39-manylinux1_x86_64.whl (471.0 kB) File type Wheel Python version cp39 Upload date Hashes
Filename, size numexpr-2.7.2-cp39-cp39-manylinux2010_i686.whl (442.0 kB) File type Wheel Python version cp39 Upload date Hashes
Filename, size numexpr-2.7.2-cp39-cp39-manylinux2010_x86_64.whl (471.0 kB) File type Wheel Python version cp39 Upload date Hashes
Filename, size numexpr-2.7.2-cp39-cp39-win32.whl (90.4 kB) File type Wheel Python version cp39 Upload date Hashes
Filename, size numexpr-2.7.2-cp39-cp39-win_amd64.whl (92.8 kB) File type Wheel Python version cp39 Upload date Hashes
Filename, size numexpr-2.7.2.tar.gz (97.8 kB) File type Source Python version None Upload date Hashes