Skip to main content

Remap and renumber numpy arrays.

Project description

[![Build Status](https://travis-ci.org/seung-lab/fastremap.svg?branch=master)](https://travis-ci.org/seung-lab/fastremap) [![PyPI version](https://badge.fury.io/py/fastremap.svg)](https://badge.fury.io/py/fastremap)

# fastremap
Relabel integers in a numpy array based on dicts, arrays, and sequential renumbering from one. This module seems trivial, but it's necessary. Python loops are slow, so often numpy is used to perform remapping on large arrays (hundreds of megabytes or gigabytes). However, in order to efficiently remap an array in numpy you need a key-value array where the index is the key and the value is the contents of that index. If there are 32 or 64 bit labels in the array, this becomes impractical despite the triviality of the operation. It's conceivable to use numba for this, but having a cython library as an option can be convenient.

Available functions:
- renumber: Relabel array from 1 to N which can often use smaller datatypes.
- remap: Custom relabeling of values in an array from a dictionary.
- remap_from_array: Same as remap, but the map is an array where the key is the array index and the value is the value.
- remap_from_array_kv: Same as remap, but the map consists of two equal sized arrays, the first containing keys, the second containing values.
- asfortranarray: Perform an in-place matrix transposition for square and cubic arrays, standard numpy algorithm otherwise.
- ascontiguousarray: Perform an in-place matrix transposition for square and cubic arrays, standard numpy algorithm otherwise.

```python
import fastremap
import numpy as np

arr = np.array(..., dtype=np.int64) # array contains 500 unique labels

# Renumber labels from 1 to 501 and preserve 0
arr, remapping = fastremap.renumber(arr, preserve_zero=True) # Returns uint16 array (smallest possible)
arr, remapping = fastremap.renumber(arr, preserve_zero=False) # Returns uint16 array, contains [1,502]

mappings = {
1: 100,
2: 200,
-3: 7,
}

arr = np.array([5, 1, 2, -5, -3, 10, 6])
# Custom remapping of -3, 5, and 6 leaving the rest alone
arr = fastremap.remap(arr, mappings, preserve_missing_labels=True)
# result: [ 5, 100, 200, -5, 7, 10, 6 ]

arr = fastremap.asfortranarray(arr) # in-place if arr is up to 4D
arr = fastremap.ascontiguousarray(arr) # in-place if arr is up to 4D

try:
arr = fastremap.remap(arr, mappings, preserve_missing_labels=False)
except KeyError:
# When preserve_missing_labels is False, a KeyError is thrown if a
# value is encountered that isn't in mappings.
pass

arr = np.array([1,2,3,3,2])
kvs = np.array([0,10,20,30]) # key is array index, value is value
result = fastremap.remap_from_array(arr, kvs)
# result: [10, 20, 30, 30, 20]

arr = np.array([1,2,3,3,2])
keys = np.array([3,4,2,1]) # key is array index, value is value
vals = np.array([10,20,30,40]) # key is array index, value is value
result = fastremap.remap_from_array_kv(arr, keys, vals)
# result: [40, 30, 10, 10, 30]
```

## `pip` Installation

*If binaries are available for your system. Numpy 1.16.0+ required for these binaries (except for python 3.4 which uses 1.15.4 as 1.16.0 dropped binaries for python 3.4).*

```bash
pip install fastremap
```

*If not, a C++ compiler is required.*

```bash
pip install numpy
pip install fastremap --no-binary :all:
```

## Manual Installation

*A C++ compiler is required.*

```bash
sudo apt-get install g++ python3-dev
mkvirtualenv -p python3 fastremap
pip install numpy

# Choose one:
python setup.py develop
python setup.py install
```

## C++ Usage

You can also use the in-place transposition function in C++ directly.

```cpp
#include "ipt.hpp"

int main() {

int sx = 128;
int sy = 124;
int sz = 103;
int sw = 3;

auto* arr = ....;

// All primitive number types supported
// The array will be modified in place,
// so these functions are void type.
ipt::ipt<int>(arr, sx, sy); // 2D
ipt::ipt<float>(arr, sx, sy, sz); // 3D
ipt::ipt<double>(arr, sx, sy, sz, sw); // 4D

return 0;
}
```

--
Made with <3



Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fastremap-1.2.1.tar.gz (217.2 kB view details)

Uploaded Source

Built Distributions

fastremap-1.2.1-cp37-cp37m-manylinux1_x86_64.whl (1.3 MB view details)

Uploaded CPython 3.7m

fastremap-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (1.3 MB view details)

Uploaded CPython 3.6m

fastremap-1.2.1-cp35-cp35m-manylinux1_x86_64.whl (1.3 MB view details)

Uploaded CPython 3.5m

fastremap-1.2.1-cp27-cp27m-manylinux1_x86_64.whl (1.2 MB view details)

Uploaded CPython 2.7m

File details

Details for the file fastremap-1.2.1.tar.gz.

File metadata

  • Download URL: fastremap-1.2.1.tar.gz
  • Upload date:
  • Size: 217.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6

File hashes

Hashes for fastremap-1.2.1.tar.gz
Algorithm Hash digest
SHA256 b03f34c3edec23ea6819215456838ef3b73256769bc2ca488b0f855fbac9cb2d
MD5 4fa9afcf2b9e45412506c7f2285c1100
BLAKE2b-256 46be1e7871c64b9aa96264f4637cfae52651c3610d637272ad9e571817991853

See more details on using hashes here.

File details

Details for the file fastremap-1.2.1-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

  • Download URL: fastremap-1.2.1-cp37-cp37m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 1.3 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6

File hashes

Hashes for fastremap-1.2.1-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 3e11d180058056f312cb8efdd48601213e4d6ef032be14f87aedb7ca86a097bf
MD5 e43a78712aa656ff7eb4fa96315cfabd
BLAKE2b-256 9c95fab42eb5025a21c8f1a36c5370054e768f7e645f9c2c67a96d17d95a2331

See more details on using hashes here.

File details

Details for the file fastremap-1.2.1-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

  • Download URL: fastremap-1.2.1-cp36-cp36m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 1.3 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6

File hashes

Hashes for fastremap-1.2.1-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 882b654d88f2ecfd98983afc2c2d171342a62afd67fdbaba23de0be2fe9665b8
MD5 304bc3a597945c96036515181fcd02a6
BLAKE2b-256 a8e791a7d5debc6659c57bf11f2a5502a44d1d6ed34289b5149e54c14a05a5df

See more details on using hashes here.

File details

Details for the file fastremap-1.2.1-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

  • Download URL: fastremap-1.2.1-cp35-cp35m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 1.3 MB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6

File hashes

Hashes for fastremap-1.2.1-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 6a7410b4bfab1c16e2cdf3dcc97d5dc4059ab695dd7d09ad86ca1cc54f6f421d
MD5 625b983c6e3e37f595309e24bc190507
BLAKE2b-256 6ee8dcde5aeb8cb43282ae00f002270e00d105e5c52f9453cb9b93271daf11e2

See more details on using hashes here.

File details

Details for the file fastremap-1.2.1-cp27-cp27m-manylinux1_x86_64.whl.

File metadata

  • Download URL: fastremap-1.2.1-cp27-cp27m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 1.2 MB
  • Tags: CPython 2.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6

File hashes

Hashes for fastremap-1.2.1-cp27-cp27m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 321b3b522d512af7f748fa5a4db77b14517c5ebea88c184bbca82c30a3fc7ba1
MD5 eced9be6dcdb42155e37a6999947597c
BLAKE2b-256 518285172be6f2497b1d27b931c172469d967bc7bd2a00691f59a52a7531bfb2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page