Remap and renumber numpy arrays.
Project description
[![Build Status](https://travis-ci.org/seung-lab/fastremap.svg?branch=master)](https://travis-ci.org/seung-lab/fastremap) [![PyPI version](https://badge.fury.io/py/fastremap.svg)](https://badge.fury.io/py/fastremap)
# fastremap
Relabel integers in a numpy array based on dicts, arrays, and sequential renumbering from one. This module seems trivial, but it's necessary. Python loops are slow, so often numpy is used to perform remapping on large arrays (hundreds of megabytes or gigabytes). However, in order to efficiently remap an array in numpy you need a key-value array where the index is the key and the value is the contents of that index. If there are 32 or 64 bit labels in the array, this becomes impractical despite the triviality of the operation. It's conceivable to use numba for this, but having a cython library as an option can be convenient.
Available functions:
- renumber: Relabel array from 1 to N which can often use smaller datatypes.
- remap: Custom relabeling of values in an array from a dictionary.
- remap_from_array: Same as remap, but the map is an array where the key is the array index and the value is the value.
- remap_from_array_kv: Same as remap, but the map consists of two equal sized arrays, the first containing keys, the second containing values.
- asfortranarray: Perform an in-place matrix transposition for square and cubic arrays, standard numpy algorithm otherwise.
- ascontiguousarray: Perform an in-place matrix transposition for square and cubic arrays, standard numpy algorithm otherwise.
```python
import fastremap
import numpy as np
arr = np.array(..., dtype=np.int64) # array contains 500 unique labels
# Renumber labels from 1 to 501 and preserve 0
arr, remapping = fastremap.renumber(arr, preserve_zero=True) # Returns uint16 array (smallest possible)
arr, remapping = fastremap.renumber(arr, preserve_zero=False) # Returns uint16 array, contains [1,502]
mappings = {
1: 100,
2: 200,
-3: 7,
}
arr = np.array([5, 1, 2, -5, -3, 10, 6])
# Custom remapping of -3, 5, and 6 leaving the rest alone
arr = fastremap.remap(arr, mappings, preserve_missing_labels=True)
# result: [ 5, 100, 200, -5, 7, 10, 6 ]
arr = fastremap.asfortranarray(arr) # in-place if arr is up to 4D
arr = fastremap.ascontiguousarray(arr) # in-place if arr is up to 4D
try:
arr = fastremap.remap(arr, mappings, preserve_missing_labels=False)
except KeyError:
# When preserve_missing_labels is False, a KeyError is thrown if a
# value is encountered that isn't in mappings.
pass
arr = np.array([1,2,3,3,2])
kvs = np.array([0,10,20,30]) # key is array index, value is value
result = fastremap.remap_from_array(arr, kvs)
# result: [10, 20, 30, 30, 20]
arr = np.array([1,2,3,3,2])
keys = np.array([3,4,2,1]) # key is array index, value is value
vals = np.array([10,20,30,40]) # key is array index, value is value
result = fastremap.remap_from_array_kv(arr, keys, vals)
# result: [40, 30, 10, 10, 30]
```
## `pip` Installation
*If binaries are available for your system. Numpy 1.16.0+ required for these binaries (except for python 3.4 which uses 1.15.4 as 1.16.0 dropped binaries for python 3.4).*
```bash
pip install fastremap
```
*If not, a C++ compiler is required.*
```bash
pip install numpy
pip install fastremap --no-binary :all:
```
## Manual Installation
*A C++ compiler is required.*
```bash
sudo apt-get install g++ python3-dev
mkvirtualenv -p python3 fastremap
pip install numpy
# Choose one:
python setup.py develop
python setup.py install
```
## C++ Usage
You can also use the in-place transposition function in C++ directly.
```cpp
#include "ipt.hpp"
int main() {
int sx = 128;
int sy = 124;
int sz = 103;
int sw = 3;
auto* arr = ....;
// All primitive number types supported
// The array will be modified in place,
// so these functions are void type.
ipt::ipt<int>(arr, sx, sy); // 2D
ipt::ipt<float>(arr, sx, sy, sz); // 3D
ipt::ipt<double>(arr, sx, sy, sz, sw); // 4D
return 0;
}
```
--
Made with <3
# fastremap
Relabel integers in a numpy array based on dicts, arrays, and sequential renumbering from one. This module seems trivial, but it's necessary. Python loops are slow, so often numpy is used to perform remapping on large arrays (hundreds of megabytes or gigabytes). However, in order to efficiently remap an array in numpy you need a key-value array where the index is the key and the value is the contents of that index. If there are 32 or 64 bit labels in the array, this becomes impractical despite the triviality of the operation. It's conceivable to use numba for this, but having a cython library as an option can be convenient.
Available functions:
- renumber: Relabel array from 1 to N which can often use smaller datatypes.
- remap: Custom relabeling of values in an array from a dictionary.
- remap_from_array: Same as remap, but the map is an array where the key is the array index and the value is the value.
- remap_from_array_kv: Same as remap, but the map consists of two equal sized arrays, the first containing keys, the second containing values.
- asfortranarray: Perform an in-place matrix transposition for square and cubic arrays, standard numpy algorithm otherwise.
- ascontiguousarray: Perform an in-place matrix transposition for square and cubic arrays, standard numpy algorithm otherwise.
```python
import fastremap
import numpy as np
arr = np.array(..., dtype=np.int64) # array contains 500 unique labels
# Renumber labels from 1 to 501 and preserve 0
arr, remapping = fastremap.renumber(arr, preserve_zero=True) # Returns uint16 array (smallest possible)
arr, remapping = fastremap.renumber(arr, preserve_zero=False) # Returns uint16 array, contains [1,502]
mappings = {
1: 100,
2: 200,
-3: 7,
}
arr = np.array([5, 1, 2, -5, -3, 10, 6])
# Custom remapping of -3, 5, and 6 leaving the rest alone
arr = fastremap.remap(arr, mappings, preserve_missing_labels=True)
# result: [ 5, 100, 200, -5, 7, 10, 6 ]
arr = fastremap.asfortranarray(arr) # in-place if arr is up to 4D
arr = fastremap.ascontiguousarray(arr) # in-place if arr is up to 4D
try:
arr = fastremap.remap(arr, mappings, preserve_missing_labels=False)
except KeyError:
# When preserve_missing_labels is False, a KeyError is thrown if a
# value is encountered that isn't in mappings.
pass
arr = np.array([1,2,3,3,2])
kvs = np.array([0,10,20,30]) # key is array index, value is value
result = fastremap.remap_from_array(arr, kvs)
# result: [10, 20, 30, 30, 20]
arr = np.array([1,2,3,3,2])
keys = np.array([3,4,2,1]) # key is array index, value is value
vals = np.array([10,20,30,40]) # key is array index, value is value
result = fastremap.remap_from_array_kv(arr, keys, vals)
# result: [40, 30, 10, 10, 30]
```
## `pip` Installation
*If binaries are available for your system. Numpy 1.16.0+ required for these binaries (except for python 3.4 which uses 1.15.4 as 1.16.0 dropped binaries for python 3.4).*
```bash
pip install fastremap
```
*If not, a C++ compiler is required.*
```bash
pip install numpy
pip install fastremap --no-binary :all:
```
## Manual Installation
*A C++ compiler is required.*
```bash
sudo apt-get install g++ python3-dev
mkvirtualenv -p python3 fastremap
pip install numpy
# Choose one:
python setup.py develop
python setup.py install
```
## C++ Usage
You can also use the in-place transposition function in C++ directly.
```cpp
#include "ipt.hpp"
int main() {
int sx = 128;
int sy = 124;
int sz = 103;
int sw = 3;
auto* arr = ....;
// All primitive number types supported
// The array will be modified in place,
// so these functions are void type.
ipt::ipt<int>(arr, sx, sy); // 2D
ipt::ipt<float>(arr, sx, sy, sz); // 3D
ipt::ipt<double>(arr, sx, sy, sz, sw); // 4D
return 0;
}
```
--
Made with <3
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
fastremap-1.2.1.tar.gz
(217.2 kB
view details)
Built Distributions
File details
Details for the file fastremap-1.2.1.tar.gz
.
File metadata
- Download URL: fastremap-1.2.1.tar.gz
- Upload date:
- Size: 217.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b03f34c3edec23ea6819215456838ef3b73256769bc2ca488b0f855fbac9cb2d |
|
MD5 | 4fa9afcf2b9e45412506c7f2285c1100 |
|
BLAKE2b-256 | 46be1e7871c64b9aa96264f4637cfae52651c3610d637272ad9e571817991853 |
File details
Details for the file fastremap-1.2.1-cp37-cp37m-manylinux1_x86_64.whl
.
File metadata
- Download URL: fastremap-1.2.1-cp37-cp37m-manylinux1_x86_64.whl
- Upload date:
- Size: 1.3 MB
- Tags: CPython 3.7m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3e11d180058056f312cb8efdd48601213e4d6ef032be14f87aedb7ca86a097bf |
|
MD5 | e43a78712aa656ff7eb4fa96315cfabd |
|
BLAKE2b-256 | 9c95fab42eb5025a21c8f1a36c5370054e768f7e645f9c2c67a96d17d95a2331 |
File details
Details for the file fastremap-1.2.1-cp36-cp36m-manylinux1_x86_64.whl
.
File metadata
- Download URL: fastremap-1.2.1-cp36-cp36m-manylinux1_x86_64.whl
- Upload date:
- Size: 1.3 MB
- Tags: CPython 3.6m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 882b654d88f2ecfd98983afc2c2d171342a62afd67fdbaba23de0be2fe9665b8 |
|
MD5 | 304bc3a597945c96036515181fcd02a6 |
|
BLAKE2b-256 | a8e791a7d5debc6659c57bf11f2a5502a44d1d6ed34289b5149e54c14a05a5df |
File details
Details for the file fastremap-1.2.1-cp35-cp35m-manylinux1_x86_64.whl
.
File metadata
- Download URL: fastremap-1.2.1-cp35-cp35m-manylinux1_x86_64.whl
- Upload date:
- Size: 1.3 MB
- Tags: CPython 3.5m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6a7410b4bfab1c16e2cdf3dcc97d5dc4059ab695dd7d09ad86ca1cc54f6f421d |
|
MD5 | 625b983c6e3e37f595309e24bc190507 |
|
BLAKE2b-256 | 6ee8dcde5aeb8cb43282ae00f002270e00d105e5c52f9453cb9b93271daf11e2 |
File details
Details for the file fastremap-1.2.1-cp27-cp27m-manylinux1_x86_64.whl
.
File metadata
- Download URL: fastremap-1.2.1-cp27-cp27m-manylinux1_x86_64.whl
- Upload date:
- Size: 1.2 MB
- Tags: CPython 2.7m
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.6.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 321b3b522d512af7f748fa5a4db77b14517c5ebea88c184bbca82c30a3fc7ba1 |
|
MD5 | eced9be6dcdb42155e37a6999947597c |
|
BLAKE2b-256 | 518285172be6f2497b1d27b931c172469d967bc7bd2a00691f59a52a7531bfb2 |