Skip to main content

Function for local patch extraction from OpenCV keypoints with proper bluring

Project description

Extract_patches

Simple function for local patch extraction from local features keypoints

Install

pip install extract_patches

How to use

extract_patches accepts following formats:

  • OpenCV keypoints
  • Ellipse format [x y a b c], see further in Oxford-Affine
  • Affine features format [x y a11 a12 a21 a22], see further here
  • OpenCV keypoints + A (a11 a12 a21, a22), say from AffNet output

First, let's delect some local features, e.g. OpenCV ORB.

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import cv2
import math
import seaborn as sns
from time import time
from PIL import Image
from extract_patches.core import extract_patches

img1 = cv2.cvtColor(cv2.imread('data/img/prague.png'), cv2.COLOR_BGR2RGB)

det = cv2.ORB_create(500)
kps1, descs1 = det.detectAndCompute(img1,None)

vis_img1 = None
vis_img1 = cv2.drawKeypoints(cv2.cvtColor(img1,cv2.COLOR_RGB2GRAY),kps1,vis_img1, 
                             flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
plt.imshow(vis_img1)
<matplotlib.image.AxesImage at 0x7feb58b46790>

png

from extract_patches.core import extract_patches

extract_patches performs extraction from the appropriate level of image pyramid, removing high freq artifacts. Border mode is set to "replicate", so the patch don't have crazy black borders.

PATCH_SIZE is output patch size.

mrSize is a scale coefficient, related to the image area covered in the original image by local feature. There are different conventions (if any common), e.g. for ORB is mrSize is recommend to set to 1.0, as kpt.size already contains correct number. For the OpenCV SIFT, on the other hand, one should use mrSize=6.0

PATCH_SIZE = 65
mrSize = 1.0
t=time()
patches = extract_patches(kps1, img1, PATCH_SIZE, mrSize, 'cv2')
print ('pyr OpenCV version for 500 kps, [s]', time()-t)

show_idx = 300
fig = plt.figure(figsize=(12, 20))
for i in range(1,6):
    fig.add_subplot(1, 5, i) 
    plt.imshow(patches[show_idx+i])
pyr OpenCV version for 500 kps, [s] 0.025847196578979492

png

Now try with ellipse (x y a b c) format. Let's download Hessian-Affine from VGG website and detect local features with it

!rm h_affine.ln
!wget http://www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/h_affine.ln.gz
!gunzip h_affine.ln.gz
!chmod +x h_affine.ln
!./h_affine.ln  -hesaff -i img/prague.png -o prague.hesaff -thres 100
--2020-01-27 15:21:55--  http://www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/h_affine.ln.gz
Resolving www.robots.ox.ac.uk (www.robots.ox.ac.uk)... 129.67.94.2
Connecting to www.robots.ox.ac.uk (www.robots.ox.ac.uk)|129.67.94.2|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3199317 (3.1M) [application/x-gzip]
Saving to: ‘h_affine.ln.gz’

h_affine.ln.gz      100%[===================>]   3.05M  1.80MB/s    in 1.7s    

2020-01-27 15:21:56 (1.80 MB/s) - ‘h_affine.ln.gz’ saved [3199317/3199317]

hessian affine  detector...
cgood 1902 cbad 560 all 2462
cor nb 1679
detection time: 0.4

number of points : 1562
output file: prague.hesaff

Now read extracted local features from txt file

from extract_patches.laf import visualize_LAFs, ells2LAFs
ells = np.loadtxt('prague.hesaff', skiprows=2).astype(np.float32)
print (f"Shape is {ells.shape}")
print (ells[0:5])
Shape is (1562, 5)
[[ 1.33920e+02  1.25280e+02  3.40137e-02 -2.62884e-02  9.84345e-02]
 [ 6.63840e+02  1.85760e+02  4.18373e-02  2.33503e-02  7.24527e-02]
 [ 6.78240e+02  1.92960e+02  1.07543e-01 -2.07333e-02  3.04518e-02]
 [ 4.14720e+02  1.98720e+02  3.24049e-02 -3.11269e-03  7.01242e-02]
 [ 5.68800e+02  2.00160e+02  2.22278e-02  3.34806e-02  1.39287e-01]]

Now visualize detected features

from extract_patches.laf import visualize_LAFs, ells2LAFs
visualize_LAFs(img1, ells2LAFs(ells))

png

And visualize some patches

show_idx=1500
PATCH_SIZE = 65
mrSize = 5.0

t=time()
patches_ells = extract_patches(ells, img1, PATCH_SIZE, mrSize, 'ellipse')
el=time()-t
print (f'extract from ellipse features for 1500 kps, {el:.5f} [s]', )
fig = plt.figure(figsize=(14, 20))
for i in range(1,6):
    fig.add_subplot(1, 5, i) 
    plt.imshow(patches_ells[show_idx+i])

extract from ellipse features for 1500 kps, 0.24170 [s]

png

Let's try now MSER detector, which could output local features in affine format

#And lets try x y a11 a12 a21 a22 format. MSER can output in it
!wget http://www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/mser.tar.gz
!tar -xf mser.tar.gz
!./mser.ln -i img/prague.png -o prague.mser  -t 4
--2020-01-27 15:22:05--  http://www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/mser.tar.gz
Resolving www.robots.ox.ac.uk (www.robots.ox.ac.uk)... 129.67.94.2
Connecting to www.robots.ox.ac.uk (www.robots.ox.ac.uk)|129.67.94.2|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 558415 (545K) [application/x-gzip]
Saving to: ‘mser.tar.gz.6’

mser.tar.gz.6       100%[===================>] 545.33K  1.66MB/s    in 0.3s    

2020-01-27 15:22:05 (1.66 MB/s) - ‘mser.tar.gz.6’ saved [558415/558415]

Read and extract from MSERs

def read_mser_file(fname):
    with open(fname, 'r') as f:
        out = []
        lines = f.readlines()
        num_feats1 = int(lines[0])
        num_feats2 = int(lines[0+num_feats1+1])
        for l in lines[1:num_feats1]:
            out.append(np.array([float(x) for x in l.strip().split(' ')]).reshape(1,-1))
        for l in lines[num_feats1+2:]:
            out.append(np.array([float(x) for x in l.strip().split(' ')]).reshape(1,-1))
    return np.concatenate(out,axis=0)[:,:6]
mser_xyA = read_mser_file('prague.mser')
print (f"Shape is {mser_xyA.shape}")
print (mser_xyA[0:5])
visualize_LAFs(img1, mser_xyA)
Shape is (361, 6)
[[ 79.2818   410.027     12.2143    -3.72324   -3.72324    3.38642 ]
 [  6.23611  433.042      7.13002   -1.6485    -1.6485     4.45321 ]
 [546.992    445.621      8.79801    2.71022    2.71022   13.7868  ]
 [659.924    344.894      7.26102    0.842102   0.842102   1.73304 ]
 [651.968    344.841     15.5017     1.13132    1.13132    3.68829 ]]

png

PATCH_SIZE = 65
mrSize = 5.0

t=time()
patches_mser = extract_patches(mser_xyA, img1, PATCH_SIZE, mrSize, 'xyA')
el = time()-t
print (f'extract from a11, a12, a21, a22 features for 360 kps, {el:.5f} [s]')

show_idx=150
fig = plt.figure(figsize=(14, 20))
for i in range(1,6):
    fig.add_subplot(1, 5, i) 
    plt.imshow(patches_mser[show_idx+i])
extract from a11, a12, a21, a22 features for 360 kps, 0.02284 [s]

png

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

extract_patches-0.1.31.tar.gz (15.4 kB view details)

Uploaded Source

Built Distribution

extract_patches-0.1.31-py3-none-any.whl (13.7 kB view details)

Uploaded Python 3

File details

Details for the file extract_patches-0.1.31.tar.gz.

File metadata

  • Download URL: extract_patches-0.1.31.tar.gz
  • Upload date:
  • Size: 15.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/45.1.0.post20200127 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.7.4

File hashes

Hashes for extract_patches-0.1.31.tar.gz
Algorithm Hash digest
SHA256 55b0e494823245ed54b6995ef252db50671dac5c36b639129fd994add8dc97df
MD5 2c85e09e0b875e0235d1dc83c1a49f0e
BLAKE2b-256 da462afe164d8bf886bbcbd4d9c4d27e3a947e0bc1c63acbb49160cfdf00ef27

See more details on using hashes here.

File details

Details for the file extract_patches-0.1.31-py3-none-any.whl.

File metadata

  • Download URL: extract_patches-0.1.31-py3-none-any.whl
  • Upload date:
  • Size: 13.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/45.1.0.post20200127 requests-toolbelt/0.9.1 tqdm/4.42.0 CPython/3.7.4

File hashes

Hashes for extract_patches-0.1.31-py3-none-any.whl
Algorithm Hash digest
SHA256 6c2070881556afd87df9513753e3303c2090cb9ee355657ea53792ac9f445c77
MD5 b03fc092b33894b5bbbc8fdcadcf6c21
BLAKE2b-256 a336cf1bec081e49ccc470199c5b7ecbaf1c663023366d08181acab6412ecac8

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page