bubblemask 1.0.1

Package for applying Gaussian 'Bubbles' masks to images.

bubblemask

Python package for applying the Gaussian 'Bubbles' mask to image stimuli, as described by Gosselin and Schyns (2001). This approach applies a mask to an images, with a number of Gaussian 'bubblemask' providing windows to the actual pixel values. The method is useful for probing the functional impact of information at different locations in an image (e.g., informativeness of different face regions for emotion recognition). The method can also be applied to examine the size of such functional regions (varying sigma of the Gaussian bubbles), or features like colour (applying the technique to RGB separately) or the spatial frequency of relevant information (applying to specific frequency bandwidths).

This method applies a mask with any number of bubbles, optionally with per-bubble sigma parameters, to a given image. The 2-D bubbles are calculated using the outer product of 1-D Gaussian densities.

Installation

Install via pip:

pip install bubblemask

Basic Usage

from bubblemask import mask
import os.path as op
from PIL import Image

mask.bubbles_mask() is the main function, which generates and applies a mask with len(sigma) bubbles to a PIL image. By default, these will be positioned randomly. Here, we add 5 bubbles, of various sigmas, to an image of a face on grey background.

face = Image.open(op.join('img', 'pre', 'face.png'))
face1, mask1, mu_x, mu_y, sigma = mask.bubbles_mask(im=face, sigma=[17, 19, 20.84, 25, 30], bg=127)

face.show(); face1.show()

The function also outputs the mask as a numpy array.

import matplotlib.pyplot as plt
plt.imshow(mask1)
plt.colorbar()

The function also outputs the x and y locations of the centres of the Gaussian bubbles (mu_x and mu_y) and the corresponding sigma values (equal to provided sigma argument).

print(mu_x)

[151.47868249  30.62953573  67.66242641 248.33505263 189.49367428]

print(mu_y)

[ 27.5013962  231.37643177 292.48458643 215.76040095  87.04159864]

print(sigma)

[17, 19, 20.84, 25, 30]

Specifying Bubble Locations

By default, bubbles_mask() will position bubbles randomly in the image. The exact desired locations of bubbles can be specified via the mu_x and mu_y arguments. Here I specify two bubbles to be centred on eyes, with different sigma values, of 20 and 10. Note that mu_x and mu_y can be floats.

face2 = mask.bubbles_mask(
    im=face, mu_x=[85, 186.7], mu_y=[182.5, 182.5], sigma=[20, 10], bg=127
)[0]

face2.show()

Using a Convolution-Based Method

Previous implementations I've seen have used a convolution-based approach, where bubble locations are convolved with a Gaussian kernel. This is also available, with the build.build_conv_mask() and mask.bubbles_conv_mask() functions. Key differences are that:

• Sigma values must be identical for all bubbles if one kernel is applied globally (could alternatively average over multiple per-sigma convolutions)
• Locations of x and y must be integers (rounded if floats) so that bubble precision is limited by resolution of the image

Here is a comparison of the methods:

mu_x = [85, 21, 47, 254, 193]
mu_y = [186, 102, 219, 63, 80]
sigma = [20, 20, 20, 20, 20]

# method using outer products of Gaussian densities
face3a, mask3a, _, _, _ = mask.bubbles_mask(im=face, mu_x=mu_x, mu_y=mu_y, sigma=sigma, bg=127)

# method using convolution with Gaussian kernel
face3b, mask3b, _, _, _ = mask.bubbles_conv_mask(im=face, mu_x=mu_x, mu_y=mu_y, sigma=sigma, bg=127)

# compare faces
face3a.show(); face3b.show()

# compare masks
plt.imshow(mask3a); plt.colorbar()
plt.imshow(mask3b); plt.colorbar()

There are only small differences in the approaches, owing to (I think?) imprecision at the extremeties of bubbles in the convolution-based method:

plt.imshow(mask3a-mask3b)
plt.colorbar()

This means that with reasonable rounding of the masks, the approaches would be functionally equivalent, except that the method using the outer product of densities allows you to give mu as floats (better precision).

The density approach of bubblemask is also slightly faster - especially for large images and high sigma values:

Time taken to create a bubbles mask for the convolution and density methods, averaged over 50 iterations per combination of size, sigma, and N bubbles.

Naturalistic Images

Examples above use artificial stimuli on grey backgrounds, but this method can also be applied to more naturalistic, colour stimuli, with the background defined by the bg argument.

cat = Image.open(op.join('img', 'pre', 'cat.jpg'))

cat1 = mask.bubbles_mask(im=cat, sigma=np.repeat(10, 20), bg=127)[0]  # grey background
cat2 = mask.bubbles_mask(im=cat, sigma=np.repeat(10, 20), bg=[127, 0, 127])[0]  # magenta background
cat3 = mask.bubbles_mask(im=cat.convert('RGBA'), sigma=np.repeat(10, 20), bg=[0, 0, 0, 0])[0]  # transparent background

cat.show(); cat1.show(); cat2.show(); cat3.show()

Avoiding Uninformative Locations

It is often more efficient to avoid adding bubbles to regions that you know have no informative information or are irrelevant to your hypothesis, such as the background. bubbles_mask_nonzero() will exclude regions of the background which are sufficiently distant from an informative region.

The centres of each bubble (mu_x, mu_y) will be within max_sigma_from_nonzero multiples of that bubble's sigma value from a non-background (by default, non-zero) pixel in a reference image, ref_im. Background pixels are identified as ref_im <= ref_bg.

The usage is similar to bubbles_mask(), but with additional arguments ref_im (reference image), ref_bg (the cutoff for deciding whether a ref_im pixel is informative), and max_sigma_from_nonzero (how far away from the informative regions can a bubble be).

Imagine we are only interested in the letter a in this image:

a_cat = Image.open(op.join('img', 'pre', 'a_cat.png'))
a_cat.show()

First, we need a reference image, where the target region has values of >0 (e.g., of 1), and the uninformative regions have values of 0. If there is an alpha channel, this should also have a value of 0 for the uninformative regions.

a_cat_ref = Image.open(op.join('img', 'pre', 'a_cat_ref.png'))
a_cat_ref.show()

Now we can apply bubbles to the original image, targeting the letter a. Here, we apply 5 bubbles and specify that the centre of each bubble should be no more than 1 standard deviation away from the non-background pixels of the letter a. Also note that we give ref_bg as [0,0,0,255], because we do not have a transparent alpha in the reference image.

a_cat1 = mask.bubbles_mask_nonzero(
    im=a_cat, ref_im=a_cat_ref, sigma=[10,10,10,10,10], ref_bg=[0,0,0,255], bg=[0,0,0,255], max_sigma_from_nonzero=1
)[0]

a_cat1.show()

Here is a snippet showing that bubbles_mask_nonzero() only selects bubble locations whose centres are $\le$max_sigma_from_nonzero standard deviations of the non-background pixels. Here we apply 1000 bubbles to the letter a, with bubbles' centres at a maximum distance of 1 standard deviations from the character.

a_cat2, maskacat2, mu_x, mu_y, sigma = mask.bubbles_mask_nonzero(
    im=a_cat, ref_im=a_cat_ref, sigma=np.repeat(3, repeats=1000), ref_bg=[0,0,0,255], bg=[0,0,0,255], max_sigma_from_nonzero=1
)

a_cat2.show()
plt.imshow(maskacat2); plt.colorbar()

Finally, you can also define per-bubble constraints for max_sigma_from_nonzero, and values of np.inf and 0 are supported:

a_cat3, maskacat3, mu_x, mu_y, sigma = mask.bubbles_mask_nonzero(
    im=a_cat, ref_im=a_cat_ref,
    sigma = [25, 10, 5],
    max_sigma_from_nonzero = [np.inf, 2.75, 0],
    ref_bg=[0,0,0,255], bg=[0,0,0,255])

a_cat3.show()

Bubble Merging Method

An advantage of this approach is that bubbles of different sizes can be merged. By default, this implementation averages the bubbles and scales the result to within [0, 1]. An alternative may be to take the sum and apply a threshold of the pre-sum maximum across the bubbles. Similarly, the method scales bubbles by default, so that bubbles of different sigma have equal maxima in their densities, where an alternative would be to leave the bubbles unscaled.

Here is a visualisation of the possible options in mask construction, using sum_merge and scale arguments, which can be passed to bubbles_mask():

from bubblemask import build

# same bubble parameters for all masks
mu_y = [20, 30, 70]
mu_x = [20, 30, 90]
sigma = [5, 10, 7.5]
sh = (100, 100)

# plot all mask options (the first is the default)
masks = [build.build_mask(mu_y, mu_x, sigma, sh, scale=True, sum_merge=False),
         build.build_mask(mu_y, mu_x, sigma, sh, scale=True, sum_merge=True),
         build.build_mask(mu_y, mu_x, sigma, sh, scale=False, sum_merge=False),
         build.build_mask(mu_y, mu_x, sigma, sh, scale=False, sum_merge=True)]

for i in range(4):
    plt.imshow(masks[i])
    plt.colorbar()

Command Line Interface

The bubblemask.mask.bubbles_mask() function can be accessed from the command line. This requires an input argument for a file path to the original image, and an --output argument, to write the result to file.

python -m bubblemask --help

usage: bubblemask [-h] -i INPUT -o OUTPUT -s SIGMA [SIGMA ...] [-x MU_X [MU_X ...]]
                  [-y MU_Y [MU_Y ...]] [-b BACKGROUND [BACKGROUND ...]] [--unscaled]
                  [--summerge] [--seed SEED]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        the file path for the input image
  -o OUTPUT, --output OUTPUT
                        the path of the desired output file
  -s SIGMA [SIGMA ...], --sigma SIGMA [SIGMA ...]
                        a list of sigmas for the bubbles, in space-separated format
                        (e.g., "10 10 15")
  -x MU_X [MU_X ...], --mu_x MU_X [MU_X ...]
                        x indices (axis 1 in numpy) for bubble locations, in space-
                        separated format - leave blank (default) for random location
  -y MU_Y [MU_Y ...], --mu_y MU_Y [MU_Y ...]
                        y indices (axis 0 in numpy) for bubble locations, in space-
                        separated format - leave blank (default) for random location
  -b BACKGROUND [BACKGROUND ...], --background BACKGROUND [BACKGROUND ...]
                        the desired background for the image, as a single integer
                        from 0 to 255 (default=0), or space-separated values for each
                        channel in the image
  --unscaled            do not scale the densities of the bubbles to have the same
                        maxima
  --summerge            sum_merge -- should merges, where bubbles overlap, be
                        completed using a simple sum of the bubbles, thresholded to
                        the maxima of the pre-merged bubbles? If not (the default),
                        densities are instead averaged (mean).
  --seed SEED           random seed to use

Example usage:

python -m bubblemask -i img/pre/face.png -o img/post/cli_masked_face.png -s 30 30 20 -b 127 --seed 42