Skip to main content

Pythonic Image Acquisition Library for Machine Vision People

Project description

The Harvesters

Pieter Bruegel the Elder, The Harvesters, 1565, (c) The Metropolitan Museum of Art

https://readthedocs.org/projects/harvesters/badge/?version=latest https://img.shields.io/pypi/v/harvesters.svg https://img.shields.io/pypi/pyversions/harvesters.svg https://zenodo.org/badge/133908095.svg

Introduction

Hello everyone. I’m Kazunari, the author of Harvester and a technical contributor to GenICam.

Since I opened the source code of Harvester in May 2018, so many people tried out that and gave me very positive feedback that improves Harvester. The number of feedback and the number of people who I discussed with was much more than I expected what I had in the beginning and it truly was one of the exciting experiences I ever had in my professional career.

The original motivation that drove me to develop Harvester was like this: Until I got the idea of Harvester, I had to learn and adapt to popular proprietary 3rd party image acquisition/processing libraries from scratch every time even though I just wanted to acquire an image. To be honest, I had felt it’s a bit annoying. A straightforward solution to the issue was to have a unified image acquisition library. In addition, as a guy who likes to work with Python, I wanted to make it a Python module to boost productivity.

I believe that Harvester can help you to concentrate on image processing that you really have to be responsible for. Of course, you would have to implement the algorithm again using other sophisticated and powerful proprietary libraries to optimize the performance in reality. However, (this is very important so let me stress this) Harvester is just a productive sandbox that encapsulates the image acquisition process and does not intend to take the place of such superior image processing libraries because they were designed for exactly that purpose. Harvester just wants to shorten the time you spend to realize your brilliant ideas as much as possible. With this meaning, Harvester can be considered as a tool that helps you to quickly iterate the prototyping process. Note that it is vital for designers because the prototyping process defines the basic quality of the final product. Fortunately, Harvester has got firm support from the high-quality GenTL Producers on the market. It means you can smoothly start working with any GenICam compliant cameras that you may want to work with.

So, everyone, there’s no worry anymore. Keep having fun working. Harvester has got your back.

Greetings, Kazunari Kudo.

Harvester as Python Module

Techincally speaking, Harvester consists of two Python modules, Harvester Core and Harvester GUI, and technically speaking, each library is responsible for the following tasks:

  • Harvester Core: Image acquisition & remote device manipulation
  • Harvester GUI: Image data visualization

Harvester consumes image acquisition libraries, so-called GenTL Producers. Just grabbing a GenTL Producer and GenICam compliant machine vision cameras, then Harvester will supply you the acquired image data as numpy array to make your image processing task productive.

You can freely use, modify, distribute Harvester under Apache License-2.0 without worrying about the use of your software: personal, internal or commercial.

Currently, Harvester is being developed by the motivated contributors from all over the world.

Where Is The Name From?

Harvester’s name was coming from the great Flemish painter, Pieter Bruegel the Elder’s painting so-called “The Harvesters”. You can see the painting in the top of this page. Harvesters harvest a crop every season that has been fully grown and the harvested crop is passed to the consumers. On the other hand, image acquisition libraries acquire images as their crop and the images are passed to the following processes. We found the similarity between them and decided to name our library Harvester.

Apart from anything else, we love its peaceful and friendly name. We hope you also like it ;-)


Disclaimer: All external pictures should have associated credits. If there are missing credits, please tell us, we will correct it. Similarly, all excerpts should be sourced. If not, this is an error and we will correct it as soon as you tell us.


Development Status

The Harvester project has started since April 2018 and it’s still under development as of October 2018 but many developers and researchers over the world have already confirmed that it is actually usable with the popular GenTL Producers and GenICam compliant cameras from the following companies. We have realized the progress had been brought by all interested people’s positive expectation in the machine vision market and we strongly believe it will sustain to the following years. Of course, we will never forget the importance of volunteer companies which provided us their products to test Harvester. Thank you very much!

Note that we as the committee have not prepared any formal certification procedure for Harvester. The following results were presented by Harvester users who confirmed Harvester worked for their use cases. It is true that the following information does not cover whole domain but as a fact it is helpful sometimes at least.

Company Name GenTL Producer for CoaXPress GenTL Producer for GigE Vision GenTL Producer for USB3 Vision GenICam compliant cameras
Active Silicon Worked Not applicable Not applicable Not applicable
Adimec Not applicable Not applicable Not applicable Worked
Allied Vision Not tested Not tested Not tested Worked
Automation Technology Not applicable Not applicable Not applicable Worked
Basler Not applicable Not applicable Worked Worked
Baumer Optronic Not applicable Worked Worked Worked
CREVIS Not applicable Not tested Not applicable Worked
CRITICAL LINK Not applicable Not applicable Worked Worked
DAHENG VISION Not applicable Worked Worked Worked
Euresys Worked Not tested Not tested Not applicable
FLIR Not applicable Not applicable Not applicable Worked
Gardasoft Not applicable Not applicable Not applicable Worked
JAI Not tested Worked Worked Worked
The IMAGING SOURCE Not tested Not tested Not tested Worked
Lucid Vision Labs Not applicable Worked Not applicable Worked
MACNICA Inc. Not tested Not tested Worked Worked
MATRIX VISION Not applicable Worked Worked Worked
Matrox Imaging Worked Not applicable Not applicable Not applicable
OMRON SENTECH Not tested Not tested Worked Worked
PCO Not applicable Not applicable Not applicable Not tested
Roboception Not applicable Not applicable Not applicable Worked
SICK Not applicable Worked Not applicable Worked
Silicon Software Not tested Not tested Not tested Not applicable
STEMMER IMAGING Not tested Worked Worked Not applicable
Teledyne DALSA Not tested Not applicable Not applicable Worked
Vieworks Not tested Not applicable Not applicable Not tested
XIMEA Not tested Not tested Not tested Not tested

Please don’t hesitate to tell us if you have tested Harvester with your GenTL Producer or GenICam compliant device. We will add your company/organization name to the list.

Need a GUI?

Would you like to have a GUI? Harvester has a sister project that is called Harvester GUI. Oops, there’s no punch line on its name! Please take a look its source repository if you are interested in it:

https://github.com/genicam/harvesters_gui

Image data visualizer

Announcements

  • Version 1.0.2: Resolves issue #109.
  • Version 1.0.1: Resolves issue #104.
  • Version 1.0.0: Resolves issue #83.
  • Version 0.2.18: Resolves issue #103.
  • Version 0.2.17: Resolves issue #100.
  • Version 0.2.16: Resolves issue #98. No changes in the functionality.
  • Version 0.2.15: Resolves issue #97. No changes in the functionality.
  • Version 0.2.14: Resolves issue #96. No changes in the functionality.
  • Version 0.2.13: Resolves issues #88, #89, #90, and #95.
  • Version 0.2.12: Resolves issue #84.
  • Version 0.2.11: Resolves issue #82.
  • Version 0.2.10: Resolves issue #81.
  • Version 0.2.9: Resolves issues #76 and #78.
  • Version 0.2.8: Resolves issues #73 and #75.
  • Version 0.2.7: Resolves issues #70 and #72.
  • Version 0.2.6: Resolves issues #67 and #68.
  • Version 0.2.5: Resolves issue #63.
  • Version 0.2.4: Resolves issue #64.
  • Version 0.2.3: Resolves issues #57, #58, #59, #60, #61, and #62.
  • Version 0.2.2: Resolves issue #56.
  • Version 0.2.1: Resolves issue #55.
  • Version 0.2.0: Resolves issue #53.
  • Version 0.1.0: Note that this version will be deprecated and the following versions will be incompatible with any of 0.1.n versions.

Online Resources

Asking Questions

We have opened a chat room for you. Please don’t hesitate to leave your message any time when you get a question regarding Harvester!

https://gitter.im/genicam-harvester/chatroom

We have also prepared an FAQ page. Perhaps your issue could be resolved just reading through it.

https://github.com/genicam/harvesters/wiki/FAQ

GenTL Producers

As of today, we have tested Harvester with the following GenTL Producers and it definitely is the shortest way to get one from the following list to get Harvester working with tangible machine vision cameras:

Company Name SDK Name Camera Manufacture Free
Basler AG Pylon No
Baumer Optronic Baumer GAPI SDK No
DAHENG VISION MER Galaxy View No
JAI JAI SDK; note that their GenTL Producer will be discontinued shortly. Yes
Matrix Vision mvIMPACT_Acquire Yes
OMRON SENTECH StCamUSBPack No
STEMMER IMAGING Common Vision Blox Yes

You might be able to directly download one at their website but please note that perhaps some of them could require you to register your information to get one. In addition, some GenTL Producers might block you to connect other competitors’ cameras.

Terminology

Before start talking about the detail, let’s take a look at some important terminologies that frequently appear in this document. These terminologies are listed as follows:

  • The GenApi-Python Binding: A Python module that communicates with the GenICam reference implementation.
  • A GenTL Producer: A library that has C interface and offers consumers a way to communicate with cameras over physical transport layer dependent technology hiding the detail from the consumer.
  • The GenTL-Python Binding: A Python module that communicates with GenTL Producers.
  • Harvester: A Python module that consists of Harvester Core and Harvester GUI.
  • Harvester Core: A part of Harvester that works as an image acquisition engine.
  • Harvester GUI: A part of Harvester that works as a graphical user interface of Harvester Core.
  • A GenICam compliant device: It’s typically a camera. Just involving the GenICam reference implementation, it offers consumers a way to dynamically configure/control the target remote devices.

The following diagram shows the hierarchy and relationship of the relevant modules:

Module hierarchy

Getting Started

In this section, we will learn how to instruct procedures to get Harvester work.

System Requirements

The following software modules are required to get Harvester working:

  • Either of Python 3.4, 3.5, 3.6, or 3.7 (Only 64bit versions are supported as of October 2018.)

In addition, please note that we don’t supported Cygwin on Windows. This restriction is coming from a fact that the GenICam reference implementation has not supported it.

In addition, you will need the following items to let Harvester make something meaningful:

  • GenTL Producers
  • GenICam compliant machine vision cameras

Harvester has been confirmed it works with the following 64-bit operating systems:

  • Fedora 27
  • macOS 10.13
  • Red Hat Enterprise Linux Workstation 7.4
  • Ubuntu 14.04
  • Windows 7
  • Windows 10

Note that it’s just a snapshot at a moment. If you are curious to know the reality, just make a try because Harvester is for free!

Installing Python

First, let’s install Python. There are several options for you but I would like to introduce you Anaconda here. You can download Anaconda from the following URL:

https://www.anaconda.com/download/

For Windows, please find a 64-Bit graphical installer that fits your machine and download it. The installation process is straightforward but it could be a bad idea to add the Anaconda Python executable directory to the PATH environment variable because it means your system begins to use your Anaconda Python instead of the system Python that had been already installed before you installed Anaconda Python.

To not letting Anaconda Python interfere in your system Python, not adding Anaconda Python to the PATH and you should always launch Anaconda Prompt in the Anaconda3 (64-bit) folder from the Windows’s start menu. It will automatically kick up the Anaconda Python so that you can immediately use the functionality that Anaconda provides you.

On Linux machines, you can make it with the following steps. First, please type the following command. Invoking that command, you will be able to use the conda command which allows you to activate an environment; note that the following code has been modified for my setup on a macOS machine:

$ echo ". /Users/kznr/anaconda3/etc/profile.d/conda.sh" >> ~/.bash_profile

Then activate the root environment:

$ conda activate

Now you can start working for installing Harvester.

Creating an Environment

After installing a Python, let’s create an isolated environment where does not interfere in your system. An environment is very helpful for developers because everything will be okay just deleting the environment if you completely corrupted it by accident. Please imagine a case where you corrupt the system-wide Python. It’s obviously a nightmare and it will enforce you to spend some days to recover it so it is very recommended to work in an isolated environment when you need to develop something.

Assume we have added the Anaconda Python executable directory to the PATH environment variable. To create an environment on a UNIX system, please type the following command; we name the environment genicam:

$ conda create -n genicam python=3.6

We have created an environment genicam with Python 3.6. If you prefer to install another version, just change the version number above.

After that, we activate the environment to work with Harvester. To activate the environment, type the following command:

$ conda activate genicam

If it works well then you will be able to find genicam in the shell prompt as follows:

(genicam) kznr@Kazunaris-MacBook:~%

Then let’s check the version number of Python. To check the version number of Python, type the following command:

$ python --version

You should be able to see the expected version number in its return as follows:

Python 3.6.5 :: Anaconda, Inc.

Finally, to deactivate the environment, type the following command:

$ conda deactivate

It’s so easy.

Installing a GenTL Producer

Now we install a GenTL Producer that works with Harvester. Harvester can’t acquire images without it.

Today, many camera manufacturers and software vendors all over the world provide GenTL Producers to support image acquisition using GenICam compliant cameras. However, you should note that some GenTL Producers may block cameras from other competitors. Though it’s perfectly legal but we recommend you here to use a GenTL Producer from MATRIX VISION as a one of reliable GenTL Producer for this tutorial because it doesn’t block cameras from other competitors. However, please respect their license and give them feedback immediately if you find something to be reported or something that you appreciate. As an open source activity, we would like to pay our best respect to their attitude and their products.

You can get their SDK from the following URL; please download mvIMPACT_Acquire and install it.

http://static.matrix-vision.com/mvIMPACT_Acquire/2.29.0/

Once you installed their SDK, you can find the appropriate GenTL Producer just grepping *.cti. Note that Harvester supports only 64-bit version of GenTL Producers as of November 2018.

This is just for your information but you can find the list of other reliable GenTL Producers here.

Installing Harvester Core

Before installing Harvester, let’s make sure that you are working in the environment that you created in the previous chapter.

After that, you can install Harvester via PyPI invoking the following command; note that the package name is harvesters but not harvester; unfortunately, the latter word had been reserved by another project:

$ pip install harvesters

For people who those have already installed it:

$ pip install --upgrade harvesters

Or more simply:

$ pip install -U harvesters

Perhaps pip could install cached package. If you want to install the newly dowloaded package, you should invoke the following command:

$ pip install -U --no-cache-dir harvesters

These commands will automatically install the required modules such as numpy or genicam2 (the Python Binding for the GenICam GenApi & the GenTL Producers) if the module has not yet installed on your environment.

Getting back to the original topic, you could install the latest development version it using setup.py cloning Harvester from GitHub:

$ git clone https://github.com/genicam/harvesters.git && cd harvesters && python setup.py install

Working with Harveseter

Harvester Core is an image acquisition engine. No GUI. You can use it as an image acquisition library which acquires images from GenTL Producers through the GenTL-Python Binding and controls the target remote device (it’s typically a camera) through the GenApi-Python Binding.

Harvester Core works as a minimalistic front-end for image acquisition. Just importing it from your Python script, you should immediately be able to set images on your table.

You’ll be able to download the these language binding runtime libraries from the EMVA website, however, it’s not available as of May 2018, because they have not officially released yet. Fortunately they are in the final reviewing process so hopefully they’ll be released by the end of 2018.

If you don’t have to care about the display rate for visualizing acquired images, the combination of Harvester Core and Matplotlib might be a realistic option for that purpose.

Tasks Harvester Core Does for You

The main features of Harvester Core are listed as follows:

  • Image acquisition through GenTL Producers
  • Multiple loading of GenTL Producers in a single Python script
  • GenICam feature node manipulation of the target remote device

Note that the second item implies you can involve multiple types of transport layers in your Python script. Each transport layer has own advantages and disadvantages and you should choose appropriate transport layers following your application’s requirement. You just need to acquire images for some purposes and the GenTL Producers deliver the images somehow. It truly is the great benefit of the GenTL Standard! And of course, not only GenTL Producers but Harvester Core offer you a way to manipulate multiple remote devices in a single Python script with an intuitive manner.

On the other hand, Harvester Core could be considered as a simplified version of the GenTL-Python Binding; actually, Harvester Core hides it in its back and shows only intuitive interfaces to its clients. Harvester Core just offers you a relationship between you and a remote device. Nothing more. We say it again, just you and a remote device. If you need to manipulate more relevant GenTL modules or have to achieve something over a hardcore way, then you should directly work with the GenTL-Python Binding.

Pixel Formats That Harvester Core Supports

Currently, Harvester Core supports the following pixel formats that are defined by the Pixel Format Naming Convention:

Mono8, Mono10, Mono12, Mono16, RGB8, RGBa8, BayerRG8, BayerGR8, BayerBG8, BayerGB8, BayerRG16, BayerGR16, BayerBG16, BayerGB16

Note that Harvester does not support packed formats which allow multiple elements of information reside in a byte data.

Harvester Core on IPython

The following code block shows Harvester Core is running on IPython. An acquired image is delivered as the payload of a buffer and the buffer can be fetched by calling the fetch_buffer method of the ImageAcquirer class. Once you get an image you should be able to immediately start image processing. If you’re running on the Jupyter notebook, you should be able to visualize the image data using Matplotlib. This step should be helpful to check what’s going on your trial in the image processing flow.

(genicam) kznr@Kazunaris-MacBook:~% ipython
Python 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 11:07:29)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from harvesters.core import Harvester

In [2]: import numpy as np  # This is just for a demonstration.

In [3]: h = Harvester()

In [4]: h.add_cti_file('/Users/kznr/dev/genicam/bin/Maci64_x64/TLSimu.cti')

In [5]: h.update_device_info_list()

In [6]: len(h.device_info_list)
Out[6]: 4

In [7]: h.device_info_list[0]
Out[7]: (id_='TLSimuMono', vendor='EMVA_D', model='TLSimuMono', tl_type='Custom', user_defined_name='Center', serial_number='SN_InterfaceA_0', version='1.2.3')

In [8]: ia = h.create_image_acquirer(0)

In [9]: ia.remote_device.node_map.Width.value = 8

In [10]: ia.remote_device.node_map.Height.value = 8

In [11]: ia.remote_device.node_map.PixelFormat.value = 'Mono8'

In [12]: ia.start_image_acquisition()

In [13]: with ia.fetch_buffer() as buffer:
    ...:     # Let's create an alias of the 2D image component:
    ...:     component = buffer.payload.components[0]
    ...:
    ...:     # Note that the number of components can be vary. If your
    ...:     # target remote device transmits a multi-part information, then
    ...:     # you'd get two or more components in the payload. However, now
    ...:     # we're working with a remote device that transmits only a 2D image.
    ...:     # So we manipulate only index 0 of the list object, components.
    ...:
    ...:     # Let's see the acquired data in 1D:
    ...:     _1d = component.data
    ...:     print('1D: {0}'.format(_1d))
    ...:
    ...:     # Reshape the NumPy array into a 2D array:
    ...:     _2d = component.data.reshape(
    ...:         component.height, component.width
    ...:     )
    ...:     print('2D: {0}'.format(_2d))
    ...:
    ...:     # Here are some trivial calculations:
    ...:     print(
    ...:         'AVE: {0}, MIN: {1}, MAX: {2}'.format(
    ...:             np.average(_2d), _2d.min(), _2d.max()
    ...:         )
    ...:     )
    ...:
1D: [123 124 125 126 127 128 129 130 124 125 126 127 128 129 130 131 125 126
 127 128 129 130 131 132 126 127 128 129 130 131 132 133 127 128 129 130
 131 132 133 134 128 129 130 131 132 133 134 135 129 130 131 132 133 134
 135 136 130 131 132 133 134 135 136 137]
2D: [[123 124 125 126 127 128 129 130]
 [124 125 126 127 128 129 130 131]
 [125 126 127 128 129 130 131 132]
 [126 127 128 129 130 131 132 133]
 [127 128 129 130 131 132 133 134]
 [128 129 130 131 132 133 134 135]
 [129 130 131 132 133 134 135 136]
 [130 131 132 133 134 135 136 137]]
AVE: 130.0, MIN: 123, MAX: 137

In [14]: ia.stop_image_acquisition()

In [15]: ia.destroy()

In [16]: h.reset()

In [17]: quit
(genicam) kznr@Kazunaris-MacBook:~%

The Harvester Workflow

Acquiring Images

First, let’s import Harvester:

from harvesters.core import Harvester

Then instantiate a Harvester object; we’re going to use h that stands for Harvester as its identifier.

h = Harvester()

And load a CTI file; loading a CTI file, you can communicate with the GenTL Producer:

# ATTENTION! Please use the CTI file in the original location!

# Why? Visit https://github.com/genicam/harvesters/wiki/FAQ and
# read "I pointed out a CTI file but Harvester says the image doesn't
# exist (Part 2)."

h.add_cti_file('path/to/gentl_producer.cti')

Note that you can add one or more CTI files on a single Harvester Core object. To add another CTI file, just repeat calling add_cti_file method passing another target CTI file:

h.add_cti_file('path/to/another_gentl_producer.cti')

And the following code will let you know the CTI files that have been loaded on the Harvester object:

h.cti_files

In a contrary sense, you can remove a specific CTI file that you have added with the following code:

h.remove_cti_file('path/to/gentl_producer.cti')

And now yol have to update the list of remote devices; it fills up your device information list and you’ll select a remote device to control from the list:

h.update_device_info_list()

The following code will let you know the remote devices that you can control:

h.device_info_list

Our friendly GenTL Producer, so called TLSimu, gives you the following information:

[(unique_id='TLSimuMono', vendor='EMVA_D', model='TLSimuMono', tl_type='Custom', user_defined_name='Center', serial_number='SN_InterfaceA_0', version='1.2.3'),
 (unique_id='TLSimuColor', vendor='EMVA_D', model='TLSimuColor', tl_type='Custom', user_defined_name='Center', serial_number='SN_InterfaceA_1', version='1.2.3'),
 (unique_id='TLSimuMono', vendor='EMVA_D', model='TLSimuMono', tl_type='Custom', user_defined_name='Center', serial_number='SN_InterfaceB_0', version='1.2.3'),
 (unique_id='TLSimuColor', vendor='EMVA_D', model='TLSimuColor', tl_type='Custom', user_defined_name='Center', serial_number='SN_InterfaceB_1', version='1.2.3')]

And you create an image acquirer object specifying a target remote device. The image acquirer does the image acquisition task for you. In the following example it’s trying to create an acquirer object of the first candidate remote device in the device information list:

ia = h.create_image_acquirer(0)

Or equivalently:

ia = h.create_image_acquirer(list_index=0)

You can connect the same remote device passing more unique information to the method. In the following case, we specify a serial number of the target remote device:

ia = h.create_image_acquirer(serial_number='SN_InterfaceA_0')

You can specify a target remote device using properties that are provided through the device_info_list property of the Harvester class object. Note that it is invalid if the specifiers gives you two ore more remote devices. Please specify sufficient information so that the combination gives you a unique target remote device.

We named the image acquirer object ia in the above example but in a practical occasion, you may give it a purpose oriented name like ia_face_detection. Note that a camera itself does NOT acquirer/receive images but it just transmits them. In a machine vision application, there should be two roles at least: One transmits images and the other acquires them. The ImageAcquirer class objects play the latter role and it holds a camera as the remote_device object, the source of images.

Anyway, then now we start image acquisition:

ia.start_image_acquisition()

Once you started image acquisition, you should definitely want to get an image. Images are delivered to the acquirer allocated buffers. To fetch a buffer that has been filled up with an image, you can have 2 options; the first option is to use the with statement:

with ia.fetch_buffer() as buffer:
    # Work with the Buffer object. It consists of everything you need.
    print(buffer)
    # The buffer will automatically be queued.

Having that code, the fetched buffer is automatically queued once the code step out from the scope of the with statement. It’s prevents you to forget queueing it by accident. The other option is to manually queue the fetched buffer by yourself:

buffer = ia.fetch_buffer()
print(buffer)
# Don't forget to queue the buffer.
buffer.queue()

In this option, again, please do not forget that you have to queue the buffer by yourself. If you forget queueing it, then you’ll lose a buffer that could be used for image acquisition. Everything is up to your design, so please choose an appropriate way for you. In addition, once you queued the buffer, the Buffer object will be obsolete. There’s nothing to do with it.

Okay, then you would stop image acquisition with the following code:

ia.stop_image_acquisition()

And the following code disconnects the connecting remote device from the image acquirer; you’ll have to create an image acquirer object again when you have to work with a remote device:

ia.destroy()

If you finished working with the Harvester object, then release the acquired resources calling the reset method:

h.reset()

Now you can quit the program! Please not that Harvester and ImageAcquirer also support the with statement. So you may write program as follows:

with Harvester() as h:
    with h.create_image_acquirer(0) as ia:
        # Work, work, and work with the ia object.
        # The ia object will automatically call the destroy method
        # once it goes out of the block.

    # The h object will automatically call the reset method
    # once it goes out of the block.

This way prevents you forget to release the acquired external resources. If this notation doesn’t block your use case then you should rely on the with statement.

Reshaping a NumPy Array as an Image

We have learned how to acquire images from a target remote device through an ImageAcquirer class object. In this section, we will learn how to reshape the acquired image into another that can be used by your application.

First, you should know that Harvester Core returns you an image as a 1D NumPy array.

buffer = ia.fetch_buffer()
_1d = buffer.payload.components[0].data

Perhaps you may expect to have it as a 2D array but Harvester Core doesn’t in reality because if Harvester Core provides an image as a specific shape, then it could limit your algorithm that you can apply to get the image that fits to your expected shape. Instead, Harvester Core provides you an image as a 1D array and also provides you required information that you would need while you’re reshaping the original array to another.

The following code is an except from Harvester GUI that reshapes the source 1D array to another to draw it on the VisPy canvas. VisPy canvas takes content as an image to draw:

from harvesters.util.pfnc import mono_location_formats, \
    rgb_formats, bgr_formats, \
    rgba_formats, bgra_formats

payload = buffer.payload
component = payload.components[0]
width = component.width
height = component.height
data_format = component.data_format

# Reshape the image so that it can be drawn on the VisPy canvas:
if data_format in mono_location_formats:
    content = component.data.reshape(height, width)
else:
    # The image requires you to reshape it to draw it on the
    # canvas:
    if data_format in rgb_formats or \
            data_format in rgba_formats or \
            data_format in bgr_formats or \
            data_format in bgra_formats:
        #
        content = component.data.reshape(
            height, width,
            int(component.num_components_per_pixel)  # Set of R, G, B, and Alpha
        )
        #
        if data_format in bgr_formats:
            # Swap every R and B:
            content = content[:, :, ::-1]
    else:
        return

Note that component.num_components_per_pixel returns a float so please don’t forget to cast it when you pass it to the reshape method of NumPy array. If you try to set a float then the method will refuse it.

It’s not always but sometimes you may have to handle image formats that require you to newly create another image calculating each pixel component value referring to the pixel location. To help such calculation, Component2DImage class provides the represent_pixel_location method to tell you the 2D pixel location that corresponds to the pixel format. The pixel location is defined by Pixel Format Naming Convention, PFNC in short. The array that is returned by the method is a 2D NumPy array and it corresponds to the model that is defined by PFNC.

pixel_location = component.represent_pixel_location()

The 2D array you get from the method is equivalent to the definition that is given by PFNC. The following screenshot is an excerpt from the PFNC 2.1:

The definition of the pixel location of LMN422 formats

For example, if you acquired a YCbCr422_8 format image, then the first and the second rows of pixel_location would look as follows; L is used to denote the 1st component, M is for the 2nd, and N is for the 3rd, and they correspond to Y, Cb, and Cr respectively; in the following description, for a given pixel, the first index represents the row number and the second index represents the column number and note that the following index notation is based on one but not zero though you will use the zero based notation in your Python script:

[Y11, Cb11, Y12, Cr11, Y13, Cb13, Y14, Cr13, ...]
[Y21, Cb21, Y22, Cr21, Y23, Cb23, Y24, Cr23, ...]

Having that pixel location, you should be able to convert the color space of each row from YCbCr to RGB.

import numpy as np
# Create the output array that has been filled up with zeros.
rgb_2d = np.zeros(shape=(height, width, 3), dtype='uint8')
# Calculate each pixel component using pixel_location.
# Calculation block follows:
#     ...

For example, if you have an 8 bits YCbCr709 image, then you can get the RGB values of the first pixel calculating the following formula:

\begin{align*} R_{11} &= 1.16438 (Y_{11} - 16) &                           & + 1.79274 (Cr_{11} - 128) \\G_{11} &= 1.16438 (Y_{11} - 16) & - 0.21325 (Cb_{11} - 128) & - 0.53291 (Cr_{11} - 128) \\B_{11} &= 1.16438 (Y_{11} - 16) & - 0.21240 (Cb_{11} - 128) \\\end{align*}

Similarly, you can get the RGB values of the second pixel calculating the following formula:

\begin{align*} R_{12} &= 1.16438 (Y_{12} - 16) &                           & + 1.79274 (Cr_{11} - 128) \\G_{12} &= 1.16438 (Y_{12} - 16) & - 0.21325 (Cb_{11} - 128) & - 0.53291 (Cr_{11} - 128) \\B_{11} &= 1.16438 (Y_{11} - 16) & - 0.21240 (Cb_{11} - 128) \\\end{align*}

Once you finished filling up each pixel with a set of RGB values, then you’ll be able to handle it as a RGB image but not a YCbCr image.

You can download the standard document of PFNC at the EMVA website.

Manipulating GenICam Feature Nodes

Probably almost of the Harvester users would be interested in manipulating GenIcam feature nodes through Harvester. Let’s assume that we are going to control a GenICam feature node called Foo.

To get the value of Foo, we code as follows:

a = ia.remote_device.node_map.Foo.value

On the other hand, if Foo is an Integer node then we code as follows to set a value:

ia.remote_device.node_map.Foo.value = 42

If Foo is a Boolean node, then you code as follows:

ia.remote_device.node_map.Foo.value = True

Or if Foo is an Enumeration node, then you code as follows; it also works for a case where Foo is a String node:

ia.remote_device.node_map.Foo.value = 'Bar'

If Foo is a Command node, then you can execute the command with the following

ia.remote_device.node_map.Foo.execute()

There you can dive much more deeper in the GenICam GenApi but the description above would be sufficient for a general use.

Ah, one more thing. You may want to know the available GenICam feature nodes in the target remote physical device. In such a case, you can probe them calling the dir function as follows:

dir(ia.remote_device.node_map)

You should be able to find (probably) familiar feature names in the output.

Appendix

Open Source Resources

Harvester Core uses the following open source libraries/resources:

Acknowledgement

The initial idea about Harvester suddenly came up to a software engineer, Kazunari Kudo’s head in the early April of year 2018 and he immediately decided to bring the first prototype to the International Vision Standards Meeting, IVSM in short, that was going to be held in Frankfurt am Main in the following early May. During the Frankfurt IVSM, interested engineers tried out Harvester and confirmed it really worked using commercial machine vision cameras provided by well-known machine vision camera manufacturers in the world. Having that fact, the attendees of the IVSM warmly welcomed Harvester.

The following individuals have directly or indirectly contributed to the development activity of Harvester or encouraged the developers by their thoughtful warm words; they are our respectable wonderful colleagues:

Rod Barman, Stefan Battmer, David Beek, Jan Becvar, David Bernecker, Chris Beynon, Eric Bourbonnais, Benedikt Busch, George Chamberlain, Thomas Detjen, Friedrich Dierks, Dana Diezemann, Emile Dodin, Reynold Dodson, Sascha Dorenbeck, Jozsa Elod, Erik Eloff, Katie Ensign, Andreas Ertl, James Falconer, Werner Feith, Maciej Gara, Andreas Gau, Sebastien Gendreau, Francois Gobeil, Werner Goeman, Jean-Paul Goglio, Markus Grebing, Eric Gross, Ioannis Hadjicharalambous, Uwe Hagmaier, Tim Handschack, Christopher Hartmann, Reinhard Heister, Gerhard Helfrich, Jochem Herrmann, Heiko Hirschmueller, Tom Hopfner, David Hoese, Karsten Ingeman Christensen, Severi Jaaskelainen, Mattias Johannesson, Mark Jones, Mattias Josefsson, Martin Kersting, Stephan Kieneke, Tom Kirchner, Lutz Koschorreck, Frank Krehl, Maarten Kuijk, Max Larin, Ralf Lay, Min Liu, Sergey Loginonvskikh, Thomas Lueck, Alain Marchand, Rocco Matano, Masahide Matsubara, Stephane Maurice, Robert McCurrach, Mike Miethig, Thies Moeller, Roman Moie, Katsura Muramatsu, Marcel Naggatz, Hartmut Nebelung, Damian Nesbitt, Quang Nhan Nguyen, Klaus-Henning Noffz, Neerav Patel, Jan Pech, Merlin Plock, Joerg Preckwinkel, Benjamin Pussacq, Dave Reaves, Thomas Reuter, Gordon Rice, Andreas Rittinger, Ryan Robe, Nicolas P. Rougier, Felix Ruess, Matthias Schaffland, Michael Schmidt, Jan Scholze, Martin Schwarzbauer, Rupert Stelz, Madhura Suresh, Chendra Hadi Suryanto, Andrew Wei Chuen Tan, Timo Teifel, Albert Theuwissen, Laval Tremblay, Tim Vlaar, Silvio Voitzsch, Stefan Von Weihe, Frederik Voncken, Roman Wagner, Ansger Waschki, Anne Wendel, Michael Williamson, Jean-Michel Wintgens, Manfred Wuetschner, Jang Xu, Christoph Zierl, Sebastian Yap, and Juraj Zopp

Column: Seeing and Recognition

The following is a short column that was written by one of my favorite photographers/philosophers, Katsura Muramatsu. Even though the column was exclusively dedicated to her exhibition called “Natura naturans”, she generously allowed me to excerpt the column for this Harvester project. It would give us an opportunity to thinking about seeing and recognition we regularly do. To me, at least, they sound like a habit on which we unconsciously premise when we do machine vision: We tend to see everything in a way that we want to interpret. Of course, I would like to take the fact in a positive way though.

“When we talk to someone, we implicitly or explicitly try to find a piece of evidence in his/her eyes that he/she is alive. The same situation happens in a case where we face a stuffed animal and the face, or especially the eyes play a much more important role rather than its fur or other parts. When people visit this place, the museum where I took photos of these stuffed animals, they would feel that they are seen by the animals rather than they see the animals even though the animals are not alive anymore. In fact, the eyes of the stuffed animals are made of glass or other materials such as plastic. I frequently ask myself why we felt that we were seen through their eyes made of those materials.” - Katsura Muramatsu

Ordo: Eastern Wolf

Title: “Ordo: Eastern Wolf” (2018)

© Katsura Muramatsu All Rights Reserved

http://hellerraum.nobody.jp

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for harvesters, version 1.0.2
Filename, size File type Python version Upload date Hashes
Filename, size harvesters-1.0.2-py3-none-any.whl (82.8 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size harvesters-1.0.2.tar.gz (110.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page