Skip to main content

Image acquisition & data visualization with Python based on GenICam

Project description

The Harvesters

Pieter Bruegel the Elder, The Harvesters, 1565, oil on wood, © 2000–2018 The Metropolitan Museum of Art

About Harvester

Harvester is a friendly companion for people who those want to learn computer vision.

Technically speaking, Harvester is a Python library which is responsible for the following tasks:

  • Image acquisition

  • Image data visualization (optional)

Harvester consumes image acquisition libraries, so-called GenTL Producers. If you have an officially certified GenTL Producer and GenICam compliant machine vision cameras, then Harvester supply you the acquired image data as numpy array to make your image processing task productive.

You can freely use, modify, distribute Harvester under Apache License-2.0 without worrying about the use of your software: personal, internal or commercial.

Currently, Harvester is being developed and maintained by the motivated volunteer contributors from all over the world.

Asking Questions

We have prepared a chat room in Gitter. Please don’t hesitate to drop your message when you get a question regarding Harvester!

https://gitter.im/genicam-harvester/chatroom

Collaborators

So far, Harvester has tested GenTL Producers and GenICam compliant devices from the following companies and they gave Harvester opportunities to improve its quality:

Company Name

GigE Vision

USB3 Vision

Devices

Active Silicon

-

-

N/A

Adimec

N/A

N/A

-

Allied Vision

-

-

-

Autometion Technology

N/A

N/A

Tested

Baumer Optronic

Tested

Tested

Tested

DAHENG VISION

-

Tested

Tested

Euresys

-

-

N/A

FLIR

N/A

N/A

-

Gardasoft

N/A

N/A

Tested

JAI

-

Tested

-

Lucid Vision Labs

-

N/A

-

MATRIX VISION

Tested

Tested

-

OMRON SENTECH

-

Tested

Tested

PCO

-

-

-

Roboception

N/A

N/A

Tested

SICK

-

N/A

-

Silicon Software

-

-

N/A

STEMMER IMAGING

Tested

Tested

N/A

Vieworks

-

-

-

XIMEA

-

-

-

Please don’t hesitate to tell us if you have tested Harvester with your GenTL Producer or GenICam compliant device. We will add your company/organization name to the list.

Terminology

Before start talking about the detail, let’s take a look at some important terminologies that frequently appear in this document. These terminologies are listed as follows:

  • The GenApi-Python Binding:

    A Python module that communicates with the GenICam reference implementation.
  • A GenTL Producer:

    A C/C++ library that offers consumers a way to communicate with cameras over physical transport layer dependent technology hiding the detail from the consumer.
  • The GenTL-Python Binding:

    A Python module that communicates with GenTL Producers.
  • Harvester:

    A Python module that consists of Harvester Core and Harvester GUI.
  • Harvester Core:

    A part of Harvester that works as an image acquisition engine.
  • Harvester GUI:

    A part of Harvester that works as a graphical user interface of Harvester Core.
  • A GenICam compliant device:

    It’s typically a camera. Just involving the GenICam reference implementation, it offers consumers a way to dynamically configure/control the target devices.

The following diagram shows the hierarchy and relationship of the relevant modules:

Module hierarchy

Features

Harvester mainly consists of the following two Python modules:

  • Harvester Core: An image acquisition engine

  • Harvester GUI: Graphical user interface between users & Harvester Core

In this section, we will learn what Harvester offers us through these components.

Harvester Core

Harvester Core is an image acquisition engine. No GUI. You can use it as an image acquisition library which acquires images from GenTL Producers through the GenTL-Python Binding and control the target device (it’s typically a camera) through the GenApi-Python Binding.

Harvester Core works as a minimalistic front-end for image acquisition. Just importing it from your Python script, you should immediately be able to set images on your table.

You’ll be able to download the these language binding runtime libraries from the EMVA website, however, it’s not available as of May 2018, because they have not officially released yet. Fortunately they are in the final reviewing process so hopefully they’ll be release by the end of 2018.

If you don’t have to visualize acquired images at high frame rate, the combination of Harvester Core and Matplotlib might be ideal for that purpose.

Harvester Core’s Main Features

The main features of Harvester Core are listed as follows:

  • Image acquisition over GenTL Producers

  • Multiple loading of GenTL Producers in a single Python script

  • GenICam feature node manipulation of the target device

Note that the second item implies you can involve multiple types of transport layers in your Python script. Each transport layer has own advantages and disadvantages and you should choose appropriate transport layers following your application’s requirement. You just need to acquire images for some purposes and the GenTL Producers deliver the images somehow. It truly is the great benefit of encapsulation by the GenTL Standard!

On the other hand, Harvester Core could be considered as a simplified version of the GenTL-Python Binding; actually Harvester Core hides it in its back and shows only intuitive interfaces to its clients. Harvester Core just offers you a relationship between you and a device. Nothing more. We say it again, just you and a device. If you need to manipulate more relevant GenTL modules or have to achieve something over a hardcore way, then you should directly work with the GenTL-Python Binding.

Harvester GUI

Harvester GUI works on the top of Harvester Core and offers you high-performance image data visualization on the fly. It involves VisPy for controlling OpenGL functionality and PyQt for providing GUI.

Harvester GUI’s Main Features

The main features of Harvester GUI are listed as follows:

  • Image data visualization of the acquired images

  • Image magnification using a mouse wheel or a trackpad

  • Image dragging using a mouse or a trackpad

  • An arbitrary selection of image displaying point in the data path (Not implemented yet)

Unlike Harvester Core, Harvester GUI limits the number of GenTL Producers to load just one. This is just a limitation to not make the GUI complicated. In general, the user should know which GenTL Producer should be loaded to control his target device. It’s not necessary to load multiple GenTL Producers for this use case. However, this is just an idea in an early stage. We might support multiple loading on even Harvester GUI in the future.

Supported Pixel Formats

Currently Harvester GUI supports the following pixel formats that are defined by the Pixel Format Naming Convention:

  • Mono8

  • RGB8

  • BayerRG8, BayerGR8, BayerBG8, and BayerGB8 (No demosaicing supported)

Screenshots

In this section, we see some useful windows which Harvester offers you.

Image Data Visualizer Window

The image data visualizer window (below) offers you a visualization of the acquired images. In this screenshot, Harvester is acquiring a 4000 x 3000 pixel of RGB8 image at 30 fps; it means it’s acquiring images at 8.6 Gbps. It’s quite fast isn’t it?

Image data visualizer

Attribute Controller Window

The attribute controller window (below) offers you to manipulate GenICam feature nodes of the target device. Changing exposure time, triggering the target device for image acquisition, storing a set of camera configuration so-called User Set, etc, you can manually control the target device anytime when you want to. It supports visibility filter feature and regular expression feature. These features are useful in a case where you need to display only the features you are interested.

Attribute Controller

Harvester Core on IPython

The following screenshot shows Harvester Core is running on IPython. Harvester Core returns the latest image data at the moment as a Numpy array every time its user call the get_image() method. Once you get an image you should be able to immediately start image processing. If you’re running on Jupyter notebook, you should be able to visualize the image data using Matplotlib. This step should be helpful to check what’s going on your trial in the image processing flow.

Harvester on IPython
(genicam) kznr@Kazunaris-MacBook:~% ipython
Python 3.5.3 |Continuum Analytics, Inc.| (default, Mar  6 2017, 12:15:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.3.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from harvesters.core import Harvester

In [2]: h = Harvester()

In [3]: h.add_cti_file('/Users/kznr/dev/genicam/bin/Maci64_x64/TLSimu.cti')

In [4]: h.update_device_info_list()

In [5]: for i, info in enumerate(h.device_info_list):
   ...:     print('{0}: {1}'.format(i, info.display_name))
   ...:
0: Test_Mono (SN0)
1: Test_Color (SN1)
2: Test_Mono (SN0)
3: Test_Color (SN1)

In [6]: h.connect_device(0)

In [7]: h.device.node_map.Width.value = 16

In [8]: h.device.node_map.Height.value = 8

In [9]: h.device.node_map.PixelFormat.value = 'Mono8'

In [10]: h.start_image_acquisition()

In [11]: with h.fetch_buffer() as b:
    ...:     print('1D: {0}'.format(b.image.ndarray))
    ...:     print('2D: {0}'.format(b.image.ndarray.reshape(b.image.height, b.image.width)))
    ...:
1D: [ 6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21  7  8  9 10 11 12 13 14
 15 16 17 18 19 20 21 22  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 10 11 12 13 14 15 16 17
 18 19 20 21 22 23 24 25 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 13 14 15 16 17 18 19 20
 21 22 23 24 25 26 27 28]
2D: [[ 6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21]
 [ 7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22]
 [ 8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
 [ 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]
 [10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25]
 [11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26]
 [12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27]
 [13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28]]

In [12]: h.stop_image_acquisition()

Requirements

System Requirements

  • Python 3.4 or higher

  • Officially certifiled GenTL Producers

  • GenICam compliant machine vision cameras

Supported Operating Systems

Harvester has been tested with the following operating systems.

  • macOS

  • Ubuntu

  • Windows

Installation

In this section, we will learn how to instruct procedures to get Harvester work.

Overview

In short, you may think which tools are required to get Harvester work. The answer is listed as follows:

  • The Python Bindings for GenApi & GenTL

  • The GenICam reference implementation

  • A certified GenTL Producer

  • A GenICam compliant machine vision camera

The first two items will be able to downloaded from the EMVA website in the future. Regarding the 3rd item, you should be able to get proprietary product from software vendors who sell image acquisition library. Regarding the 4th item, you should be able to purchase from machine vision camera manufactures all over the world.

Installing Harvester

You can install Harvester via PyPI invoking the following command:

$ pip install harvesters numpy

Or you could clone Harvester first and manually install it using setup.py.

$ git clone https://github.com/genicam/harvester.git
$ python setup.py install

If you want to use Harvester GUI, then please install the following modules:

$ pip install PyQt5 vispy

Installing the Python Bindings

Harvester requires the GenApi-Python Binding and the GenTL-Python Binding. As of July 2018, it’s not officially distributed yet by EMVA. Having that fact, the only way to get those is building them by yourself.

The source code can be downloaded from the following URL using Subversion:

$ svn co https://genicam.mvtec.com/svn/genicam/branches/_dev_teli_kazunari_1881_20180121/

Note that you need to be a member of EMVA to download a working copy from their repository. To learn about the detail please visit the following EMVA website:

https://www.emva.org/about-emva/membership/

To build the library, please read the README file which is located at the following directory in the source package:

genicam/source/Bindings/README.rst

Reading that file, you should be able to learn everything you need to build the Python Bindings by yourself.

In the final release, a setup.py for the Python Bindings will be provided with the GenICam reference implementation package that is distributed at the EMVA website but it’s not been available yet. Until it turns available, you may go on a way to manually create a symlink to bridge your target Python interpreter and the built/provided binaries.

The Python Bindings should be packed as genicam2. So on Windows, to create a symlink genicam2 in your target Python interpreter, please invoke the following command:

$ mklink /D path/to/target/site-packages/genicam2 path/to/source/genicam/bin/Win64_x64

Or, on Linux or macOS, you can do the same invoking the following command; please replace Linux64_x64 with Maci64_x64:

$ ln –s path/to/source/genicam/bin/Linux64_x64 path/to/target/site-packages/genicam2

Remarks Regaring the Python Bindings

Please remind that you need to be careful when you choose a Python version (especially Anaconda Python, maybe?) because some distributions have different directory structure or linking symbols. It simply breaks the Python Bindings. We have started collecting some results from our experiences and have listed them in the “System Configuration Matrix” section in the README file. We hope it helps you to save your time.

How to Use Harvester Core

First, let’s import Harvester:

import harvesters.core import Harvester

Then instantiate a Harvester object:

h = Harvester()

And load a CTI file:

h.add_cti_file('path/to/gentl_producer.cti')

Note that you can add more CTI files. To add another CTI file, just repeat calling add_cti_file method passing the target CTI file. In contrary sense, you can remove a specific CTI file that you have added with the following code:

h.remove_cti_file('path/to/gentl_producer.cti')

And now we have to update the list of devices:

h.update_device_info_list()

The following code will list devices that you can access:

for i, info in enumerate(h.device_info_list):
    print('{0}: {1}'.format(i, info.display_name))

And you connect a device to the Harvester object:

h.connect_device(0)

The following code starts image acquisition:

h.start_image_acquisition()

To fetch a buffer that has been filled up with an image, you can have 2 options; the first option is to use the with statement:

with h.fetch_buffer() as buffer:
    print(buffer.image.ndarray)

Having that code, the fetched buffer, buffer, is automatically queued once the code step out from the scop of the with statement. It’s prevents to queue it by accident. The other option is to manually queue the fetched buffer by yourself:

buffer = h.fetch_buffer()
print(buffer.image.ndarray)
h.queue_buffer(buffer)

In this option, again, do not forget that you have to queue the buffer by yourself. If you forgot queueing it, then you’ll lose a buffer that can be used for image acquisition.

The following code stops image acquisition:

h.stop_image_acquisition()

And the following code disconnects the device from Harvester:

h.disconnect_device()

Now you can quit the program.

How to Use Harvester GUI

You can use Harvester GUIT with the following code:

import sys
from PyQt5.QtWidgets import QApplication


if __name__ == '__main__':
    app = QApplication(sys.argv)
    h = Harvester()
    h.show()
    sys.exit(app.exec_())

Harvester GUI

Harvester GUI :: Image Data Visualizer Window

Image Data Visualizer Window :: Toolbar

Most of Harvester GUI’s features can be used through its toolbox. In this section, we describe each button’s functionality and how to use it. Regarding shortcut keys, replace Ctrl with Command on macOS.

Toolbar
Selecting a CTI file
Open file

This button is used to select a GenTL Producer file to load. The shortcut key is Ctrl+o.

Updating GenTL information
Update

This button is used to update GenTL information of the GenTL Producer that you are loading on Harvester. The shortcut key is Ctrl+u. It might be useful when you newly connect a device to your system.

Selecting a GenICam compliant device

This combo box shows a list of available GenICam compliant devices. You can select a device that you want to control. The shortcut key is Ctrl+D, i.e., Ctrl+Shift+d.

Connecting a selected device to Harvester
Connect

This button is used to connect a device which is being selected by the former combo box. The shortcut key is Ctrl+c. Once you connect the device, the device is exclusively controlled.

Disconnecting the connecting device from Harvester
Disconnect

This button is used to disconnect the connecting device from Harvester. The shortcut key is Ctrl+d.

Starting image acquisition
Start image acquisition

This button is used to start image acquisition. The shortcut key is Ctrl+j. The acquired images will be drawing in the following canvas pane.

Pausing/Resuming image drawing
Pause

This button is used to pausing/resuming drawing images on the canvas pane while it’s keep acquiring images in the background. The shortcut key is Ctrl+k. If you want to resume drawing images, just click the button again. You can do the same thing with the start image acquisition button (Ctrl+j).

Stopping image acquisition
Stop image acquisition

This button is used to stop image acquisition. The shortcut key is Ctrl+l.

Showing the device attribute dialog
Device attribute

This button is used to show the device attribute dialog. The shortcut key is Ctrl+a. The device attribute dialog offers you to a way to intuitively control device attribute over a GUI.

Showing the about dialog
About

This button is used to show the about dialog.

Image Data Visualizer Window :: Canvas

The canvas of Harvester GUI offers you not only image data visualization but also some intuitive object manipulations.

Canvas
Zooming into the displayed image

If you’re using a mouse, spin the wheel to your pointing finger points at. If you are using a trackpad on a macOS, slide two fingers to the display side.

Zooming out from the displayed image

If you’re using a mouse, spin the wheel to your side. If you are using a trackpad on a macOS, slide two fingers to your side.

Changing the part being displayed

If you’re using a mouse, grab any point in the canvas and drag the pointer as if you’re physically grabbing the image. The image will follow the pointer. If you are using a trackpad on a macOS, it might be useful if you assign the three finger slide for dragging.

Harvester GUI :: Attribute Controller Window

The attribute controller offers you an interface to each GenICam feature node that the the target device provides.

Attribute Controller Window :: Toolbar

Toolbar
Filtering GenICam feature nodes by visibility

This combo box offers you to apply visibility filter to the GenICam feature node tree. The shortcut key is Ctrl+v

GenICam defines the following visibility levels:

  • Beginner: Features that should be visible for all users via the GUI and API.

  • Expert: Features that require a more in-depth knowledge of the camera functionality.

  • Guru: Advanced features that might bring the cameras into a state where it will not work properly anymore if it is set incorrectly for the cameras current mode of operation.

  • Invisible: Features that should be kept hidden for the GUI users but still be available via the API.

The following table shows each item in the combo box and the visibility status of each visibility level:

Combo box item

Beginner

Expert

Guru

Invisible

Beginner

Visible

Invisible

Invisible

Invisible

Expert

Visible

Visible

Invisible

Invisible

Guru

Visible

Visible

Visible

Invisible

All

Visible

Visible

Visible

Visible

Filtering GenICam feature nodes by regular expression

This text edit box offers you to filter GenICam feature nodes by regular expression.

Expanding the feature node tree
Expand feature node tree

This button is used to expand the feature node tree. The shortcut key is Ctrl+e.

Collapsing the feature node tree
Collapse feature node tree

This button is used to collapse the feature node tree. The shortcut key is Ctrl+c.

Harvester Core

TODO: Finish writing article.

Acknowledgements

Open Source Resources

Harvester Core uses the following open source libraries/resources:

Harvester GUI uses the following open source libraries/resources:

Why Harvester?

Harvester’s name was derived from the great Flemish painter, Pieter Bruegel the Elder’s painting so-called “The Harvesters”. Harvesters harvest a crop every season that has been fully grown and the harvested crop is passed to the consumers. On the other hand, image acquisition libraries acquire images as their crop and the images are passed to the following processes. We found the similarity between them and decided to name our library Harvester.

Apart from anything else, we love its peaceful and friendly name. We hope you also like it ;-)

Credits

The initial idea about Harvester suddenly came up to Kazunari Kudo’s head in the early April 2018 and he decided to bring the first prototype to the following International Vision Standards Meeting. During the Frankfurt International Vision Standards Meeting which was held in May 2018, people confirmed Harvester really worked using machine vision cameras provided by well-known machine vision camera manufacturers in the world. Having that fact, the attendees warmly welcomed Harvester.

The following individuals have directly or indirectly contributed to the development activity of Harvester or encouraged the developers by their thoughtful warm words:

Rod Barman, Stefan Battmer, David Beek, Jan Becvar, David Bernecker, Chris Beynon, Eric Bourbonnais, George Chamberlain, Thomas Detjen, Friedrich Dierks, Dana Diezemann, Emile Dodin, Reynold Dodson, Sascha Dorenbeck, Erik Eloff, Katie Ensign, Andreas Ertl, James Falconer, Werner Feith, Maciej Gara, Andreas Gau, Sebastien Gendreau, Francois Gobeil, Werner Goeman, Jean-Paul Goglio, Markus Grebing, Eric Gross, Ioannis Hadjicharalambous, Uwe Hagmaier, Tim Handschack, Christopher Hartmann, Reinhard Heister, Gerhard Helfrich, Jochem Herrmann, Heiko Hirschmueller, Tom Hopfner, David Hoese, Karsten Ingeman Christensen, Severi Jaaskelainen, Mattias Johannesson, Mark Jones, Mattias Josefsson, Martin Kersting, Stephan Kieneke, Tom Kirchner, Lutz Koschorreck, Frank Krehl, Maarten Kuijk, Max Larin, Ralf Lay, Min Liu, Sergey Loginonvskikh, Thomas Lueck, Alain Marchand, Rocco Matano, Masahide Matsubara, Stephane Maurice, Robert McCurrach, Mike Miethig, Thies Moeller, Roman Moie, Marcel Naggatz, Hartmut Nebelung, Damian Nesbitt, Quang Nhan Nguyen, Klaus-Henning Noffz, Neerav Patel, Jan Pech, Merlin Plock, Joerg Preckwinkel, Benjamin Pussacq, Dave Reaves, Thomas Reuter, Andreas Rittinger, Ryan Robe, Nicolas P. Rougier, Matthias Schaffland, Michael Schmidt, Jan Scholze, Martin Schwarzbauer, Rupert Stelz, Madhura Suresh, Chendra Hadi Suryanto, Timo Teifel, Albert Theuwissen, Laval Tremblay, Tim Vlaar, Silvio Voitzsch, Stefan Von Weihe, Frederik Voncken, Roman Wagner, Ansger Waschki, Anne Wendel, Jean-Michel Wintgens, Manfred Wuetschner, Jang Xu, Christoph Zierl, and Juraj Zopp

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

harvesters-0.post0.dev322.tar.gz (1.5 MB view hashes)

Uploaded Source

Built Distribution

harvesters-0.post0.dev322-py3-none-any.whl (58.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page