Skip to main content

OCR-D wrapper for arbitrary coords-preserving image operations

Project description

PyPI version Pytest CI Docker Image CD

ocrd_wrap

OCR-D wrapper for arbitrary coords-preserving image operations

Introduction

This offers OCR-D compliant workspace processors for any image processing tools which have some (usable) CLI and do not modify/invalidate image coordinates.

It thus wraps them for OCR-D without the need to write and manage code for each of them individually (exposing/passing/documenting their parameters and usage, managing releases etc). It shifts all the burden to workflow configuration (i.e. defining a suitable parameter set on how to call what program on what data, and installing all the required tools).

It is itself written in Python, and relies heavily on the OCR-D core API. This is responsible for handling METS/PAGE, and providing the OCR-D CLI.

In addition, this aims to wrap existing Python packages for preprocessing as OCR-D processors (one at a time).

Installation

Create and activate a virtual environment as usual.

To install Python dependencies:

make deps

Which is the equivalent of:

pip install -r requirements.txt

To install this module, then do:

make install

Which is the equivalent of:

pip install .

Alternatively, download the prebuilt image from Dockerhub:

docker pull ocrd/wrap

Usage

OCR-D processor interface ocrd-preprocess-image

To be used with PAGE-XML documents in an OCR-D annotation workflow.

Usage: ocrd-preprocess-image [worker|server] [OPTIONS]

  Convert or enhance images

  > Performs coords-preserving image operations via runtime shell calls
  > anywhere.

  > Open and deserialize PAGE input files and their respective images,
  > then iterate over the element hierarchy down to the requested
  > ``level-of-operation`` in the element hierarchy.

  > For each segment element, retrieve a segment image according to the
  > layout annotation (from an existing AlternativeImage, or by cropping
  > via coordinates into the higher-level image, and - when applicable -
  > deskewing.

  > If ``input_feature_selector`` and/or ``input_feature_filter`` is
  > non-empty, then select/filter among the @imageFilename image and the
  > available AlternativeImages the last one which contains all of the
  > selected, but none of the filtered features (i.e. @comments
  > classes), or raise an error.

  > Then write that image into a temporary PNG file, create a new METS
  > file ID for the result image (based on the segment ID and the
  > operation to be run), along with a local path for it, and pass
  > ``command`` to the shell after replacing: - the string ``@INFILE``
  > with that input image path, and - the string ``@OUTFILE`` with that
  > output image path.

  > If the shell returns with a failure, skip that segment with an
  > approriate error message. Otherwise, add the new image to the
  > workspace along with the output fileGrp, and using a file ID with
  > suffix ``.IMG-``, and further identification of the input element.

  > Reference it as AlternativeImage in the element, adding
  > ``output_feature_added`` to its @comments.

  > Produce a new PAGE output file by serialising the resulting
  > hierarchy.

Subcommands:
    worker      Start a processing worker rather than do local processing
    server      Start a processor server rather than do local processing

Options for processing:
  -m, --mets URL-PATH             URL or file path of METS to process [./mets.xml]
  -w, --working-dir PATH          Working directory of local workspace [dirname(URL-PATH)]
  -I, --input-file-grp USE        File group(s) used as input
  -O, --output-file-grp USE       File group(s) used as output
  -g, --page-id ID                Physical page ID(s) to process instead of full document []
  --overwrite                     Remove existing output pages/images
                                  (with "--page-id", remove only those).
                                  Short-hand for OCRD_EXISTING_OUTPUT=OVERWRITE
  --debug                         Abort on any errors with full stack trace.
                                  Short-hand for OCRD_MISSING_OUTPUT=ABORT
  --profile                       Enable profiling
  --profile-file PROF-PATH        Write cProfile stats to PROF-PATH. Implies "--profile"
  -p, --parameter JSON-PATH       Parameters, either verbatim JSON string
                                  or JSON file path
  -P, --param-override KEY VAL    Override a single JSON object key-value pair,
                                  taking precedence over --parameter
  -U, --mets-server-url URL       URL of a METS Server for parallel incremental access to METS
                                  If URL starts with http:// start an HTTP server there,
                                  otherwise URL is a path to an on-demand-created unix socket
  -l, --log-level [OFF|ERROR|WARN|INFO|DEBUG|TRACE]
                                  Override log level globally [INFO]
  --log-filename LOG-PATH         File to redirect stderr logging to (overriding ocrd_logging.conf).

Options for information:
  -C, --show-resource RESNAME     Dump the content of processor resource RESNAME
  -L, --list-resources            List names of processor resources
  -J, --dump-json                 Dump tool description as JSON
  -D, --dump-module-dir           Show the 'module' resource location path for this processor
  -h, --help                      Show this message
  -V, --version                   Show version

Parameters:
   "level-of-operation" [string - "page"]
    PAGE XML hierarchy level to operate on
    Possible values: ["page", "region", "line", "word", "glyph"]
   "input_feature_selector" [string - ""]
    comma-separated list of required image features (e.g.
    binarized,despeckled)
   "input_feature_filter" [string - ""]
    comma-separated list of forbidden image features (e.g.
    binarized,despeckled)
   "output_feature_added" [string - REQUIRED]
    image feature(s) to be added after this operation (if multiple,
    separate by comma)
   "input_mimetype" [string - "image/png"]
    File format to save input images to (tool's expected input)
    Possible values: ["image/bmp", "application/postscript", "image/gif",
    "image/jpeg", "image/jp2", "image/png", "image/x-portable-pixmap",
    "image/tiff"]
   "output_mimetype" [string - "image/png"]
    File format to load output images from (tool's expected output)
    Possible values: ["image/bmp", "application/postscript", "image/gif",
    "image/jpeg", "image/jp2", "image/png", "image/x-portable-pixmap",
    "image/tiff"]
   "command" [string - REQUIRED]
    shell command to operate on image files, with @INFILE as place-holder
    for the input file path, and @OUTFILE as place-holder for the output
    file path

presets

The following example recipes are included in the distribution:

These presets will be distributed with as package resources and resolve by their filename, e.g. ...

ocrd-preprocess-image -p param_scribo-cli-binarize-sauvola-ms-split.json -I OCR-D-IMG -O OCR-D-BIN-OLENA

OCR-D processor interface ocrd-skimage-normalize

To be used with PAGE-XML documents in an OCR-D annotation workflow.

Usage: ocrd-skimage-normalize [worker|server] [OPTIONS]

  Equalize contrast/exposure of images with Scikit-image; stretches the color value/tone to the full dynamic range

  > Performs contrast-enhancing equalization of segment or page images
  > with scikit-image on the workspace.

  > Open and deserialize PAGE input files and their respective images,
  > then iterate over the element hierarchy down to the requested
  > ``level-of-operation`` in the element hierarchy.

  > For each segment element, retrieve a segment image according to the
  > layout annotation (from an existing AlternativeImage, or by cropping
  > via coordinates into the higher-level image, and - when applicable -
  > deskewing), in raw (non-binarized) form.

  > Next, normalize the image according to ``method`` in skimage.

  > Then write the new image to the workspace along with the output
  > fileGrp, and using a file ID with suffix ``.IMG-NRM`` with further
  > identification of the input element.

  > Produce a new PAGE output file by serialising the resulting
  > hierarchy.

Subcommands:
    worker      Start a processing worker rather than do local processing
    server      Start a processor server rather than do local processing

Options for processing:
  -m, --mets URL-PATH             URL or file path of METS to process [./mets.xml]
  -w, --working-dir PATH          Working directory of local workspace [dirname(URL-PATH)]
  -I, --input-file-grp USE        File group(s) used as input
  -O, --output-file-grp USE       File group(s) used as output
  -g, --page-id ID                Physical page ID(s) to process instead of full document []
  --overwrite                     Remove existing output pages/images
                                  (with "--page-id", remove only those).
                                  Short-hand for OCRD_EXISTING_OUTPUT=OVERWRITE
  --debug                         Abort on any errors with full stack trace.
                                  Short-hand for OCRD_MISSING_OUTPUT=ABORT
  --profile                       Enable profiling
  --profile-file PROF-PATH        Write cProfile stats to PROF-PATH. Implies "--profile"
  -p, --parameter JSON-PATH       Parameters, either verbatim JSON string
                                  or JSON file path
  -P, --param-override KEY VAL    Override a single JSON object key-value pair,
                                  taking precedence over --parameter
  -U, --mets-server-url URL       URL of a METS Server for parallel incremental access to METS
                                  If URL starts with http:// start an HTTP server there,
                                  otherwise URL is a path to an on-demand-created unix socket
  -l, --log-level [OFF|ERROR|WARN|INFO|DEBUG|TRACE]
                                  Override log level globally [INFO]
  --log-filename LOG-PATH         File to redirect stderr logging to (overriding ocrd_logging.conf).

Options for information:
  -C, --show-resource RESNAME     Dump the content of processor resource RESNAME
  -L, --list-resources            List names of processor resources
  -J, --dump-json                 Dump tool description as JSON
  -D, --dump-module-dir           Show the 'module' resource location path for this processor
  -h, --help                      Show this message
  -V, --version                   Show version

Parameters:
   "level-of-operation" [string - "page"]
    PAGE XML hierarchy level to operate on
    Possible values: ["page", "region", "line", "word", "glyph"]
   "dpi" [number - 0]
    pixel density in dots per inch (overrides any meta-data in the
    images); disabled when zero
   "black-point" [number - 1.0]
    black point point in percent of luminance/value/tone histogram; up to
    ``black-point`` darkest pixels will be clipped to black when
    stretching
   "white-point" [number - 7.0]
    white point in percent of luminance/value/tone histogram; up to
    ``white-point`` brightest pixels will be clipped to white when
    stretching
   "method" [string - "stretch"]
    contrast-enhancing transformation to use after clipping; ``stretch``
    uses ``skimage.exposure.rescale_intensity`` (globally linearly
    stretching to full dynamic range) and ``adapthist`` uses
    ``skimage.exposure.equalize_adapthist`` (applying over tiles with
    context from 1/8th of the image's width)
    Possible values: ["stretch", "adapthist"]

OCR-D processor interface ocrd-skimage-denoise-raw

To be used with PAGE-XML documents in an OCR-D annotation workflow.

Usage: ocrd-skimage-denoise-raw [worker|server] [OPTIONS]

  Denoise raw images with Scikit-image

  > Performs raw denoising of segment or page images with scikit-image
  > on the workspace.

  > Open and deserialize PAGE input files and their respective images,
  > then iterate over the element hierarchy down to the requested
  > ``level-of-operation`` in the element hierarchy.

  > For each segment element, retrieve a segment image according to the
  > layout annotation (from an existing AlternativeImage, or by cropping
  > via coordinates into the higher-level image, and - when applicable -
  > deskewing), in raw (non-binarized) form.

  > Next, denoise the image with a Wavelet transform scheme according to
  > ``method`` in skimage.

  > Then write the new image to the workspace along with the output
  > fileGrp, and using a file ID with suffix ``.IMG-DEN`` with further
  > identification of the input element.

  > Produce a new PAGE output file by serialising the resulting
  > hierarchy.

Subcommands:
    worker      Start a processing worker rather than do local processing
    server      Start a processor server rather than do local processing

Options for processing:
  -m, --mets URL-PATH             URL or file path of METS to process [./mets.xml]
  -w, --working-dir PATH          Working directory of local workspace [dirname(URL-PATH)]
  -I, --input-file-grp USE        File group(s) used as input
  -O, --output-file-grp USE       File group(s) used as output
  -g, --page-id ID                Physical page ID(s) to process instead of full document []
  --overwrite                     Remove existing output pages/images
                                  (with "--page-id", remove only those).
                                  Short-hand for OCRD_EXISTING_OUTPUT=OVERWRITE
  --debug                         Abort on any errors with full stack trace.
                                  Short-hand for OCRD_MISSING_OUTPUT=ABORT
  --profile                       Enable profiling
  --profile-file PROF-PATH        Write cProfile stats to PROF-PATH. Implies "--profile"
  -p, --parameter JSON-PATH       Parameters, either verbatim JSON string
                                  or JSON file path
  -P, --param-override KEY VAL    Override a single JSON object key-value pair,
                                  taking precedence over --parameter
  -U, --mets-server-url URL       URL of a METS Server for parallel incremental access to METS
                                  If URL starts with http:// start an HTTP server there,
                                  otherwise URL is a path to an on-demand-created unix socket
  -l, --log-level [OFF|ERROR|WARN|INFO|DEBUG|TRACE]
                                  Override log level globally [INFO]
  --log-filename LOG-PATH         File to redirect stderr logging to (overriding ocrd_logging.conf).

Options for information:
  -C, --show-resource RESNAME     Dump the content of processor resource RESNAME
  -L, --list-resources            List names of processor resources
  -J, --dump-json                 Dump tool description as JSON
  -D, --dump-module-dir           Show the 'module' resource location path for this processor
  -h, --help                      Show this message
  -V, --version                   Show version

Parameters:
   "level-of-operation" [string - "page"]
    PAGE XML hierarchy level to operate on
    Possible values: ["page", "region", "line", "word", "glyph"]
   "dpi" [number - 0]
    pixel density in dots per inch (overrides any meta-data in the
    images); disabled when zero
   "method" [string - "VisuShrink"]
    Wavelet filtering scheme to use
    Possible values: ["BayesShrink", "VisuShrink"]

OCR-D processor interface ocrd-skimage-binarize

To be used with PAGE-XML documents in an OCR-D annotation workflow.

Usage: ocrd-skimage-binarize [worker|server] [OPTIONS]

  Binarize images with Scikit-image

  > Performs binarization of segment or page images with scikit-image on
  > the workspace.

  > Open and deserialize PAGE input files and their respective images,
  > then iterate over the element hierarchy down to the requested
  > ``level-of-operation`` in the element hierarchy.

  > For each segment element, retrieve a segment image according to the
  > layout annotation (from an existing AlternativeImage, or by cropping
  > via coordinates into the higher-level image, and - when applicable -
  > deskewing).

  > Next, binarize the image according to ``method`` with skimage.

  > Then write the new image to the workspace along with the output
  > fileGrp, and using a file ID with suffix ``.IMG-BIN`` with further
  > identification of the input element.

  > Produce a new PAGE output file by serialising the resulting
  > hierarchy.

Subcommands:
    worker      Start a processing worker rather than do local processing
    server      Start a processor server rather than do local processing

Options for processing:
  -m, --mets URL-PATH             URL or file path of METS to process [./mets.xml]
  -w, --working-dir PATH          Working directory of local workspace [dirname(URL-PATH)]
  -I, --input-file-grp USE        File group(s) used as input
  -O, --output-file-grp USE       File group(s) used as output
  -g, --page-id ID                Physical page ID(s) to process instead of full document []
  --overwrite                     Remove existing output pages/images
                                  (with "--page-id", remove only those).
                                  Short-hand for OCRD_EXISTING_OUTPUT=OVERWRITE
  --debug                         Abort on any errors with full stack trace.
                                  Short-hand for OCRD_MISSING_OUTPUT=ABORT
  --profile                       Enable profiling
  --profile-file PROF-PATH        Write cProfile stats to PROF-PATH. Implies "--profile"
  -p, --parameter JSON-PATH       Parameters, either verbatim JSON string
                                  or JSON file path
  -P, --param-override KEY VAL    Override a single JSON object key-value pair,
                                  taking precedence over --parameter
  -U, --mets-server-url URL       URL of a METS Server for parallel incremental access to METS
                                  If URL starts with http:// start an HTTP server there,
                                  otherwise URL is a path to an on-demand-created unix socket
  -l, --log-level [OFF|ERROR|WARN|INFO|DEBUG|TRACE]
                                  Override log level globally [INFO]
  --log-filename LOG-PATH         File to redirect stderr logging to (overriding ocrd_logging.conf).

Options for information:
  -C, --show-resource RESNAME     Dump the content of processor resource RESNAME
  -L, --list-resources            List names of processor resources
  -J, --dump-json                 Dump tool description as JSON
  -D, --dump-module-dir           Show the 'module' resource location path for this processor
  -h, --help                      Show this message
  -V, --version                   Show version

Parameters:
   "level-of-operation" [string - "page"]
    PAGE XML hierarchy level to operate on
    Possible values: ["page", "region", "line", "word", "glyph"]
   "dpi" [number - 0]
    pixel density in dots per inch (overrides any meta-data in the
    images); disabled when zero
   "method" [string - "sauvola"]
    Thresholding algorithm to use
    Possible values: ["sauvola", "niblack", "otsu", "gauss", "yen", "li"]
   "window_size" [number - 0]
    For Sauvola/Niblack/Gauss, the (odd) window size in pixels; when zero
    (default), set to DPI
   "k" [number - 0.34]
    For Sauvola/Niblack, formula parameter influencing the threshold
    bias; larger is lighter foreground

OCR-D processor interface ocrd-skimage-denoise

To be used with PAGE-XML documents in an OCR-D annotation workflow.

Usage: ocrd-skimage-denoise [worker|server] [OPTIONS]

  Denoise binarized images with Scikit-image

  > Performs binary denoising of segment or page images with scikit-
  > image on the workspace.

  > Open and deserialize PAGE input files and their respective images,
  > then iterate over the element hierarchy down to the requested
  > ``level-of-operation`` in the element hierarchy.

  > For each segment element, retrieve a segment image according to the
  > layout annotation (from an existing AlternativeImage, or by cropping
  > via coordinates into the higher-level image, and - when applicable -
  > deskewing), in binarized form.

  > Next, denoise the image by removing too small connected components
  > with skimage. (If ``protect`` is non-zero, then avoid removing
  > specks near large connected components up to that distance.)

  > Then write the new image to the workspace along with the output
  > fileGrp, and using a file ID with suffix ``.IMG-DEN`` with further
  > identification of the input element.

  > Produce a new PAGE output file by serialising the resulting
  > hierarchy.

Subcommands:
    worker      Start a processing worker rather than do local processing
    server      Start a processor server rather than do local processing

Options for processing:
  -m, --mets URL-PATH             URL or file path of METS to process [./mets.xml]
  -w, --working-dir PATH          Working directory of local workspace [dirname(URL-PATH)]
  -I, --input-file-grp USE        File group(s) used as input
  -O, --output-file-grp USE       File group(s) used as output
  -g, --page-id ID                Physical page ID(s) to process instead of full document []
  --overwrite                     Remove existing output pages/images
                                  (with "--page-id", remove only those).
                                  Short-hand for OCRD_EXISTING_OUTPUT=OVERWRITE
  --debug                         Abort on any errors with full stack trace.
                                  Short-hand for OCRD_MISSING_OUTPUT=ABORT
  --profile                       Enable profiling
  --profile-file PROF-PATH        Write cProfile stats to PROF-PATH. Implies "--profile"
  -p, --parameter JSON-PATH       Parameters, either verbatim JSON string
                                  or JSON file path
  -P, --param-override KEY VAL    Override a single JSON object key-value pair,
                                  taking precedence over --parameter
  -U, --mets-server-url URL       URL of a METS Server for parallel incremental access to METS
                                  If URL starts with http:// start an HTTP server there,
                                  otherwise URL is a path to an on-demand-created unix socket
  -l, --log-level [OFF|ERROR|WARN|INFO|DEBUG|TRACE]
                                  Override log level globally [INFO]
  --log-filename LOG-PATH         File to redirect stderr logging to (overriding ocrd_logging.conf).

Options for information:
  -C, --show-resource RESNAME     Dump the content of processor resource RESNAME
  -L, --list-resources            List names of processor resources
  -J, --dump-json                 Dump tool description as JSON
  -D, --dump-module-dir           Show the 'module' resource location path for this processor
  -h, --help                      Show this message
  -V, --version                   Show version

Parameters:
   "level-of-operation" [string - "page"]
    PAGE XML hierarchy level to operate on
    Possible values: ["page", "region", "line", "word", "glyph"]
   "dpi" [number - 0]
    pixel density in dots per inch (overrides any meta-data in the
    images); disabled when zero
   "protect" [number - 0.0]
    avoid removing fg specks near larger fg components by up to this
    distance in pt
   "maxsize" [number - 1.0]
    maximum component size of (bg holes or fg specks) noise in pt

Testing

To install Python dependencies:

make deps-test

Which is the equivalent of:

pip install -r requirements_test.txt

To install this module, then do:

make test

Which is the equivalent of:

pytest tests

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ocrd_wrap-0.2.0.tar.gz (4.1 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ocrd_wrap-0.2.0-py3-none-any.whl (4.1 MB view details)

Uploaded Python 3

File details

Details for the file ocrd_wrap-0.2.0.tar.gz.

File metadata

  • Download URL: ocrd_wrap-0.2.0.tar.gz
  • Upload date:
  • Size: 4.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.17+

File hashes

Hashes for ocrd_wrap-0.2.0.tar.gz
Algorithm Hash digest
SHA256 0379b9209d7fa8606c3c8eafc7d6321c6bd4bcb9d7af1c3b18ffed99a58c5b0d
MD5 b62caf4ba4c1df9a58a54d2734e6fa8c
BLAKE2b-256 5d1c25ad1be5c25cd7bc0aa189d3cdffe8a330de22363d7d7f907195621891d0

See more details on using hashes here.

File details

Details for the file ocrd_wrap-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: ocrd_wrap-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 4.1 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.17+

File hashes

Hashes for ocrd_wrap-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 701d9d0cbd0c7d69aac288a9af031fa89ba4c32014aaf54918cbd751598c5fb9
MD5 994ab377d3171770c8a9149055f16b16
BLAKE2b-256 00a227b57ab00fe17ddc3c9a0ddd009afca600678b94029d292f15c0310b816b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page