Skip to main content

CAERP Split pdf

Split a large pdf file into several pieces.

Licence

This is a free software (GPLv3), see LICENCE.txt for licencing info.

What does it do

Provide a PDF file with all the salary sheets (or other type). In each page, you provide an ancode and a name (“name” of the employee and a specific key called “ancode”). This program splits the pdf into separated salary sheets.

It allows the people in charge of the salary sheet distribution to generate a file from their accounting software and to get an output with a structured directory structure.

CAERP matches that directory structure to distribute the salary sheets through its interface.

By extension, it’s possible to configure splitting for several type of data.

How

The original file is named <filetype>_<year>_<month>.pdf (e.g : salaire_2020_11.pdf).

  • The filetype is used to fetch the appropriate parsing configuration.

  • The generated pdfs are placed under a directory named <filetype>/<year>/<month>/.

Using the configuration the tool searches for an “ancode” and a “name” into the pdf pages. It generates files in the form <ancode>_<name>.pdf containing the pages with the associated keywords, into the appropriate directory.

Installation

Distro dependencies

Install the pdftotext command line tool.

Debian based distros

apt-get install poppler-utils virtualenvwrapper

Install from pypi

mkvirtualenv splitpdf
pip install caerp-split-pdf

Install from source

Setup a python virtual environment

mkvirtualenv splitpdf
pip install -r requirements.txt
pip install -e

Development mode

pip install -e .[dev]

Configuration

Main config file

Defaults to ~/.caerp_split_pdf_config.yaml Specifiable by use -c <configfile>

Format is yaml.

Configure :

  • Filetypes

  • For each filetype, a parsing strategy describing the datatype to extract, the way they are extracted and the name of the file to generate. (Several parser can be provided, here we have three parsers for ancode and two for name)

Example

salaire:
    filename_template: {data1}_{data2}_{data3}.pdf
    datatypes:
       data1:
          - type: regex
            regex: '[\s]{30,80}(?P<ancode>[A-Z]{4,12})\s+'
            from_line: 11
            to_line: 14
            strict: True
            regex_group_name: ancode

          - type: coordinates
            line: 12
            column: 40
            prefix: "     "
          - type: coordinates
            line: 13
            column: 40
            prefix: "     "

        data2:
          - type: regex
            regex: '\s+(M|Mme|Mlle)\s+(?P<name>[\w\s]+)(\n|$)'
            from_line: 6
            to_line: 9
            regex_group_name: name
            strict: True

          - type: coordinates
            line: 8
            column: 50

        data3:
          - type: coordinates
            line: 2
            column: 60
            prefix: "Section :"

Here we configured the parsing of files named salaire_<year>_<month>.pdf

For each page, we’ll extract three data and group all pages by matching data1/data2/data3 3-uples. The pages will be grouped in a file named with the filename_template.

Finders

For each data you can cumulate several “finders” to achieve data retrieval. Finders are used in the specified order.

There are two types of finders.

RegexDataFinder

Use a regex to find the data in the page’s string representation (pdftotext output)

regex

The regex to use for matching the data (ancode or name) we’re looking for.

strict (default to False)

If True a result is returned if only one item matches the regexp

from_line / to_line (optionnal)

Specify a range of lines to restrict the amount of data we look at The Finder will only look between from_line and to_line line numbers (starts with 1).

regex_group_name (optionnal)

If the regexp uses groups, specify the group matching the data

CoordinateDataFinder

Use coordinates to find the data in the page’s string representation (pdftotext output)

line

The line where to look for the data

column

Where to start in the line

prefix (optionnal)

A prefix after which we should find the data

Logging

This program supports advanced logging configuration with the following options.

verbosity: DEBUG

Available options are DEBUG/INFO/WARNING/ERROR/CRITICAL

Syslog logging

use_syslog: true

Mail logging

log_to_mail: true
mail:
    host: localhost
    from: admin@host.fr
    to: contact@example.net
    subject: '[%(hostname)s] Log of the pdf splitter'

That program is smart enough to use syslog if the config specifies it.

It logs to mail if the config contains log_to_mail: True

Examples

A full run:

caerp-split-pdf-run -c myconfig.yaml playground/salaires_2013_07.pdf

Test that the file is parseable on the 5 first pages:

caerp-split-pdf-run  -c myconfig.yaml playground/salaires_2013_07.pdf -r 5

Use -v for debug messages.

Use -h to get a complete overview of options.

Known problems

  • for payrolls

    cannot handle some PDF files, especially if there is no outline and the charset is ‘binary’. Check this with:

    file -i $filename.pdf

Known problems

When the logs returns messages like :

CRITICAL - No page of output!

If the end user generates his files with Sage “Édition pilotée”, the problem may be that the end user used “print to PDF” export instead of “Save to PDF”.

Metadata

Release files for caerp-split-pdf 2024.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for caerp-split-pdf 2024.1.1
File Size Uploaded
caerp_split_pdf-2024.1.1.tar.gz 30.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for caerp-split-pdf 2024.1.1
File Interpreter ABI Platform
caerp_split_pdf-2024.1.1-py3-none-any.whl Python 3 none any Details

Total release size: 60.6 kB

Release files / caerp_split_pdf-2024.1.1.tar.gz

Download URL caerp_split_pdf-2024.1.1.tar.gz
Size 30.9 kB
Tags Source
SHA-256 checksum
How to use checksums
57ef97d2078279a70eceabe8a538062720a20c68ae7d1b9338f8676c43cc0a05
BLAKE2b-256 checksum
How to use checksums
3792eaecf9c932ab4f66aa28d9cfd11199f20dd90f3517b648b6692086b2c8c7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.11.10

Release files / caerp_split_pdf-2024.1.1-py3-none-any.whl

Download URL caerp_split_pdf-2024.1.1-py3-none-any.whl
Size 29.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d4f370c1952d1eb33b1ad12cd9e6577ec036388ddd5be3f78dc7fde0a57309e2
BLAKE2b-256 checksum
How to use checksums
ac6d3ea5048a90c865dbaaaf00c68af7edb3af61b692e039bc37af741e62523d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.11.10

Release history Release notifications | RSS feed

This release

2024.1.1 This release

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page