geo-importer 0.0.1b1

Qt application to transform raw tables into clean geographic data.

Geo-Importer

A PySide6 application to upload statistics in Excel, CSV, or PDF formats. Extract and prepare tables, then assign geographic data (NUTS 0–3, LAU, and LOR).

---

Table of Contents

• Features

• Project Layout

• Quick Start

  • Run Locally

  • Build & Install

  • Publish to PyPI

• Documentation

• License

---

Features

Geo-Importer guides you step-by-step through:

1. Upload – load Excel, CSV or PDF files

2. PDF Area – select table regions in PDFs

3. Worksheet – choose an Excel sheet

4. Data Prep – clean and trim your data

5. Columns & Filter – pick columns and apply filters

6. Geo Filter – filter reference geodata (NUTS/LAU)

7. Mapping – auto-match statistics to regions

8. Manual Mapping – adjust any unmatched rows

9. Preview – view results on an interactive map

10. Export – save as CSV or GeoJSON

flowchart TB
    Upload[Upload]
    PDFArea[PDF Area]
    Worksheet[Worksheet]
    DataPrep[DataPrep]
    ColumnsFilter[Columns & Filter]
    GeoFilter[Geo Filter]
    Mapping[Mapping]
    ManualMapping[Manual Mapping]
    Preview[Preview]
    Export[Export]
    Upload <--> PDFArea
    Upload <--> Worksheet
    Upload <--> DataPrep
    PDFArea <--> DataPrep
    Worksheet <--> DataPrep
    DataPrep <--> ColumnsFilter
    ColumnsFilter <--> GeoFilter
    GeoFilter <--> Mapping
    Mapping <--> ManualMapping
    ManualMapping <--> Preview
    Preview <--> Export

A shared DataStore keeps intermediate results so each step stays decoupled. Core logic lives in src/core, and the Qt views are under src/views.

---

Project Layout

geo-importer/
├── src/               # Code packages
│   ├── core/          # Workflow logic & DataStore
│   ├── views/         # Qt UI for each step
│   ├── mapper/        # Matching algorithms & widgets
│   ├── models/        # Qt model classes
│   ├── geodata/       # Built-in reference CSVs
│   └── main.py        # Entry point
├── data/              # Sample tables for testing
├── static/            # Generated assets
├── tests/             # Unit tests
└── docs/              # MkDocs site

---

Quick Start

Clone the repo and enter the folder:

git clone https://github.com/frievoe97/geo-importer.git
cd geo-importer

Run Locally

1. Create a Python 3.12+ virtual env and install:

   python3.12 -m venv .venv
   source .venv/bin/activate    # Windows: .venv\Scripts\activate
   pip install .
   

2. Launch the GUI:

   python main.py
   

3. Run tests:

   pytest -v
   

4. Clean up:

   deactivate
   rm -rf .venv dist build geo_importer.egg-info
   

Build & Install Locally

1. Prepare env and install build tools:

   python3.12 -m venv .venv
   source .venv/bin/activate
   pip install .
   pip install --upgrade build
   

2. Build package:

   python -m build
   

3. Install wheel & run:

   pip install dist/geo_importer-*.whl
   geo-importer
   

4. Test and clean up as above.

Publish to PyPI

1. Build wheel and install twine:

   python3.12 -m venv .venv
   source .venv/bin/activate
   pip install --upgrade build twine
   python -m build
   

2. Upload:

   twine upload dist/*
   

3. Clean up .venv, dist/, build/.

---

Automated publishing via GitHub Actions

1. Add your PyPI API key as the PYPI_API_TOKEN secret in the GitHub repository
2. Bump the version in pyproject.toml and src/main.py using the helper script:

scripts/bump_version.py 0.0.1b1

3. Commit the changes on your release branch and create a tag that starts with v:

git tag v0.0.1b1
git push origin release --tags

Pushing the tag triggers .github/workflows/pypi.yml. It runs the test suite, builds the wheel and uploads the package to PyPI automatically.

Documentation

Run the helper script to generate API docs and serve them locally via MkDocs:

python generate_api_docs.py
mkdocs serve -f docs/mkdocs.yml

---

Continuous Integration / Tests

The project uses GitHub Actions to run formatting checks, linting, and tests on push and pull requests. The workflow supports Python 3.12 and 3.13:

name: Tests

on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install Qt dependencies
        run: sudo apt-get update && sudo apt-get install -y libegl1 libxkbfile1
      - name: Install tools
        run: pip install nox black ruff
      - name: Run black
        run: black src tests main.py --check
      - name: Run ruff
        run: ruff check .
      - name: Run tests
        env:
          QT_QPA_PLATFORM: offscreen
          QTWEBENGINE_DISABLE_SANDBOX: 1
        run: nox -s tests-${{ matrix.python-version }}

---

Code Formatting & Cleanup

You can automatically format and lint the codebase locally using Black and Ruff:

# Format all Python files
black src tests main.py

# Apply Ruff fixes (if any)
ruff check .

---

Documentation

Generate and serve docs:

python generate_api_docs.py
mkdocs serve -f docs/mkdocs.yml

Docs auto-publish to GitHub Pages on release tags.

---

License

Distributed under the MIT License. See LICENSE for details.