📋 Overview

[!NOTE] No Python or technical knowledge required! This tool is designed for everyone, regardless of programming experience.

Transform DUO's complex educational data into research-ready insights in minutes, not hours.

🚨 The Problem

DUO data comes as:

• Fixed-width ASCII files with no clear field separation
• Unstructured .txt metadata files
• Hours of manual processing required
• High risk of errors and data loss

✨ The Solution

Automated pipeline that delivers:

• ⚡ Fast - Process GB files in minutes using multiprocessing
• 🎯 Accurate - Automated validation prevents errors
• 🔒 Secure - Auto-anonymization of sensitive data (BSN, etc.)
• 📊 Ready - Clean CSV/Parquet output for immediate analysis

✨ Features

• Smart Extraction - Auto-finds field positions in messy .txt files
• Intelligent Matching - Links data files with metadata automatically
• Turbo Conversion - Multiprocessing for blazing speed
• Quality Validation - Comprehensive error checking
• Advanced Data Combination - YAML-configured joins with decoder files and reference tables
• Research-Ready Enrichment - 30+ calculated variables for academic analysis
• Switch Analysis - 27+ variables tracking student program switches
• Smart Case Handling - Supports snake_case, camelCase, PascalCase, and original formatting
• File Optimization - 60-80% size reduction via Parquet compression
• Privacy Protection - Cryptographic anonymization
• Zero Setup - uv handles all dependencies automatically

🔧 First Time Setup

[!WARNING] Do not skip these steps if this is your first time using this application. It will not work without them.

[!TIP] Save the repository in a Projects/CEDA folder on your main drive for quick access.

1. Get the Repository

Option A: Clone with Git (or Github Desktop)

git clone https://github.com/cedanl/1cijferho.git
cd 1cijferho

Option B: Download ZIP

After downloading extract the ZIP file and navigate into the folder.

2. Install

MacOS & Linux (Terminal)

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows (Powershell or Windows Terminal)

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Close and reopen your terminal after installation.

Verify installation

uv self update

See the installation documentation for details and alternative installation methods.

🚀 Running the Application

Ready to see the magic happen? Your 1CijferHO Tool is just one command away! ✨

First, get to the right spot:

Open a terminal in your 1cijferho folder - it's super easy!

• Windows: Shift + Right-click in folder → Open in Windows Terminal
• Mac: Right-click folder → New Terminal at Folder
• VS Code: Just click Terminal → New Terminal

Or simply navigate there:

cd path/to/1cijferho

Then, launch with a single command:

uv run streamlit run src/main.py

That's it! The app will automatically spring to life in your browser. If you've completed all the steps in the First Time Setup correctly, this is the only command you'll need going forward. 🎉

Pro Tip: Create a shortcut: .bat file (Windows) or .sh script (macOS/Linux) Pro Tip: Check out our architecture.md for technical details!

Happy analyzing! ✨📊📝

🛠️ Built With

🤲 Support

If you find this project helpful, please consider:

• ⭐ Starring the repo
• 🐛 Reporting bugs
• 💡 Suggesting features
• 💻 Contributing code

If you encounter any issues or need further assistance, please feel free to open an issue or contact a.sewnandan@hhs.nl | t.iwan@vu.nl

🙏 Acknowledgements

Special thanks to:

• Ash Sewnandan & Tomer Iwan for setting the foundation with a clean, user-friendly interface and robust architecture.
• CEDA & Npuls for making this project possible by providing valuable resources and support.

🫂 Contributors

Thank you to all the people who have already contributed to 1cijferho.

🚦 License