VDL Tools

This repository contains a set of tools which are used at Vibrant Data Labs.

Tools

• Network Tools
• LinkedIn - Set of methods to interact with LinkedIn via CoreSignal API
• Download Process Images - Helper methods to download, convert and save images to S3 bucket
• Py2Mappr - Python wrapper for generating the OpenMappr player
• Scrape Enrich
• Tag2Network
• Shared Tools

Installation

For Users

Install the package from PyPI:

pip install vdl-tools

For Developers

1. Clone the repository:

git clone https://github.com/vibrant-data-labs/vdl-tools.git
cd vdl-tools

Hatch

2. Install Hatch:

pip install hatch

3. Create development environment:

hatch env create

4. (Optional) Install pre-commit hooks:

pip install pre-commit
pre-commit install

Option B: Using PyEnv + pip

If you prefer managing Python versions with PyEnv:

2. Install PyEnv (if not already installed):

# macOS
brew install pyenv

# Linux
curl https://pyenv.run | bash

3. Install Python 3.12:

pyenv install 3.12.7
pyenv local 3.12.7

4. Create a virtual environment:

pyenv virtualenv 3.12.7 <yourenvname>
pyenv activate <yourenvname>

5. Install the package in editable mode:

pip install -e ".[test,lint]"

6. (Optional) Install pre-commit hooks:

pip install pre-commit
pre-commit install

Development

This project uses Hatch for environment management, building, and publishing. However, you can also use standard Python tools if you prefer.

Quick Commands (Work with both Hatch and PyEnv)

# Run tests
make test

# Run tests with coverage
make test-cov

# Run linters
make lint

# Format code
make format

# Build package
make build

# Show all available commands
make help

Using Hatch Directly

# Enter development shell
hatch shell

# Run tests
hatch run test

# Run linters
hatch run lint:all

# Format code
hatch run lint:fmt

# Build package
hatch build

# Bump version
hatch version patch  # or minor/major

# Publish to PyPI
hatch publish

Adding a New Dependency

1. Add the package to pyproject.toml in the [project.dependencies] section
2. Update your environment:

hatch env prune  # Remove old environment
hatch env create  # Create new environment with updated dependencies

Or if using pip directly:

pip install -e .

Contributing

Please see CONTRIBUTING.md for detailed guidelines on:

• Development setup
• Code style and quality standards
• Testing requirements
• Pull request process
• Release workflow

CI/CD

This project uses GitHub Actions for continuous integration and deployment:

• CI: Runs tests, linters, and builds on every push and PR
• Dependabot: Automatically updates dependencies weekly
• Release: Automatically publishes to PyPI when a version tag is pushed

License

See LICENSE file for details.