data-tasks

Small library to run data related scripts

Setting up Your data-tasks Development Environment

First you'll need to install:

• Git. On Ubuntu: sudo apt install git, on macOS: brew install git.
• GNU Make. This is probably already installed, run make --version to check.
• pyenv. Follow the instructions in pyenv's README to install it. The Homebrew method works best on macOS. The Basic GitHub Checkout method works best on Ubuntu. You don't need to set up pyenv's shell integration ("shims"), you can use pyenv without shims.
• Docker Desktop. On Ubuntu follow Install on Ubuntu. On macOS follow Install on Mac.

Then to set up your development environment:

git clone https://github.com/hypothesis/data-tasks.git
cd data-tasks
make services
make help

Releasing a New Version of the Project

1. First, to get PyPI publishing working you need to go to: https://github.com/organizations/hypothesis/settings/secrets/actions/PYPI_TOKEN and add data-tasks to the PYPI_TOKEN secret's selected repositories.

2. Now that the data-tasks project has access to the PYPI_TOKEN secret you can release a new version by just creating a new GitHub release. Publishing a new GitHub release will automatically trigger a GitHub Actions workflow that will build the new version of your Python package and upload it to https://pypi.org/project/data-tasks.

Changing the Project's Python Versions

To change what versions of Python the project uses:

1. Change the Python versions in the cookiecutter.json file. For example:

   "python_versions": "3.10.4, 3.9.12",
   

2. Re-run the cookiecutter template:

   make template
   

3. Commit everything to git and send a pull request

Changing the Project's Python Dependencies

To change the production dependencies in the setup.cfg file:

1. Change the dependencies in the .cookiecutter/includes/setuptools/install_requires file. If this file doesn't exist yet create it and add some dependencies to it. For example:

   pyramid
   sqlalchemy
   celery
   

2. Re-run the cookiecutter template:

   make template
   

3. Commit everything to git and send a pull request

To change the project's formatting, linting and test dependencies:

1. Change the dependencies in the .cookiecutter/includes/tox/deps file. If this file doesn't exist yet create it and add some dependencies to it. Use tox's factor-conditional settings to limit which environment(s) each dependency is used in. For example:

   lint: flake8,
   format: autopep8,
   lint,tests: pytest-faker,
   

2. Re-run the cookiecutter template:

   make template
   

3. Commit everything to git and send a pull request