An example package demonstrating how to make one
Project description
This is an example demonstrating how to package your code for distribution on PyPI. It is one possible way to do it in the fall of 2022. I am not going to discuss other ways it could be done. The packaging ecosystem is complex. Talking about all the options just confuses things.
This repo isn’t meant to show all of the configuration in a typical project. For example, it has no tests, it has no docs, it doesn’t use linters or type checkers. Those are good things, you should look into them, but this repo is only about packaging a project for distribution, and leaves out those other good things so we can focus on the packaging.
The files in this repo have comments throughout, to help you understand what does what, so that you can create your own files with the parts and the details that you need.
How to use this repo
You have code you want to distribute. This repo is a model of how you can configure your project. Make a copy of the files here, take the parts that you need, and leave out the parts you don’t.
Decisions
You need to make a few decisions before getting started.
Project name
You have to pick a name for your project. In this repo, it’s “pkgsample”, but you will choose a different name. Your project name will serve a few different functions: it will be the module name for importing, the name of the repo, and also the name on PyPI. Anywhere you see “pkgsample,” you will change it to your project name.
Since the project name will be the name on PyPI, you should check to see if your desired name is available there before making a decision. Search for your name on PyPI to see if it’s already taken.
Version number
The version number of your package is in src/pkgsample/__init__.py as the __version__ variable. You will update this value when making a distribution. See the note in “Upload a test” below about using .devN suffixes initially.
Optional features
This layout will install your modules so that people can import them. There is also an extra possibility included:
You might have commands you need to install so that your users have new command-line tools. These parts are marked with COMMANDS:.
Keep an eye out as you look through the files to understand what you can omit if you don’t need this option.
Project details
There are other details that you might want to change, but you don’t have to:
License: this repo uses the Apache license, but you can choose a different one.
Minimum Python version: this repo required Python 3.9 or greater. Perhaps you need to support more versions (choose a lower requirement) or you want to use some newer Python features (choose a higher requirement).
Comments throughout the files, especially in pyproject.toml show where to make these choices, and link to resources that can help.
What’s in the repo
This repo is a demonstration of making a distributable package. You should examine the files here, decide which are right for you, and adapt them to your needs.
src/
This directory is where you put your code. It should have one subdirectory named for your project, and all of your work goes into that subdirectory.
This repo has a few small files in src/pkgsample just to have something to build and distribute. You will delete that directory and make a new subdirectory for your files.
pyproject.toml
This is the heart of the process. You will be making lots of changes in this file. The metadata about your project (name, description, author, and so on) are all specified here as well as many other details.
README.rst
(This file!) You will replace this file with your own README.rst describing your project, how to install it, how to use it, how to get help, the changelog, and so on.
Bits and bobs
.gitignore is a standard git file that keeps uninteresting files from being stored in git.
LICENSE.txt is the text of your chosen license.
Some files here are not needed to get a project distributed, but are useful for me in this repo, and you might find useful also:
Makefile: organizes the commands needed to work in the repo. The commands shown in this README are also available as targets in the Makefile. You can use make help to see the targets and what they do.
.editorconfig: specifies simple formatting rules that your editor can enforce.
Preparing your environment
Get yourself a Python 3.9 environment. Using a virtualenv is recommended but not required.
You’ll need a few tools to build the installable files (called distributions). They are specified in the dev-requirements.txt file. Install them with pip:
python -m pip install -r dev-requirements.txt # or: make tools
Make your changes
If you are ready, you can make all your project changes. If you want, you can also skip this step to try some of the next steps with the pkgsample repo as it is, to see what happens.
Copy the files from this repo, then make all your changes:
Put your project’s source files in the src/ directory. Make sure to remove all traces of the pkgsample code.
Update the pyproject.toml file with all your chosen details. The word “pkgsample” shouldn’t appear in it at all when you are done.
Completely re-write the README.rst file with the description and details of your project.
Install locally
You can do some preliminary testing of your project by installing it as an “editable install” in your current environment:
python -m pip install -e .
This will make your project importable in your current Python, and you can try running your code. Here’s an example using the pkgsample code:
% python Python 3.9.15 (main, Oct 24 2022, 17:23:01) >>> from pkgsample.add import add >>> add(17, 42) 59
If you are creating command-line commands, you will be able to run them now:
% pkgsample_add 1 2 3 4 5 Your numbers are: [1, 2, 3, 4, 5] They add up to: 15
By the way, an editable install like this is also a good way to do development, because Python will import directly from the files you are editing in your working tree, so it’s quick to make changes and see their effect.
Once you do this, you will start seeing __pycache__ directories near your code. These contain compiled bytecode files, named something.something.pyc. You can safely delete these files, and the .gitignore file will keep them from being stored in git.
Making distributions
You’re ready to make installable artifacts, called distributions.
Create the files
These commands will make the files and check them for correctness:
python -m build --sdist --wheel python -m twine check dist/* # or: make clean dist
If all went well, you will now have a dist/ directory with two files:
pkgsample-0.1.0-py3-none-any.whl pkgsample-0.1.0.tar.gz
The .whl file is a “wheel”. This is the preferred file format for distributions. The .tar.gz file is a “source distribution” (sdist), which is also easy to provide, and is preferred by some users.
If something went wrong, the twine check command reported errors to fix. It might help to see what is in the files, or you are just curious. You can examine the contents:
% tar tvfz dist/*.tar.gz drwxr-xr-x 0 user group 0 Nov 15 06:25 pkgsample-0.1.0/ -rw-r--r-- 0 user group 10177 Nov 15 05:07 pkgsample-0.1.0/LICENSE.txt -rw-r--r-- 0 user group 19222 Nov 15 06:25 pkgsample-0.1.0/PKG-INFO -rw-r--r-- 0 user group 6866 Nov 15 06:24 pkgsample-0.1.0/README.rst -rw-r--r-- 0 user group 2860 Nov 15 05:28 pkgsample-0.1.0/pyproject.toml -rw-r--r-- 0 user group 38 Nov 15 06:25 pkgsample-0.1.0/setup.cfg drwxr-xr-x 0 user group 0 Nov 15 06:25 pkgsample-0.1.0/src/ drwxr-xr-x 0 user group 0 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample/ -rw-r--r-- 0 user group 246 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample/__init__.py -rw-r--r-- 0 user group 166 Nov 15 05:07 pkgsample-0.1.0/src/pkgsample/add.py -rw-r--r-- 0 user group 410 Nov 15 05:07 pkgsample-0.1.0/src/pkgsample/add_cli.py drwxr-xr-x 0 user group 0 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample.egg-info/ -rw-r--r-- 0 user group 19222 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample.egg-info/PKG-INFO -rw-r--r-- 0 user group 333 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample.egg-info/SOURCES.txt -rw-r--r-- 0 user group 1 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample.egg-info/dependency_links.txt -rw-r--r-- 0 user group 113 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample.egg-info/entry_points.txt -rw-r--r-- 0 user group 5 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample.egg-info/requires.txt -rw-r--r-- 0 user group 10 Nov 15 06:25 pkgsample-0.1.0/src/pkgsample.egg-info/top_level.txt
The distribution includes all of your files, and also new supporting files made as part of the packaging process.
Testing the distributions
You can test installing the distribution files. Make a new virtualenv, and install directly from one of the files:
python -m pip install /path/to/pkgsample-0.1.0-py3-none-any.whl
Now you should be able to import and run your modules.
Cleaning up
The dist/ and build/ directories are created as part of this process. They won’t be stored in git and you can delete them whenever you want to clean up. The Makefile includes a target to do this for you:
make clean
Uploading to PyPI
The final step to making a complete installable package is to upload your distributions to the Python Package Index, PyPI. This is where pip finds packages to install.
(If you’ve been trying these steps with the pkgsample repo unchanged, you won’t be able to do this, because you can’t upload new pkgsample distributions.)
There are actually two PyPI instances: the real one at https://pypi.org, and also a test one at https://test.pypi.org/ for you to try out distributions before publishing them for real.
Create accounts
Register two PyPI accounts, at both https://pypi.org/account/register/ and https://test.pypi.org/account/register/. PyPI has a number of options to keep your account secure, but a simple username and password will get you started.
Upload a test
Upload your distributions to test.pypi.org. The twine tool does this:
python -m twine upload --verbose --repository testpypi dist/* # or: make clean dist test_pypi
You will be prompted for your username and password, the progress will be displayed, and finally you’ll get a URL to see what your published package page will look like:
% python -m twine upload --verbose --repository testpypi dist/* Uploading distributions to https://test.pypi.org/legacy/ INFO dist/pkgsample-0.1.1.dev0-py3-none-any.whl (10.4 KB) INFO dist/pkgsample-0.1.1.dev0.tar.gz (10.6 KB) INFO Querying keyring for username Enter your username: your_username INFO Querying keyring for password Enter your password: INFO username: your_username INFO password: <hidden> Uploading pkgsample-0.1.1.dev0-py3-none-any.whl 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26.6/26.6 kB • 00:00 • 13.1 MB/s INFO Response from https://test.pypi.org/legacy/: 200 OK Uploading pkgsample-0.1.1.dev0.tar.gz 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26.7/26.7 kB • 00:00 • 27.1 MB/s INFO Response from https://test.pypi.org/legacy/: 200 OK View at: https://test.pypi.org/project/pkgsample/0.1.1.dev0/
Note: Once you upload a file to PyPI (even the test server), you cannot fix something and re-upload it. You must increment the version number somehow in order to upload an updated file. While doing your first tests of your packaging, it’s a good idea to use a .devN suffix while you work on getting it right: __version__ = "0.1.0.dev0".
Check over your page. Make sure everything looks the way you want, including the metadata in the left-hand sidebar, and the formatting of the README. If you need to fix something, increment the .devN suffix of your version number, re-make the distributions, and upload them again.
Upload for real
Once your package looks right on the test PyPI server, you can upload your package for real!
Fix the version number to get rid of the .devN suffix, then make new distributions, and upload them to PyPI:
rm -fr build/ dist/ src/*.egg-info python -m build --sdist --wheel python -m twine check dist/* python -m twine upload --verbose dist/* # or: make clean dist pypi
Your package is available
Now anyone can install your package by using pip:
pip install pkgsample
You did it!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pkgsample-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5e0d267fad9eaed4576cc5189a528b44b68bf3946f850c0181e7e85cd3295d89 |
|
MD5 | 4b6bc6a47687b9e2f1f0ff16cb25c389 |
|
BLAKE2b-256 | c25f5c3543e1e7e4ca6dadc8cfb58448a0459b6d06dbead12541a4f3a419459c |