Package your python code into one zip file, even a virtual environment.
Project description
zipapps
zipapps
is a pure-python library, to package your python code into a single zip file with its dependencies.
Depends on PEP441 + zipapp + zipimport.
What is the .pyz
?
.pyz
to Python
is like .jar
to Java
.
They are both zip archive files which aggregate many packages and associated metadata and resources (text, images, etc.) into one file for distribution.
All you need is the Python Interpreter as the runtime environment.
So, what could zipapps
be?
packaging tool
compress your package folder into a zip file.
single-file virtual environment
a portable site-packages.
dependences installer
lazy install requirements while first running.
set of import-path
add many
venv.pyz
files to PYTHONPATH automatically.requirements.txt freezing toolkit
use
venv
to freeze your pip packages.
PS: The pyz extension can be modified to any character you want, such as
.py
and.zip
.
Install & Quick Start
pip install zipapps -U
1. Source code without dependences
- add your source code into the zip file, and set the entrypoint with
-m
-
python3 -m zipapps -c -a entry.py -m entry:main -o app.pyz
entry.py
could be a packagepackage_name
-m
format ispackage.module:function
-
- run app.pyz
-
python3 app.pyz
-
2. Source code with dependences
-
zipapps with requirements
-
python3 -m zipapps -c -a entry.py -m entry:main -o app.pyz bottle
bottle
is a pure-python lib which may not be unzipped-u AUTO
or-u=*
should be set for some.so/.pyd
libs- such as
psutil
- notice that the
-u=*
!=-u *
- such as
-
you don't need to install requirements at running
- ensure the compatibility of the system environment and python version
- or add
-d
oflazy install
mode to skip the compatibility problem- but comes a long-time
pip install
process at the first running.
- but comes a long-time
-
-
run app.pyz
-
python3 app.pyz
- libs with
.pyd/.so
caches will be unzipped to the./zipapps_cache/app
by-u AUTO
-
3. Virtual environments
-
zipapps with requirements
-
python3 -m zipapps -c -u AUTO -o venv.pyz -r requirements.txt
-
-
run entry.py with venv.pyz
-
python3 venv.pyz entry.py
- the cache will be unzipped to
./zipapps_cache/venv
for-u
is not null
-
4. Activate the .pyz
environment
import zipimport; zipimport.zipimporter('bottle.pyz').load_module("ensure_zipapps")
- automatically unzip cache, and add the path to sys.path
- it can be run multiple times
- if they are all pure-python code and no need to decompress
impory sys; sys.path.insert(0, "bottle.pyz")
Command line usage
Build Args
most common args:
-c
- to compress files, only for python3.7+.
-a xxx.py
- to add some files/folders into the zipped file.
-u=AUTO
- Recommended
- auto unzip the .pyd / .so files
- or
-u=*
to unzip all the files
- or
-r requirements.txt
- install requirements with
pip install
- install requirements with
-o my_app.pyz
- output the zipped file as given path
-m app.__main__:main
- entry point
-p /usr/bin/python3
- set the
shebang
line
- set the
-d
- Recommended
- requirements will be installed with
pip
while first running in the lazy install mode - zip file size will be very small, and the default unzip path is
SELF/zipapps_cache/
--clear-zipapps-cache
,-czc
- Clear the zipapps cache folder after running, but maybe failed for .pyd/.so files.
Details:
python3 -m zipapps -h
-h, --help
- show the simple doc
--includes, --add, -a
- The given paths will be copied to
cache_path
while packaging, which can be used while running. The path strings will be splited by ",".- such as
my_package_dir,my_module.py,my_config.json
- often used for libs not from
pypi
or some special config files
- such as
- the
output
arg ofzipapps.create_app
- The given paths will be copied to
--output, -o
- The path of the output file, defaults to
app.pyz
. - the
output
arg ofzipapps.create_app
- The path of the output file, defaults to
--python, -p
- The path of the Python interpreter which will be set as the
shebang line
, defaults toNone
.- with shebang
/usr/bin/python3
you can run app with./app.pyz
directly, no need forpython3 app.pyz
- with shebang
- the
interpreter
arg ofzipapps.create_app
- The path of the Python interpreter which will be set as the
--main, -m
- The entry point function of the application, the
valid format
is:package.module:function
package.module
module:function
package
- the
main
arg ofzipapps.create_app
- WARNING: If the
--main
arg is set,python3 app.pyz
will not be able to used as venv likepython3 app.pyz xxx.py
- The entry point function of the application, the
--compress, -c
Boolean
value, compress files with the deflate method or not.- the
compressed
arg ofzipapps.create_app
--unzip, -u
- The names which need to be unzipped while running, splited by ","
without ext
, such asbottle,aiohttp
, or the complete path likebin/bottle.py,temp.py
. For.so/.pyd
files(which can not be loaded by zipimport), or packages with operations of static files.- if unzip is set to "*", then will unzip all files and folders.
- if unzip is set to AUTO, then will add the
.pyd
and.so
files automatically.
- Can be overwrite with environment variable
ZIPAPPS_UNZIP
- the
unzip
arg ofzipapps.create_app
- The names which need to be unzipped while running, splited by ","
--unzip-exclude, -ue
- The opposite of
--unzip
/-u
which will not be unzipped, should be used with--unzip
/-u
. - Can be overwrite with environment variable
ZIPAPPS_UNZIP_EXCLUDE
- The opposite of
--unzip-path, -up
3. Ifunzip
arg is not null, cache files will be unzipped to the given path while running. Defaults tozipapps_cache
, support some internal variables as runtime args:$TEMP/$HOME/$SELF/$PID/$CWD
as prefix, for example$HOME/zipapps_cache
TEMP
meanstempfile.gettempdir()
HOME
meansPath.home()
SELF
means.pyz
file path.$PID
meansos.getpid()
$CWD
meansPath.cwd()
- And you can also overwrite it with environment variables:
ZIPAPPS_CACHE
orUNZIP_PATH
- the
unzip_path
arg ofzipapps.create_app
-cc, --pyc, --compile, --compiled
- Compile .py to .pyc for fast import, but zipapp does not work unless you unzip it(so NOT very useful).
- the
compiled
arg ofzipapps.create_app
--cache-path, --source-dir, -cp
- The cache path of zipapps to store site-packages and
includes
files. If not set, will create and clean-up in TEMP dir automately. - the
cache_path
arg ofzipapps.create_app
- The cache path of zipapps to store site-packages and
--shell, -s
- Only while
main
is not set, used for shell=True insubprocess.run
.- very rarely used, because extra sub-process is not welcome
- the
shell
arg ofzipapps.create_app
- Only while
--main-shell, -ss
- Only for
main
is not null, callmain
withsubprocess.Popen
:python -c "import a.b;a.b.c()"
. This is used forpsutil
ImportError of DLL load.- very rarely used too
- the
main_shell
arg ofzipapps.create_app
- Only for
--strict-python-path, -spp
Boolean
value. Ignore global PYTHONPATH, only usezipapps_cache
andapp.pyz
.- the
ignore_system_python_path
arg ofzipapps.create_app
-b, --build-id
- The string to skip duplicate builds, it can be the paths of files/folders which splited by ",", then the modify time will be used as build_id. If build_id contains
*
, will useglob
function to get paths. For example, you can set requirements.txt as your build_id bypython3 -m zipapps -b requirements.txt -r requirements.txt
when you use pyz as venv.- very rarely used too too
- the
build_id
arg ofzipapps.create_app
- The string to skip duplicate builds, it can be the paths of files/folders which splited by ",", then the modify time will be used as build_id. If build_id contains
--zipapps, --env-paths
- Default
--zipapps
arg if it is not given while running. Support $TEMP/$HOME/$SELF/$PID/$CWD prefix.
- Default
--delay, -d, --lazy-pip, --lazy-install, --lazy-pip-install
- Install packages with pip while first running, which means requirements will not be install into pyz file.
--ensure-pip
- Add the ensurepip package to your pyz file, works for embed-python(windows) or other python versions without
pip
installed butlazy-install
mode is enabled. [EXPERIMENTAL]
- Add the ensurepip package to your pyz file, works for embed-python(windows) or other python versions without
--layer-mode
- Layer mode for the
serverless
use case,__main__.py / ensure_zipapps.py / activate_zipapps.py
files will not be set in this mode. --layer-mode-prefix
- Only work while
--layer-mode
is set, will move the files in the given prefix folder.
- Only work while
- Layer mode for the
--clear-zipapps-cache, -czc
- Clear the zipapps cache folder after running, but maybe failed for .pyd/.so files.
--clear-zipapps-self, -czs
- Clear the zipapps pyz file self after running.
--chmod
- os.chmod(int(chmod, 8)) for unzip files with
--chmod=777
- unix-like system only
- os.chmod(int(chmod, 8)) for unzip files with
--dump-config
- Dump zipapps build args into JSON string.
- A file path needed and
-
means stdout.
- A file path needed and
- Dump zipapps build args into JSON string.
--load-config
- Load zipapps build args from a JSON file.
--freeze-reqs
- Freeze package versions of pip args with venv, output to the given file path.
-
equals tostdout
- logs will be redirect to
stderr
- Based on
pip
+venv
- work folder is
tempfile.TemporaryDirectory
, prefix='zipapps_'
- work folder is
- Freeze package versions of pip args with venv, output to the given file path.
- all the other (or
unknown
) args will be used bypip install
- such as
-r requirements.txt
- such as
bottle aiohttp
- the
pip_args
arg ofzipapps.create_app
- such as
Runtime Args
available args while the
.pyz
is running
--zipapps
- including some other pyz into PYTHONPATH
- often be used as
multiple venv combination
- for example
- build
python3 -m zipapps -o six.pyz six
python3 -m zipapps -o psutil.pyz -u AUTO psutil
python3 -m zipapps -o bottle.pyz bottle
- run
python3 six.pyz --zipapps=psutil.pyz,bottle.pyz -c "import psutil, bottle"
- build
--activate-zipapps
/--ensure-zipapps
- to ensure the zipapps_cache folder but do nothing
- use environment variables to reset build args while running, and custom it with
ENV_ALIAS
arg- the upper names are environment variables
-
'unzip': 'ZIPAPPS_UNZIP', 'unzip_exclude': 'ZIPAPPS_UNZIP_EXCLUDE', 'unzip_path': 'ZIPAPPS_CACHE', 'ignore_system_python_path': 'STRICT_PYTHON_PATH', 'python_version_slice': 'PYTHON_VERSION_SLICE', 'clear_zipapps_cache': 'CLEAR_ZIPAPPS_CACHE', 'clear_zipapps_self': 'CLEAR_ZIPAPPS_SELF', 'unzip_chmod': 'UNZIP_CHMOD',
When to Use it?
-
Package your code(package or model) into one zipped file.
-
Pure python code without any 3rd-party dependencies.
-
Python code with 3rd-party dependencies installed together.
-
Python code with requirements be not installed until running.
-
[Recommended] The
lazy install
mode by the arg-d
. -
The dependences will be installed at the first running(only once).
-
This is
cross-platform
andcross-python-version
for their independent cache paths.
-
-
-
Run with python interpreter from the
venv
path.- which means the requirements(need to be unzipped) will be installed to the
venv
folder, not inpyz
file. - build your package into one
pyz
file with-m package.module:function -p /venv/bin/python
. - run the
pyz
file with/venv/bin/python app.pyz
or./app.pyz
.
- which means the requirements(need to be unzipped) will be installed to the
-
Rapidly deploy stateless applications
Serverless
- with a ready layer
Docker
- only built once for a Python environment
wget http://S3_PATH/app.pyz && python3 app.pyz
Spot instances
- AWS or Aliyun
Hadoop-Streaming
- mapper & reducer scripts
-
Simple deployment towards different servers with
jenkins
, or other CI/CD tools.- Easy to uploads a clean
standalone
zip file.
- Easy to uploads a clean
-
Share the
app.pyz
to your friends-
someone has python installed
-
or embedded python
-
or use nuitka
-
-
-
Use as a requirements zip path, or some
venv
usages.-
inner the code
-
import sys;sys.path.insert(0, 'app.pyz')
-
with .so/.pyd?
import activate_zipapps
-
-
or run it directly
python3 app.pyz script.py
-
-
Other usages need to be found, and enjoy yourself
Advanced Usage
Inspired by shiv and with the differences.
zipapps
may not always create the cache folder.pure-python
dependencies can be import directly without unzipped.- unless your package includes the C language-based libraries or dynamic modules(
.so/.pyd
), such aspsutil
.- use the arg
-u=AUTO
or-u=*
- use the arg
- The default cache path is
./zipapps_cache
instead of theHOME
path. - One app name create cache folder only once.
- You can install requirements with
pip
while first running(not packaging) bylazy install
mode- reducing your
.pyz
file size.- cross-platform and cross-python-version
python -m zipapps -c -d -a your-package -r requirements.txt
- reducing your
- Mix multiple
venv.pyz
files together.python -m app.pyz --zipapps=bottle.pyz,requests.pyz
- Freeze your requirements.txt
python -m zipapps --freeze-reqs=requirements.lock -r requirements.txt
Click here read more
1. Package your script file with only built-ins functions.
1.1 Code of script.py
print('ok')
1.2 build the app.pyz
python3 -m zipapps -c -a script.py -m script -p /usr/bin/python3
1.3 run this app
python3 app.pyz
or
./app.pyz
ouput
ok
Details:
-c:
compress files, 7.6KB => 3.3KB
-a:
add the script file to app.pyz, so you can use it while running
-m:
set the entry_point, also can be set as `-m script:main` as long as you has the main function
-p:
set the shebang line, so you can use `./app.pyz` instead of `python3 app.pyz`
2. Package your package folder into one zip file.
2.1 The package example
and the code of __main__.py
└── example
├── __init__.py
└── __main__.py
def main():
print('ok')
if __name__ == "__main__":
main()
2.2 build the example.pyz
python3 -m zipapps -c -a example -o example.pyz -m example.main:main -p /usr/bin/python3
2.3 Run with python3 example.pyz
or ./example.pyz
output
ok
Details:
-m:
set the entry_point with format like `package.model:function`
-o:
set the output file path, you can set it some other paths like `/home/centos/example.abc` and run with `python3 /home/centos/example.abc`
no more new args.
3. Package your code with requirements (bottle).
3.1 The package example
and the code of __main__.py
└── example
├── __init__.py
└── __main__.py
def main():
import bottle
print(bottle.__version__)
if __name__ == "__main__":
main()
3.2 build the example.pyz
with requirements installed
python3 -m zipapps -c -a example -o example.pyz -m example.main:main bottle
3.3 Run with python3 example.pyz
or ./example.pyz
Output
0.12.19
Details:
bottle:
all the unhandled args (like `bottle`) will be used to `pip install`, so you can write `bottle` in requirements.txt and use like `-r requirements.txt`
WARNING: if the requirements have .pyd/.so
files, you should unzip them while running, and the pure python libs like bottle
or requests
no need to unzip anything. Read the 4th paragraph for more info.
4. Package your code with the requirements which includes .pyd/.so
files.
4.1 The package example
and the code of __main__.py
└── example
├── __init__.py
└── __main__.py
def main():
import psutil
print(psutil.__version__)
if __name__ == "__main__":
main()
4.2 build the example.pyz
with requirements installed
python3 -m zipapps -c -a example -o example.pyz -m example.main:main -u AUTO -up TEMP/cache psutil
4.3 Run with python3 example.pyz
or ./example.pyz
Output
5.8.0
Details:
-u:
means the file or folder names you want to unzip while running. Here is the `AUTO`, will unzip the psutil package because of its .pyd or .so files included.
-up:
the unzip path of cache folder. $TEMP/$HOME/$SELF/$PID/$CWD are the built-in runtime args, but for the stable usage you can ignore this arg then use `./zipapps_cache/example`. The cache will be refreshed while you rebuild this pyz.
WARNING: unzip path can be overwrited by export ZIPAPPS_CACHE=./xxx
or export UNZIP_PATH=./xxx
while running.
5. Package the requirements like a virtual environment without entry_point.
5.1 Code of script.py
import bottle, requests
print(bottle.__version__, requests.__version__)
5.2 build the venv.pyz
python3 -m zipapps -c -o venv.pyz -p /usr/bin/python3 bottle requests
5.3 Some use cases
5.3.1 use the venv.pyz
like a middleware
python3 venv.pyz script.py
5.3.2 use the venv.pyz
like the interpreter
./venv.pyz script.py
even like is also valid
python3 venv.pyz -c "import bottle,requests;print(bottle.__version__, requests.__version__)"
5.3.3 use the bottle
lib of venv.pyz
in your normal code
import sys
sys.path.append('path_to_venv/venv.pyz')
import bottle
print(bottle.__file__)
5.3.3 use the pstuil
lib of venv.pyz
in other normal code
python3 -m zipapps -c -o venv.pyz psutil
import sys
sys.path.append('path_to_venv/psutil_venv.pyz')
# need to activate the unzip action, there are 2 ways for the `.so/.pyd` libs
# 1. use the default activate name
import activate_zipapps
# 2. use a distinct lib name with file name for another venv file
sys.path.append('path_to_venv/lxml_venv.pyz')
import ensure_lxml_venv
# 2.1 or the old name
# import ensure_zipapps_lxml_venv
import psutil
import lxml
print(psutil.__file__)
print(lxml.__file__)
Output
0.12.19 2.25.1
Details:
No `-m` arg here, then the pyz file will do like an interpreter which contains the installed requirements.
So you can use it like this:
> python3 venv.pyz
>>> import bottle
>>> bottle.__file__
6. Using multiple venv pyz files for your pyz model.
6.1 Here is script.py
again
import bottle, requests
print(bottle.__version__, requests.__version__)
6.2 Build the script.pyz
python3 -m zipapps -c -o script.pyz -a script.py -m script requests
6.3 Build the requests.pyz
and bottle.pyz
respectively
python3 -m zipapps -c -o requests.pyz requests
python3 -m zipapps -c -o bottle.pyz bottle
6.4 And now run the script.pyz
with two requirements
python3 script.pyz --zipapps=bottle.pyz,requests.pyz
Output
0.12.19 2.25.1
Details:
--zipapps:
This arg will help you run some zipapp with given venv.pyz files, the paths is separated by commas.
7. Package your code with lazy install mode, for a attempt of cross-platform.
7.1 Here is script.py
again
import six
print(six.__file__)
7.2 Build the script.pyz
, this is very fast without downloading and installing 3rd packages.
python3 -m zipapps -c -o script.pyz -a script.py -m script -d six
7.3 Run this .pyz
file, and the requirements will be installed while first running.
python3 script.pyz
Output
Looking in indexes: https://pypi.xxx.com/simple/
Collecting six
Using cached https://xxx/packages/ee/ff/xxxx/six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six
Successfully installed six-1.15.0
/tmp/zipapps_cache/script/_zipapps_lazy_pip/3.8/Linux/six.py
Details:
-d:
Lazy install mode is useful for distributing your cross-platform apps.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for zipapps-2024.6.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b5866fe8dd47068415a86bbf0199a4fe584e95eea38e86ff2c944011feff6c6d |
|
MD5 | 0e546263484bd3005106521fd34e016f |
|
BLAKE2b-256 | f508c9b193e8e3b3646e77e1917900bfbe90344da73d5fa44ea8ce976ee96080 |