python-package-sync-tool 0.5.2a0

Small tool to sync package from different machines

PythonPackageSyncTool

PythonPackageSyncTool is a Python utility to "fix" requirements.txt.

It is used to make manipulation on requirements.txt that is produces by 'pip freeze'

Getting Help

QuickStart

First of all you should install alex_ber_utils (this dependency is used in setup.py).

pip3 install -U alex_ber_utils

Than:

pip3 install -U python-package-sync-tool

cd /opt/anaconda3/lib/python3.7/site-packages/alexber/reqsync/data/

Note: This is Path where you're actually install my utility, it can be different in your machine.

If you use venv it will look something like:

cd /opt/MyProject/venv/Lib/site-packages/alexber/reqsync

Alternatively you can create sctipt file for yourself, named, say, driver.py:

#!/usr/bin/python3

import alexber.reqsync.app as app

if __name__ == "__main__":
   app.main()

Than create file config.yml near your script (see data/config.yml) or provide all parameter using command line argruments. Use ':' in places where you should naturally write '==' (see explanation below).

Parammeters 'source' and 'destination' are required. You should also provide (requirements) file for 'source'.

mutual_exclusion has default value True.

Now, type

python3 -m alexber.reqsync.data --add=some_new_package:1.0.0

or if you're using script (driver.py) go the directory with the script and type

./driver.py --add=some_new_package:1.0.0

or if you install my tool to Anaconda/Python/venv that has it's bin folder is in the Path you can run

python_package_sync_tool --add=some_new_package:1.0.0

or alternativley

you can run

reqsync --add=some_new_package:1.0.0

This will run quick check whether package is not in remove list. If it is, the utility will fail. You can override this beahivor by supplying --mutual_exclusion=False.

Then, this will add some_new_package with version 1.0.0 to the requirements-dest.txt

Note:

Semicolomn and not equal sign is used here due to Python limitaion of usage of equal sign in the value in the command line.

You can specified multiple packages using comma delimiter.

You can specifiy path to your config file using --config_file.

It can be absolute or relative. If you're running using script (driver.py), that it can be relative to the directory whether you put your script. If you're running as the module (python3 -m), it can be relative to /opt/anaconda3/lib/python3.7/site-packages/alexber/reqsync/data/ (exact path can be different, see above).

You can supplied multiply packages by using comma:

python3 -m alexber.reqsync.data --add=some_new_package:1.0.0,another_new_package:2.0.0

or if you're using script (driver.py) go the directory with the script and type

./driver.py --add=some_new_package:1.0.0,another_new_package:2.0.0

Installing from Github

python3 -m pip install -U https://github.com/alex-ber/PythonPackageSyncTool/archive/master.zip

Optionally installing tests requirements.

python3 -m pip install -U https://github.com/alex-ber/PythonPackageSyncTool/archive/master.zip#egg=python-package-sync-tool[tests]

Or explicitly:

wget https://github.com/alex-ber/PythonPackageSyncTool/archive/master.zip -O master.zip; unzip master.zip; rm master.zip

And then installing from source (see below).

Installing from source

python3 -m pip install . # only installs "required"

python3 -m pip install .[test] # installs dependencies for tests

Using Docker

alexberkovich/AlexBerUtils:latest contains all AlexBerUtils dependencies. This Dockerfile is very simple, you can take relevant part for you and put them into your Dockerfile.

Alternatively, you can use it as base Docker image for your project and add/upgrade another dependencies as you need.

For example:

FROM alexberkovich/python_package_sync_tool:latest

COPY requirements.txt etc/requirements.txt
COPY requirements-tests.txt etc/requirements-tests.txt

RUN set -ex && \
    #latest pip,setuptools,wheel
    pip install --upgrade pip setuptools wheel && \
    pip install -r etc/requirements.txt  && \
    pip install -r etc/requirements-tests.txt 

CMD ["/bin/sh"]
#CMD tail -f /dev/null

where requirements.txt is requirements for your project.

From the directory with setup.py

python3 setup.py test #run all tests

pytest

Requirements

PythonPackageSyncTool requires the following modules.

• Python 3.7+

• PyYAML==5.1

• alex-ber-utils==0.2.5

Changelog

All notable changes to this project will be documented in this file.

#https://pypi.org/manage/project/python-package-sync-tool/releases/

[0.5.2a0] - 2020-11-17

Breaking change

• format of config.yml file changed. TODO: change documentation
• alexber.reqsync.utils.app_conf removed.

Changed

• Pytest version updated (also some tests dependecies changed)
• alex-ber-utils dependency updated.
• config.yml format changed to one that alex-ber-utils support for a long time.

See documentation here

My parser module [https://medium.com/analytics-vidhya/my-parser-module-429ed1457718]

My ymlparsers module [https://medium.com/analytics-vidhya/my-ymlparsers-module-88221edf16a6]

My major init_app_conf module [https://medium.com/analytics-vidhya/my-major-init-app-conf-module-1a5d9fb3998c]

• Logger is enabled, relative path is resolved correctly, and other little tweaks, see

Integrating Python’s logging and warnings packages [https://medium.com/analytics-vidhya/integrating-pythons-logging-and-warnings-packages-7ffd6f65e02d]

Making relative path to file to work [https://alex-ber.medium.com/making-relative-path-to-file-to-work-d5d0f1da67bf]

Added

• Optional Docker support added. TODO: change documentation

[0.5.1] - 2020-10-25

Added

• Support for Python 3.8

Changed

[0.4.3] - 2019-10-17

Changed

• anaconda-navigator and conda-build added to config.yml.

[0.4.2] - 2019-10-16

Changed

• navigator-updater added to config.yml.

[0.4.1] - 2019-05-30

Changed

• Bug fix: adding packages before all existing one works incorrect.
• Removing alexber.reqsync.utils.parsers. It was fully duplicated by alexber.utils.parsers. So, all usage was change to the latest (part of alex-ber-utils).

[0.3.1] - 2019-05-23

Changed

• Bug fix: adding packages before all existing one works incorrect.

Added

• Unit test for bug fix that check adding packages before all existing one.
• More detail assertion to integration tests.
• Unit-test for non-sorted requirements-src.
• Unit-test that check run with empty add and empty remvoe.
• Unit-test that check removing single package.
• Unit-test that check remove first package in requirements-src.
• Unit-test that check remove last package in requirements-src.
• Unit-test that check that empty lines in requirements-src are ignored.
• Unit-test that check correct usage of file input buffer and file output buffer.

[0.2.11] - 2019-05-22

Changed

• Dependency alex-ber-utils bumped up to 0.2.5.

[0.2.8] - 2019-05-22

Changed

• Dependency alex-ber-utils bumped up to 0.2.4.

[0.2.6] - 2019-05-22

Changed

• Dependency alex-ber-utils bumped up to 0.2.3.

[0.2.5] - 2019-05-22

Changed

• Fixed bug in setup.py, incorrect order between VERSION and UploadCommand (no tag was created on upload)
• Dependency alex-ber-utils bumped up to 0.2.2.

[0.2.4] - 2019-05-22

Changed

• Adding dependency alex-ber-utils 0.2.1 to README.md.

[0.2.3] - 2019-05-22

Changed

• Upgrading urllib3, SQLAlchemy, pycrypto dependenies beacause of volnurabilities issues.

[0.2.2] - 2019-05-22

Changed

• Fixing python-package-sync-tool.
• Creating alias reqsync to python-package-sync-tool.
• Some minour fixed.

[0.2.1] - 2019-05-22

Changed

• Changing dependency version of alex-ber-utils to 0.2.1.

[0.2.0] - 2019-05-22

Changed

• Only bumping up version.

[0.1.9] - 2019-05-22

Changed

• Added alex-ber-utils as dependency.
• Deleting old README-old.rst file.
• requirements-src.txt updated.
• Clarification added to README.md that alex_ber_utils should be installed first.
• formatting CHANGELOG.MD (minor fix)
• Added alternative of usage of python_package_sync_tool to README.md
• Fixing bug that --add is empty
• Factor out tests_data to seperate folder, use importlib.resources API.
• Updated README-old.rst

[0.1.8] - 2019-05-20

Changed

• README.md change, key '-U' added to pip3 install.

[0.1.7] - 2019-05-20

Removed

• Some project cleanup.

Changed

• CHANGELOG and REAMDE now use Markdown format.
• REAMDE totally rewritten.
• Fixing bugs in the core algorithm. Simplifying code.
• Fixing correct handling of package adding to the buttom of the list.

[0.1.6] - 2019-05-20

Added

• __init__.py file added to alexber.reqsync.data.

[0.1.5] - 2019-05-20

Added

• Small tool to sync package from different machines.