pynary 0.1.0

Pynary - Python Binary provides tools to build binary python packages and upload to pypi

pynary

Converts python code to binary format

This package converts your python code to binaries for the platforms that you compile it on. It uses cython. It also allows you to push your binary package to pypi with command line parameters.

##Installation instructions:

pip install pynary

Getting Help

pynary provides comprehensive help for all commands:

# Main help
python -m pynary --help

# Build command help
python -m pynary.build --help

# Upload command help
python -m pynary.pypipush --help
# or
python pypipush.py --help

The help text includes examples, usage patterns, and authentication information.

##Requirements: Ideally your package directory structure should look like below:

your_package_name/
|-- your_package_name/
|   |-- some_module/
|       |-- module3.py
|   |-- __init__.py
|   |-- module1.py
|   |-- module2.py
|-- setup.py
|-- README.md
|-- LICENSE

Make sure some_module does not have init.py in it. your library should have init.py. You need to have a file named setup.py. A sample setup.py looks like below:

import setuptools
from setuptools.dist import Distribution

with open("README.md", "r") as fh:
    long_description = fh.read()

class BinaryDistribution(Distribution):
    """Distribution which always forces a binary package with platform name"""
    def has_ext_modules(foo):
        return True

setuptools.setup(
    name="<your_package_name>",
    version="<version_number>",
    author="<Author name>",
    author_email="<author's email address>",
    description="<short description>",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="<URL of your website if any>",
    packages=['<your_package_name>'],
    include_package_data=True,
    install_requires=[
        'cryptography',
        'numpy',
        'mysqleasy',
        'pandas',
        'zmq',
    ],
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
        # Platform-specific classifiers (pip uses these to select the correct wheel)
        "Operating System :: Microsoft :: Windows",
        "Operating System :: POSIX :: Linux",
        "Operating System :: MacOS",
    ],
    distclass=BinaryDistribution,
)

Once you have the setup.py created then you could just use the commandline to convert the python code and create a binary package.

##Sample usage

Building Your Package

Basic Build:

python -m pynary -n <your_package_name>

Build Options:

# Clean build (removes existing build directory)
python -m pynary -n <package_name> -f
# or
python -m pynary -n <package_name> --fresh

# Use new build system (python -m build)
python -m pynary -n <package_name> -m

# Verbose output
python -m pynary -n <package_name> -v

# Explicitly specify build command
python -m pynary -c build -n <package_name>
# or short form:
python -m pynary -c b -n <package_name>

Incremental Builds:

• By default, pynary performs incremental builds - only recompiles changed files
• Compiled binaries (.pyd/.so) are preserved between builds
• Cython build cache is stored in build/cython/ directory
• Use -c or --clean flag for a fresh build

Important Security Note:

• Most Python source files (.py) are compiled to binary format (.pyd/.so)
• However, __init__.py and __main__.py files remain as source code in the built package
• These files are required for Python package structure but will be visible to users
• If these files contain sensitive code, consider moving it to compiled modules
• A warning will be displayed during build if these files are present

Platform-Specific Wheels:

• pynary automatically builds platform-specific wheels (.whl files)
• Wheels are tagged with platform information (e.g., win_amd64, linux_x86_64, macosx_10_9_x86_64)
• When users run pip install yourpackage, pip automatically selects the correct wheel for their platform
• This ensures each platform gets the correct binary format

Uploading to PyPI:

You can upload using any of these equivalent methods:

# Method 1: Direct script
python pypipush.py -n mypackage -d test

# Method 2: Via module interface (recommended)
python -m pynary -c pypipush -n mypackage -d test
# or short form:
python -m pynary -c p -n mypackage -d test

# Method 3: Direct module import
python -m pynary.pypipush -n mypackage -d test

All three methods are equivalent and support all the same options.

Enhanced PyPI Upload Options

Basic Usage:

# Upload to TestPyPI (all methods are equivalent)
python pypipush.py -n mypackage -d test
python -m pynary -c pypipush -n mypackage -d test
python -m pynary -c p -n mypackage -d test

# Upload to production PyPI
python pypipush.py -n mypackage -d production
python -m pynary -c pypipush -n mypackage -d production

# Dry run (validate without uploading)
python pypipush.py -n mypackage -d production --dry-run
python -m pynary -c p -n mypackage -d production --dry-run

Using API Token (Recommended):

API tokens are the recommended authentication method for PyPI because they:

• Are more secure than passwords (can be revoked without changing your password)
• Can be scoped to specific projects
• Don't require entering credentials interactively
• Work better in CI/CD environments

How to Get a PyPI API Token:

1. Go to PyPI Account Settings:

   • Visit https://pypi.org/account/login/
   • Log in with your PyPI account

2. Create an API Token:

   • Go to https://pypi.org/manage/account/token/
   • Click "Add API token"
   • Enter a name for the token (e.g., "pynary-upload-token")
   • Choose scope:
     • Entire account: Token works for all your projects
     • Project specific: Token only works for one project (more secure)
   • Click "Add token"
   • Copy the token immediately - it will only be shown once!
     • Format: pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

3. Use the Token:

# Using command line argument
python pypipush.py -n mypackage -d production --token pypi-xxxxxxxxxxxxx

# Using environment variable (recommended for security)
export TWINE_TOKEN=pypi-xxxxxxxxxxxxx
python pypipush.py -n mypackage -d production

# On Windows (PowerShell)
$env:TWINE_TOKEN="pypi-xxxxxxxxxxxxx"
python pypipush.py -n mypackage -d production

# On Windows (CMD)
set TWINE_TOKEN=pypi-xxxxxxxxxxxxx
python pypipush.py -n mypackage -d production

Security Best Practices:

• Never commit tokens to version control
• Use environment variables instead of command-line arguments when possible
• Use project-specific tokens when you only need to upload one package
• Revoke tokens if they're compromised or no longer needed

Using Username/Password:

# Using command line arguments
python pypipush.py -n mypackage -d production --username myuser --password mypass

# Using environment variables
export TWINE_USERNAME=myuser
export TWINE_PASSWORD=mypass
python pypipush.py -n mypackage -d production

Custom Repository:

# Upload to a custom PyPI-compatible repository
python pypipush.py -n mypackage -r https://custom-pypi.example.com/legacy/

Additional Options:

• --skip-existing: Skip files that already exist (default: True for production)
• --verbose: Enable verbose output
• --dry-run: Validate package without uploading

##Using your package

Once your package is built and uploaded to PyPI, users can install it with:

pip install your_package_name

Platform-Specific Installation:

• When users run pip install, pip automatically detects their platform
• pip selects the correct wheel (Windows .pyd, Linux/Mac .so) for their system
• No additional configuration needed - pip handles platform selection automatically

Important Note on Dependencies:

• Your package built with pynary will install correctly
• However, if your package has dependencies that need compilation (like blis, numpy, etc.), users may encounter build errors
• This happens when dependencies don't have pre-built wheels for the user's platform/Python version
• Solution: Ensure your dependencies are available as pre-built wheels, or document that users need build tools (C compiler, etc.)
• Common dependencies with pre-built wheels: numpy, pandas, scipy (usually available)
• Dependencies that may need compilation: blis, custom C extensions, etc.

Using the installed package:

from your_package_name import module1
from your_package_name import module3 

Cross-Platform Building

Important Limitations:

• Cython compiles to platform-specific binaries (.pyd on Windows, .so on Linux/Mac)
• You cannot cross-compile from Windows to macOS - macOS binaries require macOS toolchains
• You cannot cross-compile from Windows to Linux easily - requires Docker or WSL

Practical Solutions:

Option 1: Build on Each Platform

Build on the target platform:

• Windows: Run python -m pynary -n <package_name> on Windows
• Linux: Run the same command on a Linux machine or WSL
• macOS: Run the same command on a macOS machine

Option 2: Use Docker (Linux builds from Windows)

If you have Docker installed, you can build Linux binaries from Windows:

# Build for Linux using Docker
docker run --rm -v "%cd%":/workspace -w /workspace python:3.9 bash -c "pip install pynary && python -m pynary -n <package_name>"

Option 3: Use CI/CD (Recommended)

Use GitHub Actions or similar CI/CD services to build for all platforms automatically:

• GitHub Actions provides free runners for Windows, Linux, and macOS
• Quick Setup: Copy the workflow files from .github/workflows/ in this repository
• See .github/workflows/README.md for detailed setup instructions
• Two workflows are provided:
  • build-and-publish.yml: Builds and publishes to PyPI on version tags
  • build-only.yml: Builds and tests without publishing (for CI)

Option 4: Use Virtual Machines

• Use VMs for Linux builds
• Use macOS VMs (requires macOS license) for macOS builds

Note: The most practical approach is to use CI/CD services like GitHub Actions, which can build for all three platforms automatically without requiring local setup for each platform.