# core-ftp

This project/library provides a comprehensive set of common components and interfaces designed to facilitate and streamline FTP connections, ensuring efficient communication and data transfer…

Installation

Install from PyPI using pip:

pip install core-ftp
uv pip install core-ftp  # Or using UV...

Features

SFTP Client

• Simplified SFTP connection management with context manager support

• Multiple authentication methods:

  • Password-based authentication

  • SSH private key authentication with optional passphrase

  • Customizable transport and connection parameters

• Comprehensive file operations:

  • List files and directories with detailed attributes

  • Upload files from local filesystem or file-like objects

  • Download files to local filesystem

  • Delete files and directories

  • Change and retrieve current working directory

• Built-in error handling with custom exceptions

• Automatic resource cleanup

ETL Support

• Abstract base class for SFTP-based ETL tasks

• Seamless integration with core-etl framework

• File filtering capabilities:

  • Filter by file extension

  • Filter by filename prefix

  • Time-based filtering with delay options

• Optional automatic file cleanup after successful processing

Examples

Password authentication

from core_ftp.clients.sftp import SftpClient, SftpConnectionConfig

config = SftpConnectionConfig(host="test.rebex.net", port=22, user="demo", password="password")

with SftpClient(config) as client:
    for name, attr in client.list_files("/"):
        print(name, attr)

SSH key authentication

from core_ftp.clients.sftp import SftpClient, SftpConnectionConfig

config = SftpConnectionConfig(host="localhost", port=23, user="foo", private_key_path="/path/to/id_rsa")

with SftpClient(config) as client:
    client.download_file("remote/file.txt", "/local/file.txt")

Upload a file

from core_ftp.clients.sftp import SftpClient, SftpConnectionConfig

config = SftpConnectionConfig(host="localhost", user="foo", password="pass")

with SftpClient(config) as client:
    client.chdir("/upload")
    client.upload_file("/local/report.csv", "report.csv", confirm=True)

ETL task (file-based processing)

from core_ftp.clients.sftp import SftpConnectionConfig
from core_ftp.etls.file_based import IBaseEtlFromFtpFile, SftpFileConfig

class DataProcessor(IBaseEtlFromFtpFile):
    @classmethod
    def registered_name(cls) -> str:
        return "DataProcessor"

    def process_file(self, path: str, **kwargs):
        print(f"Processing {path}")

DataProcessor(
    connection=SftpConnectionConfig(host="data.server.com", user="admin", password="secret"),
    file_config=SftpFileConfig(path="/data/csv", file_ext=".csv", delete_file_on_success=True),
).execute()

Local SFTP server using Docker

Functional tests require two local SFTP servers running via Docker (atmoz/sftp): one for password authentication (port 22) and one for SSH key authentication (port 23).

1. Password authentication (port 22)

docker run \
  --mount type=bind,source="$(pwd)/tests/resources/upload",target=/home/foo/upload \
  -p 22:22 -d atmoz/sftp foo:pass:::upload

2. SSH key authentication (port 23)

docker run \
  --mount type=bind,source="$(pwd)/tests/resources/ssh_keys/id_rsa.pub",target=/home/foo/.ssh/keys/id_rsa.pub,readonly \
  --mount type=bind,source="$(pwd)/tests/resources/upload",target=/home/foo/upload \
  -p 23:22 -d atmoz/sftp foo::1001

Run functional tests

python manager.py run-tests --test-type functional --pattern "*.py"

Quick Start

pip install core-ftp
uv pip install core-ftp     # Or using UV...
pip install -e ".[dev]"     # For development...

pip install --upgrade pip
pip install virtualenv

virtualenv --python=python3.12 .venv

source .venv/bin/activate

pip install .
pip install -e ".[dev]"

python manager.py run-tests
python manager.py run-coverage

Contributing

Contributions are welcome! Please:

1. Fork the repository

2. Create a feature branch

3. Write tests for new functionality

4. Ensure all tests pass: python manager.py run-tests

5. Run linting: pylint core_ftp

6. Run security checks: bandit -r core_ftp

7. Submit a pull request

License

This project is licensed under the MIT License. See the LICENSE file for details.

Links

• Documentation: https://core-ftp.readthedocs.io/en/latest/

• Repository: https://gitlab.com/bytecode-solutions/core/core-ftp

• Issues: https://gitlab.com/bytecode-solutions/core/core-ftp/-/issues

• Changelog: https://gitlab.com/bytecode-solutions/core/core-ftp/-/blob/master/CHANGELOG.md

• PyPI: https://pypi.org/project/core-ftp/

Support

For questions or support, please open an issue on GitLab or contact the maintainers.

Authors

• Alejandro Cora González - Initial work - alek.cora.glez@gmail.com