LOGYCA public libraries

About us

• LOGYCA Company
• LOGYCA Youtube Channel
• 
• 
• 

LOGYCA public libraries: To interact with the blob container files in your Azure Storage account.

Source code | Package (PyPI) | Samples

To install the libraries

# Check SQLAlchemy dependency that is installed
pip install logyca_azure_storage_blob

How azcopy performs file consistency validation

Like azcopy, Microsoft's client for uploading large files to Blob, the md5 hash must be computed locally and uploaded to the blob properties along with the file, the blob service will commit this at the end of the process, returning an error if presents inconsistency in value.

Calculates the MD5 hash of the file content and saves it as the Content-MD5 property of the blob:

azcopy --put-md5 #

References:

- https://learn.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-copy

This library uses the concepts of the azcopy client for its implementation.

Test upload

bandwith movistar: upload/download: 300 mbps

Test 1, function: container_blob_upload_staging_blocks_commit with console output

file: 20240126084440-backup.dump
size: 2.03 GB
parameters: 
    - self.__chunk_size_bytes=4*1024*1024 # Defaults 4 MiB
Elapsed time: 00:05:22
Output:
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[42/522], Percentage[8.05%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[43/522], Percentage[8.24%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[44/522], Percentage[8.43%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[45/522], Percentage[8.62%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[46/522], Percentage[8.81%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[47/522], Percentage[9.00%]
    ...
    ...
    ...
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[517/522], Percentage[99.04%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[518/522], Percentage[99.23%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[519/522], Percentage[99.43%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[520/522], Percentage[99.62%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[521/522], Percentage[99.81%]
    Upload process: file_to_upload[20240126084440-backup.dump], Chunks[522/522], Percentage[100.00%]

Test 2, function: container_blob_upload_data_transfer_options no console output

file: 20240126084440-backup.dump
size: 2.03 GB        
parameters:
    - self.__chunk_size_bytes=4*1024*1024 # 4 MiB
    - self.__max_single_put_size=8*1024*1024 # 8 MiB.
    - max_concurrency=1
Elapsed time: 00:11:19
Unsupported formats: ---
Output: Not available

Test 3, function: container_blob_upload_data_transfer_options no console output

file: 20240126084440-backup.dump
size: 2.03 GB        
parameters:
    - self.__chunk_size_bytes=4*1024*1024 # 4 MiB
    - self.__max_single_put_size=8*1024*1024 # 8 MiB.
    - max_concurrency=2
Elapsed time: 00:03:50
Unsupported formats: ---
Output: Not available

Test 4, function: container_blob_upload_data_transfer_options no console output

file: 20240126084440-backup.dump
size: 2.03 GB        
parameters:
    - self.__chunk_size_bytes=4*1024*1024 # 4 MiB
    - self.__max_single_put_size=8*1024*1024 # 8 MiB.
    - max_concurrency=3
Elapsed time: 00:03:38
Unsupported formats: ---
Output: Not available

Semantic Versioning

logyca_azure_storage_blob < MAJOR >.< MINOR >.< PATCH >

• MAJOR: version when you make incompatible API changes
• MINOR: version when you add functionality in a backwards compatible manner
• PATCH: version when you make backwards compatible bug fixes

Definitions for releasing versions

• https://peps.python.org/pep-0440/

  • X.YaN (Alpha release): Identify and fix early-stage bugs. Not suitable for production use.
  • X.YbN (Beta release): Stabilize and refine features. Address reported bugs. Prepare for official release.
  • X.YrcN (Release candidate): Final version before official release. Assumes all major features are complete and stable. Recommended for testing in non-critical environments.
  • X.Y (Final release/Stable/Production): Completed, stable version ready for use in production. Full release for public use.

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Types of changes

• Added for new features.
• Changed for changes in existing functionality.
• Deprecated for soon-to-be removed features.
• Removed for now removed features.
• Fixed for any bug fixes.
• Security in case of vulnerabilities.

[0.0.1rc1] - 2024-05-24

Added

• First tests using pypi.org in develop environment.

[0.1.0] - 2024-05-24

Added

• Completion of testing and launch into production.

[0.1.2] - 2024-07-18

Changed

• Two filter sa container blob list are added. modified hours ago to return modified blobs after x hours and include_subfolders to return only blobs from the received folder list.

[0.1.3] - 2025-09-26

Changed

• A new parameter is added when downloading a file to rename it on the fly on disk to the following function: container_blob_download_data_transfer_options

[0.1.4] - 2025-10-05

Fixed

• Adjustment in the container_create function when selecting the type of container to create, public, container, etc.

[0.1.5] - 2025-10-05

Fixed

• Correction of error messages when handling existing containers or containers with different states.

[0.2.0] - 2025-10-05

Added

• The new function container_blob_delete_by_age has been added to purge files by modification date. Possible uses include temporary files or archiving compliance.

[0.3.0] - 2025-11-11

Added

The container_blob_set_properties function is created. The functionality file_new_name is added to the container_blob_upload_data_transfer_options function to allow changing the file name when uploading it to a container without changing the original name on disk. A new feature called FileAnalyzer has been created. It analyzes and retrieves the metadata and file system properties of a given file at its source upon loading. The data read includes: file name and extension, size (bytes, KB, MB, GB), creation date and time, modification date and time, and last access date.

Changed

The container_blob_get_properties function is renamed to container_blob_get_properties to specify when data is retrieved and modified. The examples cover most of the library's functionalities and are organized in such a way that the numerical sequence allows them to be executed in an orderly fashion, creating a continuous flow.

[0.3.1] - 2025-11-14

Added

A parameter is added to the container_blob_upload_data_transfer_options function to load the properties of the local timestamp file for access, modification, and creation into the blob's metadata. Validation corrections are made when downloading and uploading files, specifically regarding checksum and file size.