Skip to main content
https://badge.fury.io/py/filesplit.png

filesplit

File splitting and merging made easy for python programmers!

This module
  • Can split files of any size into multiple chunks and also merge them back.

  • Can handle both structured and unstructured files.

System Requirements

Operating System: Windows/Linux/Mac

Python version: 3.x.x

Installation

The module is available as a part of PyPI and can be easily installed using pip

pip install filesplit

Split

Create an instance

from filesplit.split import Split

split = Split(inputfile: str, outputdir: str)

inputfile (str, Required) - Path to the original file.

outputdir (str, Required) - Output directory path to write the file splits.

With the instance created, the following methods can be used on the instance

bysize (size: int, newline: Optional[bool] = False, includeheader: Optional[bool] = False, callback: Optional[Callable] = None) -> None

Splits file by size.

Args:

size (int, Required): Max size in bytes that is allowed in each split.

newline (bool, Optional): Setting this to True will not produce any incomplete lines in each split. Defaults to False.

includeheader (bool, Optional): Setting this to True will include header in each split. The first line is treated as a header. Defaults to False.

callback (Callable, Optional): Callback function to invoke after each split. The callback function should accept two arguments [func (str, int)] - full path to the split file, split file size (bytes). Defaults to None.

Returns:

None

bylinecount(self, linecount: int, includeheader: Optional[bool] = False, callback: Optional[Callable] = None) -> None

Splits file by line count.

Args:

linecount (int, Required): Max lines that is allowed in each split.

includeheader (bool, Optional): Setting this to True will include header in each split. The first line is treated as a header. Defaults to False.

callback (Callable, Optional): Callback function to invoke after each split. The callback function should accept two arguments [func (str, int)] - full path to the split file, split file size (bytes). Defaults to None.

Returns:

None

The file splits are generated in this fashion [original_filename]_0001.ext, [original_filename]_0002.ext, .., [original_filename]_n.ext.

A manifest file is also created in the output directory to keep track of the file splits. This manifest file is required for merge operation.

Moreover,
  • The delimiter for the generated splits can be changed by setting splitdelimiter property like split.splitdelimiter='$'. Default is _ (underscore).

  • The number of zero fill digits for the generated splits can be changed by setting splitzerofill property like split.splitzerofill=10. Default is 4.

  • The manifest file name for the generated splits can be changed by setting manfilename property like split.manfilename='man'. Default is manifest.

  • To forcefully and safely terminate the process set the property terminate to True while the process is running.

Merge

Create an instance

from filesplit.merge import Merge

merge = Merge(inputdir: str, outputdir: str, outputfilename: str)

inputdir (str, Required) - Path to the directory containing file splits.

outputdir (str, Required) - Output directory path to write the merged file.

outputfilename (str, Required) - Name to use for the merged file.

With the instance created, the following method can be used on the instance

merge(cleanup: Optional[bool] = False, callback: Optional[Callable] = None) -> None

Merges the split files back into one single file.

Args:

cleanup (bool, Optional): If True, all the split files and manifest file will be purged after successful merge. Defaults to False.

callback (Callable, Optional): Callback function to invoke after merge. The callback function should accept two arguments [func (str, int)] - full path to the merged file, merged file size (bytes). Defaults to None.

Returns:

None

Moreover,
  • The manifest file name can be changed by setting manfilename property like merge.manfilename='man'. The manifest file name should match with the one used during the file split process and should be available in the same directory as that of file splits. Default is manifest.

  • To forcefully and safely terminate the process set the property terminate to True while the process is running.

Metadata

Release files for filesplit 4.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for filesplit 4.1.0
File Size Uploaded
filesplit-4.1.0.tar.gz 7.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for filesplit 4.1.0
File Interpreter ABI Platform
filesplit-4.1.0-py3-none-any.whl Python 3 none any Details

Total release size: 16.9 kB

Release files / filesplit-4.1.0.tar.gz

Download URL filesplit-4.1.0.tar.gz
Size 7.4 kB
Tags Source
SHA-256 checksum
How to use checksums
1aceb3a8bea84743254683e6b97056aa24593783f3b7e35dac10bac706e184b3
BLAKE2b-256 checksum
How to use checksums
0f1739439b12d77c4ca76e795832b3d3209609b58bc5a0a375630e271b5d7b88
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / filesplit-4.1.0-py3-none-any.whl

Download URL filesplit-4.1.0-py3-none-any.whl
Size 9.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5244718d37302b5741a7ffe11e7379bd178bcf31d8350632be200ba94c74a12c
BLAKE2b-256 checksum
How to use checksums
ee8b8381669a91a04834c5111e0ff1d56efb5c2779ba6e7410678f4ee4799083
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/5.1.1 CPython/3.12.7

Release history Release notifications | RSS feed

This release

4.1.0 This release

2 release files

4.0.1

2 release files

4.0.0

2 release files

3.0.2

1 release file

3.0.1

1 release file

3.0.0

1 release file

2.0.0

1 release file

1.0.4

1 release file

1.0.3

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page