Multiprocessing from the command-line
Project description
Description
===========
Multiprocessing from the command-line.
Multiproc allows one to use a pool of workers with bash. It provides a
reliable way to launch a lot of concurrent processes while keeping the
control of the number of these processes.
This program uses python3.
Documentation
=============
Usage: multiproc [-h] [-0] [-v] [-p N] [-1] FORMAT
Arguments:
FORMAT The command to be run in each process.
"%s" input string
"%n" number of the process
"%%" litteral "%"
Options:
-h, --help Print this help and exit.
-v, --verbose Set verbose output
-0 Use NUL as delimiter
This doesn't cover potential command output.
-p, --process N Number of processes to be used.
Default is the number of CPU.
-1, --uniq Equivalent to -p1
Example: download multiple files where urls.txt contains the urls list
$ cat urls.txt | multiproc -p 10 "wget %s -O file.%n"
Example
=======
Download multiple files where urls.txt contains the urls list with 10 processes:
$ cat urls.txt | multiproc -p 10 "wget %s -O file.%n"
Work with files
$ find . -iname "*.c" -print0 | multiproc -v -0 "gcc %n"
Install
=======
The easiest is
pip install multiproc
or, in this directory
python3 setup install
Dependencies
============
docopt https://github.com/docopt/docopt or "pip install docopt"
License
=======
This program is under the GPLv3 License.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Contact
=======
Main developper: Cédric Picard
Email: cedric.picard@efrei.net
Twitter: @Cym13
GPG: 383A 76B9 D68D 2BD6 9D2B 4716 E3B9 F4FE 5CED 42CB
===========
Multiprocessing from the command-line.
Multiproc allows one to use a pool of workers with bash. It provides a
reliable way to launch a lot of concurrent processes while keeping the
control of the number of these processes.
This program uses python3.
Documentation
=============
Usage: multiproc [-h] [-0] [-v] [-p N] [-1] FORMAT
Arguments:
FORMAT The command to be run in each process.
"%s" input string
"%n" number of the process
"%%" litteral "%"
Options:
-h, --help Print this help and exit.
-v, --verbose Set verbose output
-0 Use NUL as delimiter
This doesn't cover potential command output.
-p, --process N Number of processes to be used.
Default is the number of CPU.
-1, --uniq Equivalent to -p1
Example: download multiple files where urls.txt contains the urls list
$ cat urls.txt | multiproc -p 10 "wget %s -O file.%n"
Example
=======
Download multiple files where urls.txt contains the urls list with 10 processes:
$ cat urls.txt | multiproc -p 10 "wget %s -O file.%n"
Work with files
$ find . -iname "*.c" -print0 | multiproc -v -0 "gcc %n"
Install
=======
The easiest is
pip install multiproc
or, in this directory
python3 setup install
Dependencies
============
docopt https://github.com/docopt/docopt or "pip install docopt"
License
=======
This program is under the GPLv3 License.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Contact
=======
Main developper: Cédric Picard
Email: cedric.picard@efrei.net
Twitter: @Cym13
GPG: 383A 76B9 D68D 2BD6 9D2B 4716 E3B9 F4FE 5CED 42CB
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
multiproc-1.0.1.tar.gz
(3.3 kB
view details)
File details
Details for the file multiproc-1.0.1.tar.gz
.
File metadata
- Download URL: multiproc-1.0.1.tar.gz
- Upload date:
- Size: 3.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6c8a33ca3076e5154efd543d5423d0578bfdf0a7349bf0d736e901b0f4768d97 |
|
MD5 | d1bf3b0b95677fd0ab2c4bd8d19ce8fa |
|
BLAKE2b-256 | cb95c623ff6042f5b5eb195dcfd35a3a2325f712a8379921918162075630e924 |