Remote file system tools for data manipulation between remote /and local host/ (cp, mv, ls, rm,..)
Project description
rfstools
Remote file system tools for data manipulation between remote and local host (cp, mv, ls, rm,..) written in Python3.
This package provides you executables pcp
, pls
, pmv
,..., which enables you to do painless data operations on remote/between remote and local host using SFTP/FTP/FTPS/SMB12(SMB version 1 or 2)/SMB23(SMB version 2 or 3)/FS(local file system) protocols.
Furthermore you can enable data reencoding during data transfer (CRLF/LF and character reencoding).
Warning: This package is still under heavy development and the API may change.
Examples
In further subchapters, there are examples of solving some of basic tasks using this tool. Only the first chapter contains the configuration file setup...in the others it can be done analogically.
Copy a file from a SMB2 server to local host
First of all, we need to set up configuration file.
cat > /some-path/rfstools.conf << EOF
connection-type=SMB12
# For SMB we always need to say, which remote folder do we use.
service-name=special-folder
host=speciserver-smb.cz
username=speciman
password=mysecret
EOF
After that, we set up an enviroment variable RFSTOOLS_CONFIG
, so the tool knows, which configuration file should it use.
export RFSTOOLS_CONFIG=/some-path/rfstools.conf
Finally, we run
pcp r:/some-path2/file ./
Downloading all files from remote folder /etc ending with .conf
pcp r:/etc/*.conf ./
Listing all subfolders in some folder
pls r:/some-path/*/
Recursive copying of remote folder to local
pcp -r r:/folder ./
Moving all remote temporary files in /tmp to local host tmp
pmv r:/tmp/* /tmp
Cross copying (using local and remote files) to remote host
pcp r:/file1 /file2 r:/target-folder
Deletetion of all remote files (with directories) ending with .tmp
prm -r r:/**/*.tmp
Copiing greped files to local host
pls -p r:/some-path | grep "^.*/SOME_REGEX$" | xargs pcp -t /target-folder
Recoding from CRLF to LF while copying from remote host to local host
export RFSTOOLS_REMOTE_CRLF=TRUE
pcp -X r:/crlf-file ./lf-file
Recoding from local UTF8 file to UTF16 remote encoding while copying
export RFSTOOLS_REMOTE_ENCODING=UTF16
pcp -X ./utf8-file r:/utf16-file
Installation
We recommend, that you upgrade your pip version before doing any further step.
pip3 install --user -U pip
# If there is no pip3
pip install --user -U pip
Be sure, that your pip
installation folder/bin is correctly in your PATH
enviroment variable.
For example, if you are on Debian and you are not installing under root, you have ~/.local/bin
in $PATH.
If it is not present, you will not able to call rfstools
from command line.
Packages rfslib
and rfstools
are private and therefore it is needed, that you add extra indexes to pip
configuration.
For example this can be done by adding file .pip/pip.conf
with
[global]
extra-index-url = https://<token_name>:<token>@git.profinit.eu/api/v4/projects/551/packages/pypi/simple
https://<token_name>:<token>@git.profinit.eu/api/v4/projects/552/packages/pypi/simple
where (token_name
, token
) pair might be ether personal access token to Profinit Gitlab or deploy token to Profinit Gitlab.
If you want to generate personal access token, you can do it here: https://git.profinit.eu/-/profile/personal_access_tokens
If you want to generate deploy token, you can do it here: https://git.profinit.eu/groups/rfs/-/settings/repository
Be aware, that if private pypi repository on Gitlab doesn't work correctly, the useless dummy public version of this package (squatting-attack prevension) might be downloaded.
Finally, install the package using pip
pip install rfstools
Upgrade
If you installed this utility using the manual above, you should be able to easily upgrade rfstools and rfslib to the current version using
pip install -U rfstools rfslib
Configuration
Args that start with '--' (eg. -U) can also be set in a config file (/etc/rfstools.conf or ~/.rfstools.conf or specified via -c). Config file syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at https://goo.gl/R74nmi). If an arg is specified in more than one place, then commandline values override environment variables which override config file values which override defaults.
Documentation
Documentation can be found in autogenerated repository rfstools-docs.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.