Skip to main content

XRootDPyFS

https://github.com/inveniosoftware/xrootdpyfs/actions?query=workflow%3ACI.svg?branch=master https://coveralls.io/repos/inveniosoftware/xrootdpyfs/badge.svg?branch=master https://pypip.in/v/xrootdpyfs/badge.svg

XRootDPyFS is a PyFilesystem interface to XRootD.

XRootD protocol aims at giving high performance, scalable fault tolerant access to data repositories of many kinds. The XRootDPyFS adds a high-level interface on top of the existing Python interface (pyxrootd) and makes it easy to e.g. copy a directory in parallel or recursively remove a directory.

Further documentation is available on https://xrootdpyfs.readthedocs.io/.

Getting started

If you just want to try out the library, the easiest is to use Docker.

Build the image:

$ docker build --platform linux/amd64 -t xrootd .

Run the container and launch xrootd:

$ docker run --platform linux/amd64 -h xrootdpyfs -it xrootd bash

You will see the logs in the stdout. Next, in another shell, connect the container and fire up an ipython shell:

$ docker ps  # find the container id
$ docker exec -it <container-id> bash
[xrootdpyfs@xrootdpyfs code]$ ipython

Quick examples

Here is a quick example of a file listing with the xrootd PyFilesystem integration:

>>> from xrootdpyfs import XRootDPyFS
>>> fs = XRootDPyFS("root://localhost//tmp/")
>>> fs.listdir("xrootdpyfs")
['test.txt']

Or, alternatively using the PyFilesystem opener (note the first import xrootdpyfs is required to ensure the XRootDPyFS opener is registered):

>>> import xrootdpyfs
>>> from fs.opener import open_fs
>>> fs = open_fs("root://localhost//tmp/")
>>> fs.listdir("xrootdpyfs")
['test.txt']

Reading files:

>>> f = fs.open("xrootdpyfs/test.txt")
>>> f.read()
b'Hello XRootD!\n'
>>> f.close()

Reading files using the readtext() method:

>>> fs.readtext("xrootdpyfs/test.txt")
b'Hello XRootD!\n'

Writing files:

>>> f = fs.open("xrootdpyfs/hello.txt", "w+")
>>> f.write("World")
>>> f.close()

Writing files using the writetext() method:

>>> fs.writetext("xrootdpyfs/test.txt", "World")

Development

The easiest way to develop is to build the Docker image and mount the source code as a volume to test any code modification with a running XRootD server:

$ docker build --platform linux/amd64 -t xrootd --progress=plain .
$ docker run --platform linux/amd64 -h xrootdpyfs -it -v <absolute path to this project>:/code xrootd bash
[xrootdpyfs@xrootdpyfs code]$ xrootd

In another shell:

$ docker ps  # find the container id
$ docker exec -it <container-id> bash
[xrootdpyfs@xrootdpyfs code]$ python -m pytest -vvv tests

If you want to test a specific version of xrootd, run:

$ docker build --platform linux/amd64 --build-arg xrootd_version=4.12.7 -t xrootd --progress=plain .

Documentation

Documentation is available at <http://xrootdpyfs.readthedocs.io/> or can be build using Sphinx:

pip install Sphinx
python setup.py build_sphinx

Testing

Running the tests are most easily done using docker:

$ docker build --platform linux/amd64 -t xrootd . && docker run --platform linux/amd64 -h xrootdpyfs -it xrootd

Release files for xrootdpyfs 2.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xrootdpyfs 2.0.0
File Size Uploaded
xrootdpyfs-2.0.0.tar.gz 41.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for xrootdpyfs 2.0.0
File Interpreter ABI Platform
xrootdpyfs-2.0.0-py2.py3-none-any.whl Python 3, Python 2 none any Details

Total release size: 63.5 kB

Release files / xrootdpyfs-2.0.0.tar.gz

Download URL xrootdpyfs-2.0.0.tar.gz
Size 41.9 kB
Tags Source
SHA-256 checksum
How to use checksums
98921f179302afba14d747cc27f925125b6936016bc2fa12126cc2feb7535442
BLAKE2b-256 checksum
How to use checksums
037975676fbb781c8e9efdaae8f9ec933f8194a993dc157e435d32eaa65f702c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.0.0 CPython/3.8.18

Release files / xrootdpyfs-2.0.0-py2.py3-none-any.whl

Download URL xrootdpyfs-2.0.0-py2.py3-none-any.whl
Size 21.6 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
4340fe2cb506706d8734882cfc346a76b86d371de6b559a2f9c98430ac10cc1d
BLAKE2b-256 checksum
How to use checksums
61bff9118f013b175a29055599f556bf4e28c36c0e367a6fb9c5a3e0518ca260
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.0.0 CPython/3.8.18
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page