HdfsCLI 
API and command line interface for HDFS.
$ hdfscli --alias=dev
Welcome to the interactive HDFS python shell.
The HDFS client is available as `CLIENT`.
In [1]: CLIENT.list('models/')
Out[1]: ['1.json', '2.json']
In [2]: CLIENT.status('models/2.json')
Out[2]: {
'accessTime': 1439743128690,
'blockSize': 134217728,
'childrenNum': 0,
'fileId': 16389,
'group': 'supergroup',
'length': 48,
'modificationTime': 1439743129392,
'owner': 'drwho',
'pathSuffix': '',
'permission': '755',
'replication': 1,
'storagePolicy': 0,
'type': 'FILE'
}
In [3]: with CLIENT.read('models/2.json', encoding='utf-8') as reader:
...: from json import load
...: model = load(reader)
...:
Features
-
Python 3 bindings for the WebHDFS (and HttpFS) API, supporting both secure and insecure clusters.
-
Command line interface to transfer files and start an interactive client shell, with aliases for convenient namenode URL caching.
-
Additional functionality through optional extensions:
avro, to read and write Avro files directly from HDFS.dataframe, to load and save Pandas dataframes.kerberos, to support Kerberos authenticated clusters.
See the documentation to learn more.
Getting started
$ pip install hdfs
Then hop on over to the quickstart guide. A Conda feedstock is also available.
Testing
HdfsCLI is tested against both WebHDFS and HttpFS. There are two ways
of running tests (see scripts/ for helpers to set up a test HDFS cluster):
$ HDFSCLI_TEST_URL=http://localhost:50070 pytest # Using a namenode's URL.
$ HDFSCLI_TEST_ALIAS=dev pytest # Using an alias.
Contributing
We'd love to hear what you think on the issues page. Pull requests are also most welcome!
1 maintainer
Metadata
1 maintainer
Release files for hdfs 2.7.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| hdfs-2.7.3.tar.gz | 43.5 kB | Details |
Release files / hdfs-2.7.3.tar.gz
| Download URL | hdfs-2.7.3.tar.gz |
|---|---|
| Size | 43.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
752a21e43f82197dce43697c73f454ba490838108c73a57a9247efb66d1c0479
|
|
BLAKE2b-256 checksum How to use checksums |
29c71be559eb10cb7cac0d26373f18656c8037553619ddd4098e50b04ea8b4ab
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.10.13
|
