HDFS Python client based on WebHDFS REST API
Project description
# WebHDFS Python Client Implementation
WebHDFS is REST-API to HDFS. To facilitate access to WebHDFS from Python, webhdfs-py was developed. The library can easily be installed via easy_install or pip:
easy_install webhdfs
Webhdfs-py has no further dependencies and solely relies on the Python standard library. Similar to the Python os package, webhdfs-py provides basic capabilities like the creation/listing and deletion of directories on files.
## Hadoop configuration
Ensure that WebHDFS is enabled in the `hdfs-site.xml`:
Relevant properties:
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
see <http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html>
## Limitations
* Kerberos security not supported
WebHDFS is REST-API to HDFS. To facilitate access to WebHDFS from Python, webhdfs-py was developed. The library can easily be installed via easy_install or pip:
easy_install webhdfs
Webhdfs-py has no further dependencies and solely relies on the Python standard library. Similar to the Python os package, webhdfs-py provides basic capabilities like the creation/listing and deletion of directories on files.
## Hadoop configuration
Ensure that WebHDFS is enabled in the `hdfs-site.xml`:
Relevant properties:
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
see <http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html>
## Limitations
* Kerberos security not supported
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
WebHDFS-0.2.0.tar.gz
(3.7 kB
view details)
File details
Details for the file WebHDFS-0.2.0.tar.gz
.
File metadata
- Download URL: WebHDFS-0.2.0.tar.gz
- Upload date:
- Size: 3.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 21bf2c1cc59cbefb903834b5b22be12cfe761b27a51711fc2a69472dbace4b2a |
|
MD5 | ede91c9a5ded529d23c40a25d660e12c |
|
BLAKE2b-256 | 55b110bd31afd91217e203f0c1dd9f538b96b9d38b433b6eb63124b663c647ab |