Skip to main content

fURI File access through URIs.

Project description

# fURI

[![Build Status](](
[![PyPI version](](

Interact with local & remote files by URI

## Installation

pip install furi # Install basic support

pip install furi[all] # Install AWS & SFTP dependencies

pip intsall furi[aws] # Install AWS dependencies

pip install furi[sftp] # Install SFTP dependencies

## Usage

#### Reading Files

import furi

with'/path/to/some/file.ext') as local:
# => Hello, world!

with'file:///path/to/local/file.ext') as local:
# => Hello from Local file system

If `furi` was installed with AWS dependencies S3 files can be read as well:

with's3://bucket/path/to/key') as s3:
# => Hello from S3!

Same for SFTP files:

with'sftp://user:pass@host/path/to/file.ext') as sftp:
# => Hello from SFTP


#### Walking Directories

# Walk S3 key
for dirpath, dirnames, filenames in furi.walk('s3://bucket/path/to/key/'):
print dirpath
print dirnames
print filenames

# Walk S3 with supplied credentials
credentials = {
'aws_access_key_id' : '<access_key>',
'aws_secret_access_key' : '<secret_key>' }
for dirpath, dirnames, filenames in furi.walk('s3://bucket/path/to/key/', **credentials):
print dirpath
print dirnames
print filenames


## S3-backed files

Example S3-file access:

# Using ~/.boto or ENV variables to authenticate
s3file ='s3://bucket/path/to/key')

# Supply credentials
s3file ='s3://bucket/path/to/key',
aws_access_key_id='ACCESS', aws_secret_access_key='SECRET' )

## SFTP-backed files

Supply the credentials as a part of the URI:

sftpfile ='sftp://user:password@host/workdir/file.ext')

Or with a key:

sftpfile ='sftp://user@host/workdir/file.ext', private_key='/path/to/ssh_id')

## Supported operations

# Open a file in a supported open-mode
with'<uri>', mode='<mode>') as furifile:
furifile.exists() # Test if file exists
furifile.matches('regex pattern') # Match pattern to filename (not including path) # Read file contents' stream as string # Get handle to file contents stream
furifile.write('str or stream') # Write a string or stream to file
furifile.connect(**credentials) # Connect to a remote file service (such as S3)

## Configurations/Mappings

Structured JSON or YAML files can be loaded into a mapping object using the `` function.

### Local & S3 File Mapping

Mapping local or S3-backed file contents operations are not cached so every read has some cost to it. If numerous reads are expected it is recommended that the object is mapped into a dictionary. File-based mapping objects do not support write operations at this time.

mymap ='s3://buket/path/to/mapping.json')
mymap ='/path/to/local/mapping.yaml')

print mymap['key']
# => "Hello, world!"

print mymap['otherkey'] # Reads file from S3 again
# => "Goodby, cruel world!"

# Reads the file once and caches it in `mymap`
mymap = dict('s3://buket/path/to/mapping.json'))

### DynamoDB Mapping

Single partition-key DynamoDB tables without sort-keys are also supported and use the URI format `"dynamodb://<tablename>/"`. The returned mapping object can be queried like a dictionary.

mynamo = furi.mapping('dynamodb://mytable/', region_name='us-east-1')

print mynamo['partition_key_value']
# => { "partition_key_name" : "partition_key_value",
# "field1": "Hello, world!",
# "field2": "Goodbye, cruel world!" }

## Chained Mappings

It may be the case that you have a prioritized list of mapping files, where if a key cannot be found in the first map, the second is queried, then the third, and so on until a match is found or a `KeyError` is thrown. Use the `furi.chain()` function to chain mappings together.

chains = 's3://bucket/path/to/map.yml', '/path/to/local.json'
chainmap = furi.chain(*map(, chains))

print chainmap['key']
# => "Hello, world!"

print chainmap['otherkey']
# => WARNING:root:s3://bucket/path/to/map.yml :: KeyError('otherkey',)
# "Goodby, cruel world!"

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

furi-0.7.0.tar.gz (9.1 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page