Fused Public Python package
🌎 Geospatial, with Python, at scale.
Fused.io is a Python library to process and store geospatial data - at scale. Express workflows as a set of shareable UDFS (user defined functions) without thinking about the underlying compute. The Fused Python library is maintained by Fused.io.
Prerequisites
Python >= 3.8
Install
The Fused Python package is currently distributed via a private beta. Email info@fused.io for access.
Quickstart
import fused
# Load data
census = 's3://fused-asset/infra/census_bg_us/'
buildings = 's3://fused-asset/infra/building_msft_us/'
# Declare UDF
@fused.udf()
def census_buildings_join(left, right):
import fused
df_joined = fused.utils.geo_join(left.data, right.data)
df_cnt = df_joined.groupby(['fused_index','GEOID']).size().to_frame('count_building').reset_index()
return df_cnt
# Instantiate job configuration that runs the data against the UDF
job = census_buildings_join(census, buildings)
# Run locally
job.run_local()
# Run on remote compute managed by Fused and view logs
job_id = job.run_remote(output_table='s3://my-s3-bucket/census_buildings_join')
job_id.tail_logs()
# Export job to local directory
job.export('census_buildings_join', overwrite=True)
# Re-import job
fused.load_job('census_buildings_join')
Available operations
The following are some of the key functions:
- ingest: Upload a dataset into S3 with the Fused format.
- open_table: Open a Table object given a path to the root of the table
- run_local: Execute data processing tasks locally while you test and debug.
- run_remote: Submit jobs to run on a remote clusters - by changing a single line of code.
- export: Save a job and its configuration as a local directory, zip file, or gist.
- load_job: Open a previously saved job.
- load_udf: Open a previously saved UDF.
- show: Debugger tool.
- render: Render job or UDF to new Notebook cell and edit.
See the Fused documentation for the full list of available functions.
Docs
The documentation is a work in progress. It follows the Diátaxis system:
- Getting Started Tutorial: A hands-on introduction to Fused.
- How-to guides: Simple step-by-step user guides to accomplish specific tasks.
- Reference guide: Commands, modules, classes and methods.
- Explanation: Discussion of key decisions and design philosophy of the library.
Changelog
See the changelog for the latest changes.
Releases
The project manages releases with Semantic Versioning Specification (SemVer).
Release files for fused 1.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| fused-1.1.2.tar.gz | 150.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| fused-1.1.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 346.3 kB
Release files / fused-1.1.2.tar.gz
| Download URL | fused-1.1.2.tar.gz |
|---|---|
| Size | 150.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dd05958545b0a638139f2f44ef61b5f3b017b6a6a66492a47969acf11288c740
|
|
BLAKE2b-256 checksum How to use checksums |
2e35e40363ec9e8756e8924d2ac69eb51f6ad575fcd2a59f329d9810fab5aac8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.6.1 CPython/3.11.4 Darwin/23.2.0
|
Release files / fused-1.1.2-py3-none-any.whl
| Download URL | fused-1.1.2-py3-none-any.whl |
|---|---|
| Size | 195.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a19de38a4920b9d01d91f894ab0f48cfa095a68ce1e7b09c3eb07ab701b6e524
|
|
BLAKE2b-256 checksum How to use checksums |
9be2326658b7ec400b8cd64707ebdc7822f8195aaaefbe355d4ce08fe80f5c58
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.6.1 CPython/3.11.4 Darwin/23.2.0
|
