Skip to main content

A Filesystem-like mult-contents manager backend for Jupyter

Project description

jupyter-fs jupyter-fs

build status pypi package npm package binder link

A plugin for JupyterLab that lets you set up and use as many filebrowsers as you like, connected to whatever local and/or remote filesystem-like resources you want.

The backend is built on top of PyFilesystem and fsspec, while the frontend is built on top of tree-finder.

Install

pip install jupyter-fs

Configure

Add the following to your jupyter_server_config.json:

{
  "ServerApp": {
    "contents_manager_class": "jupyterfs.MetaManager",
    "jpserver_extensions": {
      "jupyterfs.extension": true
    }
  }
}

Resources can then be added via the Settings -> Settings Editor.

Simple use (no auth/credentials)

Add specifications for additional contents managers in your user settings (in the Settings menu under Advanced Settings Editor -> jupyter-fs). Here's an example config that sets up several new filebrowsers side-by-side:

{
  "resources": [
    {
      "name": "root at test dir",
      "url": "osfs:///Users/foo/test"
    },
    {
      "name": "s3 test bucket",
      "url": "s3://test"
    },
    {
      "name": "s3 test key",
      "url": "s3://test-2/prefix/",
      "defaultWritable": false
    },
    {
      "name": "samba guest share",
      "url": "smb://guest@127.0.0.1/test?name-port=3669"
    }
  ]
}

You should see your new filebrowsers pop up in the left-hand sidebar instantly when you save your settings:

Use with auth/credentials

Any stretch of a "url" that is enclosed in double-brackets {{VAR}} will be treated as a template, and will be handled by jupyter-fs's auth system. For example, you can pass a username/password to the "samba guest share" resource in the Simple use example above by modifying its "url" like so:

{
  "resources": [
    ...

    {
      "name": "samba share",
      "url": "smb://{{user}}:{{passwd}}@127.0.0.1/test?name-port=3669"
    }
  ]
}

When you save the above "resouces" config, a dialog box will pop asking for the username and passwd values:

Once you enter those values and hit ok, the new filebrowsers will then immediately appear in the sidebar:

The auth dialog will only appear when needed

The jupyter-fs auth dialog will only appear when:

  • JupyterLab first loads, if any fs resources require auth
  • a new fs resouce is added that requires auth, or its "url" field is modified

[!NOTE] Additional options are overrideable via environment variables by most backends for PyFilesystem and fsspec

Supported filesystems

The type of resource each filebrowser will point to is determined by the protocol at the start of its url:

PyFilesystem

  • osfs: os filesystem. The will open a new view of your local filesystem, with the specified root
  • s3: opens a filesystem pointing to an Amazon S3 bucket
  • smb: opens a filesystem pointing to a Samba share

jupyter-fs can open a filebrowser pointing to any of the diverse resources supported by PyFilesystem. Currently, we test only test the S3 and smb/samba backends as part of our CI, so your milleage may vary with the other PyFilesystem backends.

fsspec

  • local / file: Local filesystem
  • s3fs: S3 filesystem

jupyter-fs should also support any of the fsspec builtin or known backends.

In many cases, these will be customized via environment variables. As an example for s3fs, to customize the backend and auth:

export FSSPEC_S3_ENDPOINT_URL=<YOUR BACKEND>
export FSSPEC_S3_KEY=<YOUR KEY>
export FSSPEC_S3_SECRET=<YOUR SECRET>

Choosing the backend

Your resource can include a "type" field, set to either pyfs or fsspec. The default is pyfs. This field can be configured via JSON Settings or graphically.

The filesystem url

PyFilesystem

The "url" field jupyter-fs config is based on the PyFilesystem opener url standard. For more info on how to write these urls, see the documentation of the relevant PyFilesystem plugin:

fsspec

Similar to PyFilesystem, fsspec also allows for a "url" based opening scheme as documented here.

Server-side settings

If you prefer to set up your filesystem resources in the server-side config, you can do so. For example, you can set up a local filesystem by adding the following to your jupyter_server_config.py file:

c.JupyterFs.resources = [
    {
        "name": "local_test",
        "url": "osfs:///Users/foo/test"
    },
]

ALternatively, you can add resource specifications alongside the basic jupyter-fs config in your jupyter_server_config.json file:

{
  "ServerApp": {
    "contents_manager_class": "jupyterfs.metamanager.MetaManager",
    "jpserver_extensions": {
      "jupyterfs.extension": true
    }
  },
  "JupyterFs": {
    "resources": [
      {
        "name": "local_test",
        "url": "osfs:///Users/foo/test"
      }
    ]
  }
}

Any filesystem resources specified in any server-side config file will be merged with the resources given in a user's settings.

Development

See CONTRIBUTING.md for guidelines.

License

This software is licensed under the Apache 2.0 license. See the LICENSE and AUTHORS files for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jupyter_fs-1.1.0.tar.gz (1.3 MB view details)

Uploaded Source

Built Distribution

jupyter_fs-1.1.0-py3-none-any.whl (1.9 MB view details)

Uploaded Python 3

File details

Details for the file jupyter_fs-1.1.0.tar.gz.

File metadata

  • Download URL: jupyter_fs-1.1.0.tar.gz
  • Upload date:
  • Size: 1.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.10

File hashes

Hashes for jupyter_fs-1.1.0.tar.gz
Algorithm Hash digest
SHA256 5967baf485db81b09292afda0bbc8f69a70d02b19eba135aa72bd3d3ccbdf7c9
MD5 8ba8bcd51b846f48ee9ffab12386da7a
BLAKE2b-256 9b9f982b1b8c9b4434b730e285813cf1157aba2e47f5a04d4dcf37411ad8a2b5

See more details on using hashes here.

File details

Details for the file jupyter_fs-1.1.0-py3-none-any.whl.

File metadata

  • Download URL: jupyter_fs-1.1.0-py3-none-any.whl
  • Upload date:
  • Size: 1.9 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.10

File hashes

Hashes for jupyter_fs-1.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 43f3821f98948dff2a6ff00a3829df6fc1e79bb95b63a5fecccf58c8c2b075ee
MD5 3b98608da9747e5ceaaa3e5576d28711
BLAKE2b-256 5800b9759af34cf820e24dc8c2c5c97bc8942f0a11ab8f8b768f4b80993f3408

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page