sciencehub

Use the functions and data of ScienceHub projects from Python.

A ScienceHub project is a marimo notebook and the files beside it. Every function its author wrote as a reusable function (@app.function) can be called from your own code, and every file can be read. Each project page lists them under Functions and Files.

pip install sciencehub

Load a project

import sciencehub as sh

keeling_curve = sh.load("sciencehub-team/keeling-curve")
keeling_curve

sciencehub-team/keeling-curve · Keeling curve
version NS52YCXV, release 1, https://doi.org/10.83371/sh.2610.jd9q4 · https://sciencehub.sh/sciencehub-team/keeling-curve
Monthly CO₂ at Mauna Loa since 1958: NOAA's record and the functions that read it.

Functions
  load_co2(path: str = DATA) -> pd.DataFrame
      Monthly mean CO₂ in the air at Mauna Loa, Hawaii, in ppm, from March 1958.
  yearly_mean(path: str = DATA) -> pd.Series
      The deseasonalized concentration averaged over each complete year, in ppm, by year.
  growth_rate(start_year: int, end_year: int, path: str = DATA) -> float
      Average growth of CO₂ between two complete years, in ppm a year.
  seasonal_cycle(path: str = DATA) -> pd.Series
      Each month's mean departure from the deseasonalized trend, in ppm: the breath of the northern forests.

Files
  data/co2_mm_mlo.csv  38.8 kB
  README.md            1.2 kB

sh.load takes the name from the project's address: "owner/project" for its main notebook, "owner/project/notebook" for another one, or the address itself.

Full addresses must use the site selected by url=, SCIENCEHUB_URL, or https://sciencehub.sh by default. An address on another site raises ValueError before a request is sent. To choose another site, set url= or SCIENCEHUB_URL to its base address. Use a token issued by that site for its private projects.

Call a function

help(keeling_curve.growth_rate)
keeling_curve.growth_rate(2015, 2025)   # 2.63

help(), inspect.signature, Jupyter's ? and editor completion show the signature and docstring the author wrote, before anything is downloaded.

The first call downloads the notebook and its files into a cache, imports the notebook, and calls the function on your computer, with the notebook's folder as the working directory: load_co2() finds data/co2_mm_mlo.csv there, as it does on ScienceHub.

The notebook's dependencies have to be installed in your Python. If one is missing, the error names it; keeling_curve.install() installs everything the notebook declares, with pip or uv.

Calling a function runs its author's code, as installing their package would.

Read the data

Any library that reads through fsspec reads a project's files by their sciencehub:// address (sh:// for short): pandas, polars, xarray, Dask and DuckDB among them.

pip install "sciencehub[fsspec]"

import pandas as pd

co2 = pd.read_csv("sciencehub://sciencehub-team/keeling-curve/data/co2_mm_mlo.csv", comment="#")

The address is owner/project[@version]/path: @release-1 reads what release 1's DOI names, @ver_… one saved version. Files are downloaded once and checked against their SHA-256.

The package reads files without fsspec too:

path = keeling_curve.path("data/co2_mm_mlo.csv")   # downloaded once, checked, then read from disk
keeling_curve.download()                            # the notebook and every file, in one folder

File lists and notebook source filenames use relative paths with / between folders. The site, owner, project, notebook and version names used in the cache each use one name part. The SDK checks these names when it loads the metadata, before reading cached files or downloading content.

Name parts cannot be empty, . or .., contain backslashes, Windows reserved characters (<>:"|?*) or ASCII controls, or end in a dot or space. Windows device names such as CON, COM1 and COM¹ are refused, including with an extension. These rules apply on every operating system; nested Unicode file names stay valid. An invalid name raises ValueError and identifies the path or cache name to fix.

Files of public projects also have a plain address any program can read: https://sciencehub.sh/sciencehub-team/keeling-curve/raw/data/co2_mm_mlo.csv.

Run it on your computer

One command copies a version of the notebook with its files into a folder and opens it in marimo; with uv installed it runs in a sandbox that installs what the notebook declares.

uvx sciencehub edit sciencehub-team/keeling-curve              # into ./keeling-curve, in marimo's editor
uvx sciencehub run sciencehub-team/keeling-curve --release 1   # what DOI release 1 names, as an app
uvx sciencehub download sciencehub-team/keeling-curve          # only the copy

Changes stay on your computer. Upload the notebook on the project's History page to make them a version.

Check what a notebook asks for

A notebook says what hardware it needs in its script header, under [tool.sciencehub.compute] (how). check reads it as ScienceHub does and names the line of anything it cannot use, the way a compiler would, so an editor or CI can point at it:

uvx sciencehub@0.2.1 check drift.py
# drift.py:7: memory is not a compute setting. Use ram_gb: ram_gb = 16.

It exits with 1 when a header has a problem or a file cannot be read. A file that is not UTF-8 text is reported without stopping checks on the other files. The header uses TOML 1.1 on every supported Python version. The summary includes the run kind and time limit even when no hardware is named. If a header reaches the parser's integer or nesting limit, check reports it and continues with the other files.

Pin a version

A load is always one version, and keeling_curve.version says which. Pin it so your results do not change when the author saves a new one:

sh.load("sciencehub-team/keeling-curve", version="ver_01M4V36RH5ZV0RGFPSNS52YCXV")   # one saved version
sh.load("sciencehub-team/keeling-curve", release=1)            # what DOI release 1 names, files included

Private projects

Make a personal access token with the read scope at sciencehub.sh/settings/tokens and set it as SCIENCEHUB_TOKEN, or pass token= to sh.load.

Settings

Apache 2.0.