jupyter-storage-explorer-polars-support

Polars support for Storage Explorer, the Jupyter Storage Explorer plugin for Polars as an engine: the data browser on Delta Lake tables, Parquet, CSV, JSON Lines and Excel, and code loading them.

Installation

pip install "jupyter-storage-explorer[polars]"          # the plugin and its libraries
pip install "jupyter-storage-explorer[polars-support]"  # the plugin only, with the libraries you have

The libraries are not the plugin's dependencies, so that installing it never changes those of a shared environment. The first command installs them, as does jupyter-storage-explorer-polars-support[full]:

Library	Needed for
`polars`	Mandatory: the engine
`deltalake`	Delta Lake tables
`fastexcel`	Excel workbooks, read by polars' `calamine`

Without deltalake or fastexcel, the other formats still work.

Prerequisites

The plugin expects polars to reach your storage already, as configured in the environment: polars reads the data without options. You can still give options in Engine options.
polars and deltalake do not use fsspec: they read cloud storage with Rust's object_store. Its configuration keys, which it also reads from environment variables, are listed for AWS, GCP and Azure. For Delta Lake tables, see delta-rs' storage options.

Plugin options

plugin_options gives the plugin's options:

StorageExplorer(
    "s3://my-bucket/lake", plugin_options={"polars-support": {"priority": 10}}
)

In the launcher's tabs, plugin_options() in launcher.py gives them:

def plugin_options():
    return {"polars-support": {"enabled": False}}

Option	Default	Meaning
`enabled`	`True`	Whether explorers use the plugin. `False`: not even imported.
`priority`	lowest	Plugins of a higher priority come first: their supports are tried before the others'. Equal ones by key.

Engine options

The supports' options, in engine_options["polars"] (by format, or "*" for all), for the data browser and the code:

StorageExplorer(
    "s3://my-bucket/lake",
    engine_options={
        "polars": {
            "*": {"storage_options": {"aws_region": "eu-west-1"}},
            "csv": {"null_values": ["NA"], "try_parse_dates": True},
            "delta": {"version": 3},
        },
    },
)

Every format but JSON and Excel takes:

Option	Default	Meaning
`storage_options`	`None`	polars' `storage_options`: object_store's keys, not fsspec's (AWS, GCP, Azure; Delta Lake's are delta-rs'). Without them, polars reads credentials from the environment.
`credential_provider`	`"auto"`	polars' `credential_provider`: a function giving cloud storage credentials, in the data browser only (never in the code).

The other options are each format's reader's arguments, of the same names, with their descriptions from polars' API documentation. Those not set are not passed, so polars' defaults, shown below, apply, except where the explorer sets its own. Values that are Python objects (e.g. data types such as pl.String in schema_overrides) are used in the data browser but left out of the code, as are options that look like credentials.

CSV (`csv`)

CsvByPolarsOptions: the arguments of pl.scan_csv.

Option	Default	Meaning
`has_header`	`True`	Indicate if the first row of the dataset is a header or not. If set to False, column names will be autogenerated in the following format: `column_x`, with `x` being an enumeration over every column in the dataset, starting at 0.
`separator`	detected	Single byte character to use as separator in the file.
`comment_prefix`	`None`	A string used to indicate the start of a comment line. Comment lines are skipped during parsing. Common examples of comment prefixes are `#` and `//`.
`quote_char`	`'"'`	Single byte character used for csv quoting, default = `"`. Set to None to turn off special handling and escaping of quotes.
`skip_rows`	`0`	Start reading after `skip_rows` rows. The header will be parsed at this offset. Note that we respect CSV escaping/comments when skipping rows. If you want to skip by newline char only, use `skip_lines`.
`skip_lines`	`0`	Start reading after `skip_lines` lines. The header will be parsed at this offset. Note that CSV escaping will not be respected when skipping lines. If you want to skip valid CSV rows, use `skip_rows`.
`schema`	`None`	Provide the schema. This means that polars doesn't do schema inference. This argument expects the complete schema, whereas `schema_overrides` can be used to partially overwrite a schema. Note that the order of the columns in the provided `schema` must match the order of the columns in the CSV being read.
`schema_overrides`	`None`	Overwrite dtypes during inference; should be a {colname:dtype,} dict or, if providing a list of strings to `new_columns`, a list of dtypes of the same length.
`null_values`	`None`	Values to interpret as null values. You can provide a: • `str`: All values equal to this string will be null. • `List[str]`: All values equal to any string in this list will be null. • `Dict[str, str]`: A dictionary that maps column name to a null value string.
`empty_string_is_null`	`True`	By default a missing string value is considered to be null. If `empty_string_is_null` is set to False, missing string values are considered to decoded as empty strings.
`ignore_errors`	`False`	Try to keep reading lines if some lines yield errors. First try `infer_schema=False` to read all columns as `pl.String` to check which values might cause an issue.
`infer_schema`	`True`	When `True`, the schema is inferred from the data using the first `infer_schema_length` rows. When `False`, the schema is not inferred and will be `pl.String` if not specified in `schema` or `schema_overrides`.
`infer_schema_length`	`100`	The maximum number of rows to scan for schema inference. This applies individually to each file included according to `infer_schema_files`. If set to `None`, the full data will be scanned into memory (this is slow). Alternatively set `infer_schema=False` to read all columns as `pl.String`.
`infer_schema_files`	`10`	How many files to use when inferring schema. Unstable in polars: it may change at any time.
`n_rows`	`None`	Stop reading from CSV file after reading `n_rows`.
`encoding`	`'utf8'`	Lossy means that invalid utf8 values are replaced with `�` characters. Defaults to "utf8".
`low_memory`	`False`	Reduce memory pressure at the expense of performance.
`skip_rows_after_header`	`0`	Skip this number of rows when the header is parsed.
`row_index_name`	`None`	If not None, this will insert a row index column with the given name into the DataFrame.
`row_index_offset`	`0`	Offset to start the row index column (only used if the name is set).
`try_parse_dates`	`False`; data browser: `True`	Try to automatically parse dates. Most ISO8601-like formats can be inferred, as well as a handful of others. If this does not succeed, the column remains of data type `pl.String`.
`eol_char`	`'\n'`	Single byte end of line character (default: `\n`). When encountering a file with windows line endings (`\r\n`), one can go with the default `\n`. The extra `\r` will be removed when processed.
`new_columns`	`None`	Provide an explicit list of string column names to use (for example, when scanning a headerless CSV file). If the given list is shorter than the width of the DataFrame the remaining columns will have their original name.
`truncate_ragged_lines`	`None`; data browser: `True`	Truncate lines that are longer than the schema.
`raise_if_empty`	`None`	Control behavior when scanning empty files: • `True`: Raise a `NoDataError`. • `False`: Return 0 rows. This defaults to `False` if `has_header=False` and a `schema` is specified, and `True` otherwise.
`decimal_comma`	`False`	Parse floats using a comma as the decimal separator instead of a period.
`glob`	`True`	Expand path given via globbing rules.
`include_file_paths`	`None`	Include the path of the source file(s) as a column with this name.
`extra_columns`	`None`	Configuration for behavior when extra columns outside of the defined schema are encountered in the data: • `ignore`: Silently ignores. • `raise`: Raises an error. Unstable in polars: it may change at any time.
`missing_columns`	`None`	Configuration for behavior when columns defined in the schema are missing from the data: • `"insert"`: Insert the missing columns with NULL values. • `"raise"`: Raise an error. Unstable in polars: it may change at any time.

JSON Lines (`json-lines`)

JsonLinesByPolarsOptions: the arguments of pl.scan_ndjson.

Option	Default	Meaning
`schema`	`None`	The DataFrame schema may be declared in several ways: • As a dict of {name:type} pairs; if type is None, it will be auto-inferred. • As a list of column names; in this case types are automatically inferred. • As a list of (name,type) pairs; this is equivalent to the dictionary form. If you supply a list of column names that does not match the names in the underlying data, the names given here will overwrite them. The number of names given in the schema should match the underlying data dimensions.
`schema_overrides`	`None`	Support type specification or override of one or more columns; note that any dtypes inferred from the schema param will be overridden.
`infer_schema_length`	`100`	The maximum number of rows to scan for schema inference. If set to `None`, the full data may be scanned (this is slow).
`batch_size`	`1024`	Number of rows to read in each batch.
`n_rows`	`None`	Stop reading from JSON file after reading `n_rows`.
`low_memory`	`False`	Reduce memory pressure at the expense of performance.
`row_index_name`	`None`	If not None, this will insert a row index column with give name into the DataFrame
`row_index_offset`	`0`	Offset to start the row index column (only use if the name is set)
`ignore_errors`	`False`	Return `Null` if parsing fails because of schema mismatches.
`include_file_paths`	`None`	Include the path of the source file(s) as a column with this name.

JSON (`json`)

JsonByPolarsOptions: the arguments of pl.read_json.

Option	Default	Meaning
`schema`	`None`	The DataFrame schema may be declared in several ways: • As a dict of {name:type} pairs; if type is None, it will be auto-inferred. • As a list of column names; in this case types are automatically inferred. • As a list of (name,type) pairs; this is equivalent to the dictionary form. If you supply a list of column names that does not match the names in the underlying data, the names given here will overwrite them. The number of names given in the schema should match the underlying data dimensions.
`schema_overrides`	`None`	Support type specification or override of one or more columns; note that any dtypes inferred from the schema param will be overridden.
`infer_schema_length`	`100`	The maximum number of rows to scan for schema inference. If set to `None`, the full data may be scanned (this is slow).

Parquet (`parquet`)

ParquetByPolarsOptions: the arguments of pl.scan_parquet.

Option	Default	Meaning
`n_rows`	`None`	Stop reading from parquet file after reading `n_rows`.
`row_index_name`	`None`	If not None, this will insert a row index column with the given name into the DataFrame
`row_index_offset`	`0`	Offset to start the row index column (only used if the name is set)
`parallel`	`'auto'`	This determines the direction and strategy of parallelism. 'auto' will try to determine the optimal direction. The `prefiltered` strategy first evaluates the pushed-down predicates in parallel and determines a mask of which rows to read. Then, it parallelizes over both the columns and the row groups while filtering out rows that do not need to be read. This can provide significant speedups for large files (i.e. many row-groups) with a predicate that filters clustered rows or filters heavily. In other cases, `prefiltered` may slow down the scan compared other strategies. The `prefiltered` settings falls back to `auto` if no predicate is given. The `prefiltered` strategy is unstable in polars: it may change at any time.
`use_statistics`	`True`	Use statistics in the parquet to determine if pages can be skipped from reading.
`hive_partitioning`	by the folders detected	Infer statistics and schema from hive partitioned URL and use them to prune reads.
`glob`	`True`	Expand path given via globbing rules.
`hidden_file_prefix`	`None`; data browser: `['.', '_']`	Skip reading files whose names begin with the specified prefixes. Unstable in polars: it may change at any time.
`schema`	`None`	Specify the datatypes of the columns. The datatypes must match the datatypes in the file(s). If there are extra columns that are not in the file(s), consider also passing `missing_columns='insert'`. Unstable in polars: it may change at any time.
`hive_schema`	`None`	The column names and data types of the columns by which the data is partitioned. If set to `None` (default), the schema of the Hive partitions is inferred. Unstable in polars: it may change at any time.
`try_parse_hive_dates`	`True`	Whether to try parsing hive values as date/datetime types.
`low_memory`	`False`	Reduce memory pressure at the expense of performance.
`cache`	`True`	Cache the result after reading.
`include_file_paths`	`None`	Include the path of the source file(s) as a column with this name.
`missing_columns`	`'raise'`	Configuration for behavior when columns defined in the schema are missing from the data: • `insert`: Inserts the missing columns using NULLs as the row values. • `raise`: Raises an error.
`extra_columns`	`'raise'`	Configuration for behavior when extra columns outside of the defined schema are encountered in the data: • `ignore`: Silently ignores. • `raise`: Raises an error.

Excel (`excel`)

ExcelByPolarsOptions: the arguments of pl.read_excel. The data browser shows a workbook's sheets on tabs below the table (only one sheet: that one), each read when chosen; with sheet_id, sheet_name or table_name, the sheet or table they give (several: on tabs).

Option	Default	Meaning
`sheet_id`	`None`	Sheet number(s) to convert (set `0` to load all sheets as DataFrames) and return a `{sheetname:frame,}` dict. (Defaults to `1` if neither this nor `sheet_name` are specified). Can also take a sequence of sheet numbers.
`sheet_name`	`None`	Sheet name(s) to convert; cannot be used in conjunction with `sheet_id`. If more than one is given then a `{sheetname:frame,}` dict is returned.
`table_name`	`None`	Name of a specific table to read; note that table names are unique across the workbook, so additionally specifying a sheet id or name is optional; if one of those parameters is specified, an error will be raised if the named table is not found in that particular sheet.
`engine`	`'calamine'`	Library used to parse the spreadsheet file; defaults to "calamine". • "calamine": this engine can be used for reading all major types of Excel Workbook (`.xlsx`, `.xlsb`, `.xls`) and is dramatically faster than the other options, using the `fastexcel` module to bind the Rust-based Calamine parser. • "openpyxl": this engine is significantly slower than both `calamine` and `xlsx2csv`, but can provide a useful fallback if you are otherwise unable to read data from your workbook. • "xlsx2csv": converts the data to an in-memory CSV before using the native polars `read_csv` method to parse the result.
`engine_options`	`None`	Additional options passed to the underlying engine's primary parsing constructor (given below), if supported: • "calamine": n/a (can only provide `read_options`) • "openpyxl": load_workbook • "xlsx2csv": Xlsx2csv
`read_options`	`None`	Options passed to the underlying engine method that reads the sheet data. Where supported, this allows for additional control over parsing. The specific read methods associated with each engine are: • "calamine": load_sheet_by_name (or load_table if using the `table_name` parameter). • "openpyxl": n/a (can only provide `engine_options`) • "xlsx2csv": see `read_csv`
`has_header`	`True`	Indicate if the first row of the table data is a header or not. If False, column names will be autogenerated in the following format: `column_x`, with `x` being an enumeration over every column in the dataset, starting at 0.
`columns`	`None`	Columns to read from the sheet; if not specified, all columns are read. Can be given as a sequence of column names or indices, or a single column name.
`schema_overrides`	`None`	Support type specification or override of one or more columns.
`infer_schema_length`	`100`	The maximum number of rows to scan for schema inference. If set to `None`, the entire dataset is scanned to determine the dtypes, which can slow parsing for large workbooks. Note that only the "calamine" and "xlsx2csv" engines support this parameter.
`include_file_paths`	`None`	Include the path of the source file(s) as a column with this name.
`drop_empty_rows`	`True`	Indicate whether to omit empty rows when reading data into the DataFrame.
`drop_empty_cols`	`True`	Indicate whether to omit empty columns (with no headers) when reading data into the DataFrame (note that empty column identification may vary depending on the underlying engine being used).
`raise_if_empty`	`True`	When there is no data in the sheet,`NoDataError` is raised. If this parameter is set to False, an empty DataFrame (with no columns) is returned instead.

Excel's code reads the file with fsspec, then polars reads its bytes: fsspec_storage_options (default None) are fsspec.open's keyword arguments there.

Delta Lake (`delta`)

DeltaLakeByPolarsOptions: the arguments of pl.scan_delta.

Option	Default	Meaning
`version`	`None`	Numerical version or timestamp version of the Delta lake table. Note: If `version` is not provided, the latest version of delta lake table is read.
`delta_table_options`	`None`	Additional keyword arguments while reading a Delta lake Table.
`use_pyarrow`	`False`	Flag to enable pyarrow dataset reads.
`pyarrow_options`	`None`	Keyword arguments while converting a Delta lake Table to pyarrow table. Use this parameter when filtering on partitioned columns or to read from a 'fsspec' supported filesystem.

Development Status
- 4 - Beta
Framework
- Jupyter
Operating System
- OS Independent
Programming Language
- Python :: 3
- Python :: 3 :: Only
Topic
- Scientific/Engineering

Metadata

Development Status
- 4 - Beta
Framework
- Jupyter
Operating System
- OS Independent
Programming Language
- Python :: 3
- Python :: 3 :: Only
Topic
- Scientific/Engineering

Release files for jupyter-storage-explorer-polars-support 0.21.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for jupyter-storage-explorer-polars-support 0.21.0
File	Size	Uploaded	Provenance
jupyter_storage_explorer_polars_support-0.21.0.tar.gz	25.3 kB	Oct 23, 2026	With provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for jupyter-storage-explorer-polars-support 0.21.0
File	Interpreter	ABI	Platform	Reset
jupyter_storage_explorer_polars_support-0.21.0-py3-none-any.whl 15.8 kB Oct 23, 2026 With provenance	Python 3	none	any	Details

Total release size: 41.1 kB

Release files / jupyter_storage_explorer_polars_support-0.21.0.tar.gz

Download URL	jupyter_storage_explorer_polars_support-0.21.0.tar.gz
Size	25.3 kB
Tags	Source
SHA-256 checksum How to use checksums	`575698e2c3efc8e0d13966c209170aecac2e550dfbf37361f9afc33acb4c2904`
BLAKE2b-256 checksum How to use checksums	`30a602898427acb0d9db5d805aef028be33c52636c55d62c5b1c9382ace65874`
Upload date	Oct 23, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.16`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/sugibuchi/jupyter-storage-explorer
Publishing commit: github.com/sugibuchi/jupyter-storage-explorer/tree/84b1c4117c01fa9066a92fd7d62bafe907981522

Workflow

Publishing configuration: github.com/sugibuchi/jupyter-storage-explorer/blob/84b1c4117c01fa9066a92fd7d62bafe907981522/.github/workflows/release-polars-support.yaml
Publishing logs: github.com/sugibuchi/jupyter-storage-explorer/actions/runs/40005383773/attempts/1

Artifact

Subject: jupyter_storage_explorer_polars_support-0.21.0.tar.gz
SHA-256 checksum: 575698e2c3efc8e0d13966c209170aecac2e550dfbf37361f9afc33acb4c2904 Verifying attestations

Release files / jupyter_storage_explorer_polars_support-0.21.0-py3-none-any.whl

Download URL	jupyter_storage_explorer_polars_support-0.21.0-py3-none-any.whl
Size	15.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`95a322ad2a138050520360285f7a93d37b62e63dd78f30b517e52af7ada857c7`
BLAKE2b-256 checksum How to use checksums	`4f76430da2c5a178051b20832bca216c58d0584a1d6ea0f9dfd7d9c7145dd560`
Upload date	Oct 23, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.16`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/sugibuchi/jupyter-storage-explorer
Publishing commit: github.com/sugibuchi/jupyter-storage-explorer/tree/84b1c4117c01fa9066a92fd7d62bafe907981522

Workflow

Publishing configuration: github.com/sugibuchi/jupyter-storage-explorer/blob/84b1c4117c01fa9066a92fd7d62bafe907981522/.github/workflows/release-polars-support.yaml
Publishing logs: github.com/sugibuchi/jupyter-storage-explorer/actions/runs/40005383773/attempts/1

Artifact

Subject: jupyter_storage_explorer_polars_support-0.21.0-py3-none-any.whl
SHA-256 checksum: 95a322ad2a138050520360285f7a93d37b62e63dd78f30b517e52af7ada857c7 Verifying attestations

Security for jupyter-storage-explorer-polars-support 0.21.0

Report project as malware

Provenance

Full provenance

All 2 files in this release have been uploaded with provenance metadata.

Attestation for jupyter_storage_explorer_polars_support-0.21.0.tar.gz

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/sugibuchi/jupyter-storage-explorer
Publishing commit: github.com/sugibuchi/jupyter-storage-explorer/tree/84b1c4117c01fa9066a92fd7d62bafe907981522

Workflow

Publishing configuration: github.com/sugibuchi/jupyter-storage-explorer/blob/84b1c4117c01fa9066a92fd7d62bafe907981522/.github/workflows/release-polars-support.yaml
Publishing logs: github.com/sugibuchi/jupyter-storage-explorer/actions/runs/40005383773/attempts/1

Artifact

Subject: jupyter_storage_explorer_polars_support-0.21.0.tar.gz
SHA-256 checksum: 575698e2c3efc8e0d13966c209170aecac2e550dfbf37361f9afc33acb4c2904 Verifying attestations

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.