GraphQL service for arrow tables and parquet data sets.
Project description
GraphQL service for arrow tables and parquet data sets. The schema is derived automatically.
Usage
% env PARQUET_PATH=... uvicorn graphique.service:app
Open http://localhost:8000/graphql to try out the API in GraphiQL.
There is a test fixture at ./tests/fixtures/zipcodes.parquet
.
Configuration
Graphique uses Starlette's config: in environment variables or a .env
file.
Config variables are used as input to ParquetDataset.
- COLUMNS = None
- DEBUG = False
- DICTIONARIES = None
- INDEX = None
- MMAP = True
- PARQUET_PATH
Queries
A Table
is the primary interface. It has fields for filtering, sorting, and grouping.
"""a column-oriented table"""
type Table {
"""number of rows"""
length: Long!
"""column names"""
names: [String!]!
"""fields for each column"""
columns: Columns!
"""
Return column of any type by name.
This is typically only needed for aliased columns added by `apply` or `Groups.aggregate`.
If the column is in the schema, `columns` can be used instead.
"""
column(name: String!): Column!
"""Return scalar values at index."""
row(index: Long! = 0): Row!
"""Return table slice."""
slice(offset: Long! = 0, length: Long): Table!
"""
Return tables grouped by columns, with stable ordering.
`length` is the maximum number of tables to return.
`count` filters and sorts tables based on the number of rows within each table.
"""
group(by: [String!]!, reverse: Boolean! = false, length: Long, count: CountQuery): Groups!
"""
Return table of first or last occurrences grouped by columns, with stable ordering.
Optionally include counts in an aliased column.
Faster than `group` when only scalars are needed.
"""
unique(by: [String!]!, reverse: Boolean! = false, count: String! = ""): Table!
"""Return table slice sorted by specified columns."""
sort(by: [String!]!, reverse: Boolean! = false, length: Long): Table!
"""Return table with minimum values per column."""
min(by: [String!]!): Table!
"""Return table with maximum values per column."""
max(by: [String!]!): Table!
"""
Return table with rows which match all (by default) queries.
`invert` optionally excludes matching rows.
`reduce` is the binary operator to combine filters; within a column all predicates must match.
"""
filter(query: Filters!, invert: Boolean! = false, reduce: Operator! = AND): Table!
"""
Return view of table with functions applied across columns.
If no alias is provided, the column is replaced and should be of the same type.
If an alias is provided, a column is added and may be referenced in the `column` interface,
and in the `by` arguments of grouping and sorting.
"""
apply(...): Table!
}
Performance
Graphique relies on native pyarrow routines wherever possible. Otherwise it falls back to using NumPy, with zero-copy views. Graphique also has custom optimizations for grouping, dictionary-encoded arrays, and chunked arrays.
Specifying an INDEX
of columns indicates the table is sorted, and enables a binary search interface.
"""
Return table with matching values for compound `index`.
Queries must be a prefix of the `index`.
Only one non-equal query is allowed, and applied last.
"""
search(...): Table!
Installation
% pip install graphique
Dependencies
- pyarrow >=2
- strawberry-graphql >=0.42
- pytz (optional timestamp support)
Tests
100% branch coverage.
% pytest [--cov]
Changes
0.2
ListColumn
andStructColumn
typesGroups
type withaggregate
fieldgroup
andunique
optimized- pyarrow >= 2 required
- Statistical fields:
mode
,stddev
,variance
is_in
,min
, andmax
optimized
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for graphique-0.2-cp38-cp38-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 61cdc78706e802e8fb6cc0bdbdebbf25cf55d37f579311c938af1ddbce358486 |
|
MD5 | bbd62bb802ef711106fa35ca729b7364 |
|
BLAKE2b-256 | e554bdd7b85e3af9e2f4553f3f057496d53de9ca1741db6ba66e404746735b29 |
Hashes for graphique-0.2-cp38-cp38-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b18e53144ee3d589c01bd025701fed284288c8905e558176bd23b68c248935c6 |
|
MD5 | eb2770532f2ccc09970d6abc438c3f28 |
|
BLAKE2b-256 | 3cc157bd2d335adf444e827c1840436b5d24b17d6d21392db609c2d3b87800ce |
Hashes for graphique-0.2-cp37-cp37m-manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 58b1cca3ff71494f7e6c56c88b3b65f5fb963e3685f0f3fdcbfc80eb5370f894 |
|
MD5 | 4f54e0ab750b7b23251e4483218fbdc1 |
|
BLAKE2b-256 | 331d91dddbbe9136b326db8699086a64aca0122c29bc799c5221a9474eeecde8 |
Hashes for graphique-0.2-cp37-cp37m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c01572bca558e97477a4dbb82c6fe650c491aa559c241cff73bfe5fb0e08f3e |
|
MD5 | 2d60024606c9f6e5ef6ffd9ef5a990c9 |
|
BLAKE2b-256 | 81da4f6ca05429968315e650f7e80e71025f331dcf1011acb583226b2064f7b8 |