full_outer_join
Lazy iterator implementations of a full outer join, inner join, and left and right joins of Python iterables.
This implements the sort-merge join, better known as the merge join, to join iterables in O(n) time with respect to the length of the longest iterable.
Note that the algorithm requires input to be sorted by the join key.
Example
(whitespace to make things explicit)
>>> list(full_outer_join.full_outer_join(
[{"id": 1, "val": "foo"} ],
[{"id": 1, "val": "bar"}, {"id": 2, "val": "baz"}],
key=lambda x: x["id"]
))
[
(1, ([{'id': 1, 'val': 'foo'}], [{'id': 1, 'val': 'bar'}])),
(2, ([ ], [{'id': 2, 'val': 'baz'}]))
]
To consume the output, your business logic might look like:
for group_key, key_batches in full_outer_join.full_outer_join(left, right):
left_rows, right_rows = key_batches
if left_rows and right_rows:
# This is the inner join case.
pass
elif left_rows and not right_rows:
# This is the left join case (no matching right rows)
pass
elif not left_rows and right_rows:
# This is the right join case (no matching left rows)
pass
elif not left_rows and not right_rows:
raise Exception("Unreachable")
Functions
| name | description |
|---|---|
full_outer_join(*iterables, key=lambda x: x) |
Do a full outer join on any number of iterables, returning (key, (list[row], ...)) for each key across all iterables. |
inner_join(*iterables, key=lambda x: x) |
Do an inner join across all iterables, returning (key, (list[row], ...)) for keys only in all iterables |
left_join(left_iterable, right_iterable, key=lambda x: x) |
Do a left join on both iterables, returning keys for each unique key in left_iterable |
right_join(left_iterable, right_iterable, key=lambda x: x) |
Do a right join on both iterables, returning keys for each unique key in right_iterable |
cross_join(join_output, null=None) |
Do the cross (Cartesian) join on the output of full_outer_join or inner_join, yielding (key, (iter1_row, ...)) for each row. This is implemented for completeness and is probably not useful. Iterables lacking any rows for key are replaced with null in the output. |
Why?
- Your input is already sorted and you don't want to consume your input iterators.
- Your business logic that consumes the joined output benefits from explicitly handling the match and no-match cases from each input iterable.
- You're insane. Your brain is irreparably broken by the relational model.
More examples
See test_insanity.py for a silly example of a SQL query hand-compiled into iterators.
Thanks
This was originally a PR to the more_itertools project who gave some excellent feedback on the design but ultimately did not want to merge it in.
Metadata
Release files for full-outer-join 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| full_outer_join-1.0.0.tar.gz | 7.9 kB | Without provenance | Details for full_outer_join-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| full_outer_join-1.0.0-py3-none-any.whl | Python 3 | none | any | Details for full_outer_join-1.0.0-py3-none-any.whl |
Total release size: 13.2 kB
Release files / full_outer_join-1.0.0.tar.gz
| Download URL | full_outer_join-1.0.0.tar.gz |
|---|---|
| Size | 7.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f8a22b73e6d30070f48522d2a01d56f64c6c0411d4468cc88854ce969c43e8e1
|
|
BLAKE2b-256 checksum How to use checksums |
fe02ff8350797577527ed7868070f8a689d7a3ad7a259f7752820f72d85edba9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.5
|
Release files / full_outer_join-1.0.0-py3-none-any.whl
| Download URL | full_outer_join-1.0.0-py3-none-any.whl |
|---|---|
| Size | 5.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f5c02dc0b11f4380f6c5645c2737190c49e4e6e5b9fafec2ea01c72d367b92d8
|
|
BLAKE2b-256 checksum How to use checksums |
a70bf71655666bd21eec2256a3a9c265e5491d43f6697caf313b9795301256ec
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.11.5
|
Security for full-outer-join 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.