extratools 0.6.18

Extra Functional Tools beyond Standard and Third-Party Libraries

Featured on GitHub's Trending Python repos on May 25, 2018. Thank you so much for support!

120+ extra higher-level functional tools that go beyond standard library's itertools, functools, etc. and popular third-party libraries like toolz, fancy, and more-itertools.

• Like toolz and others, most of the tools are designed to be efficient, pure, and lazy. Several useful yet non-functional tools are also included.

• While toolz and others target basic scenarios, most tools in this library target more advanced and complete scenarios.

• A few useful CLI tools for respective functions are also installed. They are available as extratools-[funcname].

Full documentation is available here.

Current Progress and Future Plans

There are currently more than 100 functions among 14 categories, 3 data structures, and 3 CLI tools.

• Currently adopted by TopSim and PrefixSpan-py.

This library is under active development, and new tools are added on almost weekly basis.

• Any idea or contribution is highly welcome.

Besides many other interesting ideas, I am planning to make the following updates in recent days/weeks/months.

• Add dicttools.unflatten and jsontools.unflatten.

• Add trie and suffixtree (according to generalized suffix tree).

• Update seqtools.commonsubseq, seqtools.commonsubseqwithgap, seqtools.align, and strtools.commonsubstr to support more than two sequences/strings.

Index of Available Tools

• Functions:

debugtools dicttools jsontools mathtools misctools printtools rangetools recttools seqtools settools sortedtools stattools strtools tabletools

• Data Structures:

defaultlist disjointsets segmenttree

• CLI Tools:

dicttools.remap jsontools.flatten stattools.teststats

Examples

Here are a few examples out of dozens of our tools.

• seqtools.compress(data, key=None) compresses the sequence by encoding continuous identical Item to (Item, Count), according to run-length encoding.

from extratools.seqtools import compress

list(compress([1, 2, 2, 3, 3, 3, 4, 4, 4, 4]))
# [(1, 1), (2, 2), (3, 3), (4, 4)]

• rangetools.gaps(covered, whole=(-inf, inf)) computes the uncovered ranges of the whole range whole, given the covered ranges covered.

from math import inf
from extratools.rangetools import gaps

list(gaps(
    [(-inf, 0), (0.1, 0.2), (0.5, 0.7), (0.6, 0.9)],
    (0, 1)
))
# [(0, 0.1), (0.2, 0.5), (0.9, 1)]

• jsontools.flatten(data, force=False) flattens a JSON object by returning (Path, Value) tuples with each path Path from root to each value Value.

import json
from extratools.jsontools import flatten

flatten(json.loads("""{
  "name": "John",
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "New York"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [],
  "spouse": null
}"""))
# {'name': 'John',
#  'address.streetAddress': '21 2nd Street',
#  'address.city': 'New York',
#  'phoneNumbers[0].type': 'home',
#  'phoneNumbers[0].number': '212 555-1234',
#  'phoneNumbers[1].type': 'office',
#  'phoneNumbers[1].number': '646 555-4567',
#  'children': [],
#  'spouse': None}

strtools.learnrewrite(src, dst, minlen=3) learns the respective regular expression and template to rewrite src to dst.

from extratools.strtools import learnrewrite

learnrewrite(
    "Elisa likes icecream.",
    "icecream is Elisa's favorite."
)
# ('(.*) likes (.*).',
#  "{1} is {0}'s favorite.")

tabletools.parsebymarkdown(text) parses a text of multiple lines to a table, according to Markdown format.

from extratools.tabletools import parsebymarkdown

list(parsebymarkdown("""
| foo | bar |
| --- | --- |
| baz | bim |
"""))
# [['foo', 'bar'],
#  ['baz', 'bim']]

Installation

This package is available on PyPI. Just use pip3 install -U extratools to install it.

Other Libraries

The following libraries are highly recommended to use together with extratools.

toolz sortedcontainers

Reference

When using for research purpose, please cite this library as follows.

@misc{extratools,
  author = {Chuancong Gao},
  title = {{extratools}},
  howpublished = "\url{https://github.com/chuanconggao/extratools}",
  year = {2018}
}