Toolshed: Less Boiler-Plate

This is a collection of well-tested, simple modules and functions that I use frequently.

Files

If you have a “proper” CSV file with quoting and such, use python’s csv module.

If all you have is a file with a header and you want to get a dictionary for each row:

>>> from toolshed import reader, header, nopen
>>> for d in reader('toolshed/tests/data/file_data.txt'):
...    print d['a'], d['b'], d['c']
1 2 3
11 12 13
21 22 23

Or as a namedtuple:

>>> from collections import namedtuple
>>> for d in reader('toolshed/tests/data/file_data.txt', header=namedtuple):
...    print d.a, d.b, d.c
1 2 3
11 12 13
21 22 23

works the same for gzipped, bzipped, and .xls files and for stdin (via “-“) and for files over http/ftp:

>>> for drow in (d for d in reader('toolshed/tests/data/file_data.txt.gz') if int(d['a']) > 10):
...    print drow['a'], drow['b'], drow['c']
11 12 13
21 22 23

if one can specify the header to a file without one using the header= kwarg. If header is “ordered” then an OrderedDictionary will be used so that drow.keys() and d.values() will return the values in the order they appeared in the file.

If header is a callable (a function or class) then, for each row, that callable will be called for each row with a single argument which is the list of columns in the future, it may be called as: callable(*row) instead of callable(row). comments welcome.

the toolshed.nopen can open a file over http, https, ftp, a gzipped file, a bzip file, or a subprocess with the same syntax:

>>> nopen('toolshed/tests/data/file_data.txt.gz') # doctest: +ELLIPSIS
<gzip open file ...>
>>> nopen('|ls') # doctest: +ELLIPSIS
<generator object process_iter at ...>

#you may need to send stdin to a proc:

# NOTE mode is None
>>> proc = nopen("|awk '(NR % 2 == 1)'", mode=None)

# write some stuff to STDIN
>>> proc.stdin.write("number\n")
>>> for i in range(5):
...    proc.stdin.write("%i\n" % i)

# IMPORTANT! close stdin
>>> proc.stdin.close()

# the read stdout
>>> for d in reader(proc.stdout, header=True):
...    print d
{'number': '1'}
{'number': '3'}

#In addition, you can skip the first lines of a file with a function like::

skipper = lambda toks: toks[0].startswith('#')
for d in reader('file-with-extra-header.txt', skip_while=skipper):
    do_stuff(d)

Pools

ctrl+c on a long-running multi-processing pool is often non-responsive. if we use toolshed.pool(), that is fixed (using signal).

this module also provides pmap, which wraps multiprocessing.Pool.map() to expand args, so we can do:

>>> def fn(a, b):  return a + b

>>> from toolshed import pmap
>>> list(pmap(fn, [(1, 1), (2, 3)]))
[2, 5]

and the fn will be mapped in parallel and we didn’t need a wrapper function for fn like:

def wrapper(args):
    return fn(*args)

as we would normally.

Note that this is like:

>>> from itertools import starmap
>>> list(starmap(fn, [(1, 1), (2, 3)]))
[2, 5]

But Pool.starmap is not available until python 3.3

This can cause problems in cases where your ‘fn’ expects args, instead of the exploded arguments. In the future, it may introspect fn, but that is not implemented for now.

Metadata

Release files for toolshed 0.4.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for toolshed 0.4.8
File	Size	Uploaded
toolshed-0.4.8.tar.gz	11.6 kB	Feb 16, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for toolshed 0.4.8
File	Interpreter	ABI	Platform	Reset
toolshed-0.4.8-py3-none-any.whl 10.2 kB Feb 16, 2026	Python 3	none	any	Details

Total release size: 21.8 kB

Release files / toolshed-0.4.8.tar.gz

Download URL	toolshed-0.4.8.tar.gz
Size	11.6 kB
Tags	Source
SHA-256 checksum How to use checksums	`a47c3ab8cec47c75a7a6d142c73663d928a9e57f6d7739108607508c8fe06e3b`
BLAKE2b-256 checksum How to use checksums	`dea6e4d602259136557b8873981e11f3445880f236e963e769f892ae314fb1d9`
Upload date	Feb 16, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.5.23`

Release files / toolshed-0.4.8-py3-none-any.whl

Download URL	toolshed-0.4.8-py3-none-any.whl
Size	10.2 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`d27f293cdf4c5c6b3045268526e9c3e0eab49b53a77adfa0ae1ee3c2eb0ee1d3`
BLAKE2b-256 checksum How to use checksums	`e08f637fc7ba68b277ee0c5cbd5368f14a752b2d2df3131c6cd439d309887f24`
Upload date	Feb 16, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.5.23`