Skip to main content

Loads data from various formats

Project description

==============
data-dispenser
==============

.. image:: https://travis-ci.org/catherinedevlin/data-dispenser.svg?branch=master
:alt: Travis build status

Given a source of rowlike data, acts as a generator of OrderedDicts.

Usage::

src = Source('mydata.csv')
for row in src:
print(row)

data-dispenser thus serves as a single API for a variety of data sources.

* Free software: MIT license

Data source types supported
...........................

* file names / paths
* open file objects
* pymongo Collection objects
* strings interpretable as data
* URLs beginning with http:// or https://

Will work most reliably against filenames with extensions that indicate
the data format; otherwise data-dispenser may guess the input format wrong.

Data input formats supported
............................

* csv
* yaml (requires ``pyyaml``)
* json
* pickle
* ``eval``-able Python
* xls
* xml (experimental)
* HTML with ``<table>``s

Multiple files
..............

File paths with wildcards will be
effectively concatenated into one large data source.

Load limits
...........

Large data sources could overwhelm your system's memory. Passing a ``limit``
keyword to the ``Source`` constructor limits the rows returned from each
source. For file paths with wildcards, the limit applies to each file
source, not to the number of file sources.

Code
----

https://pypi.python.org/pypi/data_dispenser

Source and bug tracker
----------------------

https://github.com/catherinedevlin/data-dispenser





History
-------

0.1.0 (2014-05-21)
++++++++++++++++++

* First release on PyPI.

0.1.1 (2014-05-23)
++++++++++++++++++

* Fixed bugs in handling non-listlike YAML files

0.2.0 (2014-07-14)
++++++++++++++++++

* Support .xls
* Support URLs
* Support wildcards

0.2.1 (2014-27-14)
++++++++++++++++++

* Support .html

0.2.2 (2014-27-14)
++++++++++++++++++

* Fix bug with .html tables with <thead>

0.2.3 (2014-09-14)
++++++++++++++++++

* Fix for URLs with no file extension

0.2.4 (2014-09-14)
++++++++++++++++++

* Better error messages on parse failure

0.2.5 (2015-02-01)
++++++++++++++++++

* Directly interpret SQLAlchemy sources

0.2.5.1 (2015-02-04)
++++++++++++++++++++

* Emit SQLAlchemy tables in order required by foreign keys

Project details


Release history Release notifications

This version
History Node

0.2.5.1

History Node

0.2.5

History Node

0.2.4

History Node

0.2.3

History Node

0.2.2

History Node

0.2.1

History Node

0.2.0

History Node

0.1.1

History Node

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
data_dispenser-0.2.5.1-py2.7.egg (17.3 kB) Copy SHA256 hash SHA256 Egg 2.7 Feb 7, 2015
data_dispenser-0.2.5.1-py3.4.egg (17.8 kB) Copy SHA256 hash SHA256 Egg 3.4 Feb 7, 2015
data_dispenser-0.2.5.1.tar.gz (11.4 kB) Copy SHA256 hash SHA256 Source None Feb 7, 2015

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page