Skip to main content

Download U.S. census data and reformat it for humans

Project description

census-data-downloader

Download American Community Survey data from the U.S. Census Bureau and reformat it for humans.

What's available

All of the data files processed by this repository are published in the data/processed/ folder. They can be called in to applications via their raw URLs, like https://raw.githubusercontent.com/datadesk/census-data-downloader/master/data/processed/acs5_2017_population_counties.csv

The command-line interface

The library can be installed as a command-line interface that lets you download files on demand.

Installation

$ pipenv install census-data-downloader

Command-line usage

There's now a tool named censusdatadownloader ready for you.

Usage: censusdatadownloader [OPTIONS] TABLE COMMAND [ARGS]...

  Download Census data and reformat it for humans

Options:
  --data-dir TEXT  The folder where you want to download the data
  --year [2009-2019]   The years of data to download. By default it gets only the
                   latest year. Not all data are available for every year. Submit 'all' to get every year.
  --force          Force the downloading of the data
  --help           Show this message and exit.

Commands:
  aiannhhomelands            Download American Indian, Alaska Native and...
  cnectas                    Download combined New England city and town...
  congressionaldistricts     Download Congressional districts
  counties                   Download counties in all states
  csas                       Download combined statistical areas
  divisions                  Download divisions
  elementaryschooldistricts  Download elementary school districts
  everything                 Download everything from everywhere
  msas                       Download metropolitian statistical areas
  nationwide                 Download nationwide data
  nectas                     Download New England city and town areas
  places                     Download Census-designated places
  pumas                      Download public use microdata areas
  regions                    Download regions
  secondaryschooldistricts   Download secondary school districts
  statelegislativedistricts  Download statehouse districts
  states                     Download states
  tracts                     Download Census tracts
  unifiedschooldistricts     Download unified school districts
  urbanareas                 Download urban areas
  zctas                      Download ZIP Code tabulation areas

Before you can use it you will need to add your CENSUS_API_KEY to your environment. If you don't have an API key, you can go here. One quick way to add your key:

$ export CENSUS_API_KEY='<your API key>'

Using it is as simple as providing one our processed table names to one of the download subcommands.

Here's an example of downloading all state-level data from the medianage dataset.

$ censusdatadownloader medianage states

You can specify the download directory with --data-dir.

$ censusdatadownloader --data-dir ./my-special-folder/ medianage states

And you can change the year you download with --year.

$ censusdatadownloader --year 2010 medianage states

That's it. Mix and match tables and subcommands to get whatever you need.

Python usage

You can also download tables from Python scripts. Import the class of the processed table you wish to retrieve and pass in your API key. Then call one of the download methods.

This example brings in all state-level data from the medianhouseholdincomeblack dataset.

>>> from census_data_downloader.tables import MedianHouseholdIncomeBlackDownloader
>>> downloader = MedianHouseholdIncomeBlackDownloader('<YOUR KEY>')
>>> downloader.download_states()

You can specify the data directory and the years by passing in the data_dir and years keyword arguments.

>>> downloader = MedianHouseholdIncomeBlackDownloader('<YOUR KEY>', data_dir='./', years=2016)
>>> downloader.download_states()

Usage examples

A gallery of graphics powered by our data is available on Observable.

Black and Latino U.S. population shares

The Los Angeles Times used this library for an analysis of Census undercounts on Native American reservations. The code that powers it is available as an open-source computational notebook.

The 2020 census is coming. Will Native Americans be counted?

Contributing to the library

Adding support for a new table

Subclass our downloader and provided it with its required inputs.

import collections
from census_data_downloader.core.tables import BaseTableConfig
from census_data_downloader.core.decorators import register


@register
class MedianHouseholdIncomeDownloader(BaseTableConfig):
    PROCESSED_TABLE_NAME = "medianhouseholdincome"  # Your humanized table name
    UNIVERSE = "households"  # The universe value for this table
    RAW_TABLE_NAME = 'B19013'  # The id of the source table
    RAW_FIELD_CROSSWALK = collections.OrderedDict({
        # A crosswalk between the raw field name and our humanized field name.
        "001": "median"
    })

Add it to the imports in the __init__.py file and it's good to go.

Developing the CLI

The command-line interface is implemented using Click and setuptools. To install it locally for development inside your virtual environment, run the following installation command, as prescribed by the Click documentation.

$ pip install --editable .

That's it. If you make some good ones, please consider submitting them as pull requests so everyone can benefit.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

census-data-downloader-0.0.29.tar.gz (27.5 kB view details)

Uploaded Source

Built Distribution

census_data_downloader-0.0.29-py2.py3-none-any.whl (38.3 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file census-data-downloader-0.0.29.tar.gz.

File metadata

  • Download URL: census-data-downloader-0.0.29.tar.gz
  • Upload date:
  • Size: 27.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.6.0 requests-toolbelt/0.9.1 tqdm/4.40.2 CPython/3.7.5

File hashes

Hashes for census-data-downloader-0.0.29.tar.gz
Algorithm Hash digest
SHA256 768b730322028dd0ab2f20b0f9319f97d51c60b5ef5a53a11df40c7481901e95
MD5 c2b842560cbc7fc15d2e32e2f91fabd0
BLAKE2b-256 df75345a177206824b85ef3bd6e5e924483a7b3dd5fbc335dde43caf0930f6f6

See more details on using hashes here.

File details

Details for the file census_data_downloader-0.0.29-py2.py3-none-any.whl.

File metadata

  • Download URL: census_data_downloader-0.0.29-py2.py3-none-any.whl
  • Upload date:
  • Size: 38.3 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.6.0 requests-toolbelt/0.9.1 tqdm/4.40.2 CPython/3.7.5

File hashes

Hashes for census_data_downloader-0.0.29-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 b4347dba982355fa85087989fd44b93189d09bf4c1c34d45f09c496f967a00e9
MD5 df559523cedf919f5eb44039098c40e4
BLAKE2b-256 d0c3bcbf5c9b31697cd1b7e6d159713ffd3df18d90a95b15ca6162ce098751ac

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page