This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description
# ckanny

## Introduction

ckanny is a [command line interface](#cli) for interacting with remote and local [CKAN](http://ckan.org/) instances. Under the hood, it uses [ckanutils](https://github.com/reubano/ckanutils)

With ckanny, you can

- Download a CKAN resource
- Create a CKAN package
- Update a CKAN DataStore from data in the FileStore
- Copy a FileStore resource from one CKAN instance to another
- and much more...

ckanny performs smart updates by computing the hash of a file and will only update the datastore if the file has changed. This allows you to schedule a script to run on a frequent basis, e.g., `@hourly` via a cron job, without updating the CKAN instance unnecessarily.

## Requirements

ckanny has been tested on the following configuration:

- MacOS X 10.9.5
- Python 2.7.9

ckanny requires the following in order to run properly:

- [Python >= 2.7](http://www.python.org/download) (MacOS X comes with python preinstalled)

## Installation

(You are using a [virtualenv](http://www.virtualenv.org/en/latest/index.html) right?)

sudo pip install ckanny

## CLI

ckanny comes with a built in command line interface `ckanny`.

### Usage

ckanny [<namespace>.]<command> [<args>]

### Examples

*show help*

ckanny -h

```bash
usage: ckanny [<namespace>.]<command> [<args>]

positional arguments:
command the command to run

optional arguments:
-h, --help show this help message and exit

available commands:
ver Show ckanny version

[ds]
delete Deletes a datastore table
update Updates a datastore table based on the current filestore resource
upload Uploads a file to a datastore table

[fs]
fetch Downloads a filestore resource
migrate Copies a filestore resource from one ckan instance to another
upload Updates the filestore of an existing resource or creates a new one

[hdx]
customize Introspects custom organization values

[pk]
create Creates a package (aka dataset)
```

*show version*

ckanny ver

*fetch a resource*

ckanny fs.fetch -k <ckan_api_key> -r <ckan_url> <resource_id>

*show fs.fetch help*

ckanny fs.fetch -h

```bash
usage: ckanny fs.fetch
[-h] [-q] [-n] [-c CHUNKSIZE_BYTES] [-u UA] [-k API_KEY] [-r REMOTE]
[-d DESTINATION]
[resource_id]

Downloads a filestore resource

positional arguments:
resource_id the resource id

optional arguments:
-h, --help show this help message and exit
-q, --quiet suppress debug statements
-n, --name-from-id Use resource id for filename
-c CHUNKSIZE_BYTES, --chunksize-bytes CHUNKSIZE_BYTES
number of bytes to read/write at a time (default:
1048576)
-u UA, --ua UA the user agent (uses `CKAN_USER_AGENT` ENV if
available) (default: None)
-k API_KEY, --api-key API_KEY
the api key (uses `CKAN_API_KEY` ENV if available)
(default: None)
-r REMOTE, --remote REMOTE
the remote ckan url (uses `CKAN_REMOTE_URL` ENV if
available) (default: None)
-d DESTINATION, --destination DESTINATION
the destination folder or file path (default:
.)
```

*create a package*

ckanny pk.create -k <ckan_api_key> -r <ckan_url> <org_id>

*create a package with resources*

ckanny pk.create -k <ckan_api_key> -r <ckan_url> -f 'file1.csv,file2.csv' <org_id>

*show pk.create help*

ckanny pk.create -h

```bash
usage: /Users/reubano/.virtualenvs/ckan/bin/ckanny pk.create
[-h] [-q] [-u UA] [-k API_KEY]
[-r REMOTE] [-e END] [-S START]
[-L LOCATION] [-c CAVEATS] [-y TYPE]
[-T TAGS] [-t TITLE]
[-m {observed,other,census,survey,registry}]
[-d DESCRIPTION] [-f FILES] [-s SOURCE]
[-l LICENSE_ID]
[org_id]

Creates a package (aka dataset)

positional arguments:
org_id the organization id

optional arguments:
-h, --help show this help message and exit
-q, --quiet suppress debug statements
-u UA, --ua UA
the user agent (uses `CKAN_USER_AGENT` ENV if
available) (default: None)
-k API_KEY, --api-key API_KEY
the api key (uses `CKAN_API_KEY` ENV if available)
(default: None)
-r REMOTE, --remote REMOTE
the remote ckan url (uses `CKAN_REMOTE_URL` ENV if
available) (default: None)
-e END, --end END
Data end date
-S START, --start START
Data start date (default: 09/25/2015)
-L LOCATION, --location LOCATION
Location the data represents (default: world)
-c CAVEATS, --caveats CAVEATS
Package caveats
-y TYPE, --type TYPE
Package type (default: dataset)
-T TAGS, --tags TAGS
Comma separated list of tags
-t TITLE, --title TITLE
Package title (default: Untitled 2015-09-25
12:36:14.141533)
-m {observed,other,census,survey,registry}, --methodology {observed,other,census,survey,registry}
Data collection methodology (default: observed)
-d DESCRIPTION, --description DESCRIPTION
Dataset description (default: same as `title`)
-f FILES, --files FILES
Comma separated list of file paths to add
-s SOURCE, --source SOURCE
Data source (default: Multiple sources)
-l LICENSE_ID, --license-id LICENSE_ID
Data license (default: cc-by-igo)
```
## Configuration

ckanny will use the following [Environment Variables](http://www.cyberciti.biz/faq/set-environment-variable-linux/) if set:

Environment Variable|Description
--------------------|-----------
CKAN_API_KEY|Your CKAN API Key
CKAN_REMOTE_URL|Your CKAN instance remote url
CKAN_USER_AGENT|Your user agent

## Hash Table

In order to support file hashing, ckanny creates a hash table resource called `hash_table.csv` with the following schema:

field|type
------|----
datastore_id|text
hash|text

By default the hash table resource will be placed in the package `hash_table`. ckanny will create this package if it doesn't exist. Optionally, you can set the hash table package in the command line with the `-H, --hash-table` option, or in a Python file as the `hash_table` keyword argument to `CKAN`.

Example:

ckanny ds.update -H custom_hash_table 36f33846-cb43-438e-95fd-f518104a32ed

## Scripts

ckanny comes with a built in task manager `manage.py` and a `Makefile`.

### Setup

pip install -r dev-requirements.txt

### Examples

*Run python linter and nose tests*

```bash
manage lint
manage test
```

Or if `make` is more your speed...

```bash
make lint
make test
```

## Contributing

View [CONTRIBUTING.rst](https://github.com/reubano/ckanny/blob/master/CONTRIBUTING.rst)

## License

ckanny is distributed under the [MIT License](http://opensource.org/licenses/MIT) the same as [ckanutils](https://github.com/reubano/ckanutils)


=========
Changelog
=========

Here you can find the recent changes to ckanny..

.. changelog::
:version: dev
:released: Ongoing

.. change::
:tags: docs

Updated CHANGES.

.. changelog::
:version: 0.1.0
:released: 2015-06-12

.. change::
:tags: project

First release on PyPi.

.. todo:: vim: set filetype=rst:
Release History

Release History

0.17.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.17.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.17.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.16.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.15.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.15.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.15.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.14.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.13.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.11.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.11.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.11.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
ckanny-0.17.2-py27-none-any.whl (18.6 kB) Copy SHA256 Checksum SHA256 py27 Wheel Dec 3, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting