Utilities for editing CKAN using its API.
Project description
Introduction
This library assists CKAN editors with doing batch edits and pairs well with a library like pandas.
Installation
pip install ckan-editor-utils
The requests
package is used for all the underlying API calls.
The boto3
AWS SDK package is used for accessing and uploading files from S3.
Usage
import ckan_editor_utils
Simple API commands
For the basic API commands, much of the requests
boilerplate code is done for you.
However, the URL must already have a suffix like /api/action/
.
url = 'https://horizon.uat.gsq.digital/api/action/'
api_key = os.environ.get('CKAN_API_KEY')
dataset_id = 'my-test-dataset'
res_create = ckan_editor_utils.package_create(url,
api_key,
{
'name': dataset_id,
'extra:identifier': dataset_id,
'notes': 'my description',
'owner_org': 'geological-survey-of-queensland',
})
res_create
# <Response [200]>
# This requests response can be viewed by using the .text or .json() methods
res_create.json()
# {"help": "https://uat-external.dnrme-qld.links.com.au/api/3/action/help_show?name=package_create", "success": true, "result": {...
The response text shows the entire package as CKAN has recorded it. It will populate additional items like the Organisation description automatically. Always check the HTTP status before interacting with the data payload. A 409 will be received if it already exists or if we did not provide enough information for the type of dataset we want to be created, among other reasons.
We can use package_show
to get the metadata for an existing dataset:
res_show = ckan_editor_utils.package_show(url, api_key, dataset_id)
res_show.json()
# {'help': 'https://uat-external.dnrme-qld.links.com.au/api/3/action/help_show?name=package_show', 'success': True, 'result': {'extra:theme': []...
More examples of basic API usage can be found at the Open Data API page by GSQ.
Simplified CKAN Responses
When interacting with the CKAN API, it can be difficult to get a consistent result. Some errors are text not JSON, and the JSON errors sometimes contain different attributes depending on the context. Managing the variety of these responses means a lot of extra logic is needed, which clutters up your script.
This library offers a new CKANReponse
object that can convert requests
responses from CKAN into something
more consistent and manageable. To use it, simply pass it a CKAN response you received when using requests
.
check_res_show = ckan_editor_utils.CKANResponse(res_show) # (response from earlier example)
print(check_res_show)
# Response 200 OK
check_res_show.ok
# True
check_res_show.result
# {'extra:theme': [], 'license_title': None, 'maintainer': None, ...
A JSON response will always be present in the result
attribute of the CKAN response.
This means you can reliably use result
to capture output and it will always be relevant.
Furthermore the API action made will be logged to stdout/the console, so you can easily track progress.
Managed API actions
Some common workflows have been developed and make it easier to do simple actions.
As an editor doing bulk changes, you might not be sure if every package already exists before you can safely
call package_update()
. Instead, you can just call put_dataset()
, and the managed session will either create or
update the dataset depending on what it finds.
The following managed actions are available via the CKANEditorSession
context manager class:
- put_dataset (create or update)
- delete_dataset (delete and purge)
- put_resource_from_s3 (automatically does multipart uploads)
Additionally, the CKANEditorSession
will fix up any URLs that are missing the required extensions.
with ckan_editor_utils.CKANEditorSession(url, api_key) as ckaneu:
return ckaneu.delete_dataset(dataset_id).result
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ckan_editor_utils-0.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8da2524ea688e1e16296d5228355a832d4bfb8fc6c8ce7ea65d46825de8e164d |
|
MD5 | 9cb26f541d43055cf9110447647b2c29 |
|
BLAKE2b-256 | 45d7cd9db716641d3fe25a186064682689a8c8debb6483f66a54c39eb1f53051 |