Manages data storage for CKAN/DCOR (import, symlink, etc.)
Project description
This plugin manages how data are stored in DCOR. There are two types of files in DCOR:
Resources uploaded by users, imported from figshare, or imported from a data archive
Ancillary files that are generated upon resource creation, such as condensed DC data, preview images (see ckanext-dc_view).
This plugin implements:
Data storage management. All resources uploaded by a user are moved to /data/users-HOSTNAME/USERNAME-ORGNAME/PK/ID/PKGNAME_RESID_RESNAME and symlinks are created in /data/ckan-HOSTNAME/resources/RES/OUR/CEID. CKAN itself will not notice this. The idea is to have a filesystem overview about the datasets of each user.
Import datasets from figshare. Existing datasets from figshare are downloaded to the /data/depots/figshare directory and, upon resource creation, symlinked there from /data/ckan-HOSTNAME/resources/RES/OUR/CEID (Note that this is an exemption of the data storage management described above). When running the following command, the “figshare-import” organization is created and the datasets listed in figshare_dois.txt are added to CKAN:
ckan -c /etc/ckan/default/ckan.ini import-figshare
Import datasets from an internal depot. The depot should be present at data/depots/internal/ and follow the directory structure 201X/2019-08/20/2019-08-20_1126_c083de* where the allowed file names in this case are
2019-08-20_1126_c083de.sha256sums a file containing SHA256 sums
2019-08-20_1126_c083de_v1.rtdc the actual measurement
2019-08-20_1126_c083de_v1_condensed.rtdc the condensed dataset
2019-08-20_1126_c083de_ad1_m001_bg.png an ancillary image
2019-08-20_1126_c083de_ad2_m002_bg.png another ancillary image
…
ckan -c /etc/ckan/default/ckan.ini import-internal
Please make sure that the necessary file permissions are given in /data.
Installation
pip install ckanext-dcor_depot
Add this extension to the plugins and defaul_views in ckan.ini:
ckan.plugins = [...] dcor_depot ckan.storage_path=/data/ckan-HOSTNAME ckanext.dcor_depot.depots_path=/data/depots ckanext.dcor_depot.users_depot_name=users-HOSTNAME
This plugin stores resources to /data:
mkdir -p /data/depots/users-$(hostname) chown -R www-data /data/depots/users-$(hostname)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for ckanext-dcor_depot-0.7.11.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 458a11d60ad613611ba35ffbd6ba865fec8e2ddd3b2533ca518902f8619dfe4e |
|
MD5 | ae5e7ddc7a07a2d48de1afd8e24dc3f2 |
|
BLAKE2b-256 | 47b8a3b172ad9306b9809d328d460166f2ae73d6d5011154f6f399cc8ca59618 |