A python library for sending usage stats events from Dspace to Matomo & ELK
============================ DSpace Usage Stats Collector
.. image:: https://readthedocs.org/projects/dspace-stats-collector/badge/?version=latest :target: https://dspace-stats-collector.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status
.. image:: https://img.shields.io/pypi/l/dspace-stats-collector.svg :target: https://pypi.python.org/pypi/dspace-stats-collector :alt: License
A python agent for sending DSpace usage statistics events to Matomo/OpenAIRE.
Free software: GNU General Public License v3
Implementation of a lightweight, easy-to-deploy, read-only alternative for a DSpace usage data collector compatible with Matomo and OpenAire usage statistics infrastructure. It sends usage data from individual repositories to an external regional aggregator by issuing read-only queries to the out-of-the-box DSpace Solr statistics subsystem.
A regional usage statistics service allows the sharing of data on item access across repositories, e-journals and CRIS systems in order to support evaluation, management and reporting. The success of this kind of service depends on installing a collector component in every repository, so one of the main requirements was to provide a user-friendly, non-invasive and reliable deploying process for repository managers.
This development is part of LA Referencia´s tasks in OpenAIRE Advance project, aimed to build a pilot on usage data exchange between Latin America and Europe open science infrastructures.
The design and the development of this usage data collector agent have been based on the following fundamental principles:
open-source, collaborative development
straightforward installation procedure for non-expert Linux users without root or superuser privileges
capable of running in a sandbox without the need for installing system-wide packages in the host system
light-weight and preserving system stability and performance
fully compatible with OpenAIRE Usage Statistics Service 
adaptable to other software platforms and aggregator services
The solution is based on a “pipe and filter” architecture with input, filter and output stages for events. This approach aims to factorize the problem in independent components, so more stages can be added/connected in the future, allowing to cover other software platforms.
In this first version of the agent, the following stages have been implemented for DSpace versions 4, 5 and 6, sending events to a Matomo instance, which is analysis platform used by the OpenAIRE :
DSpace Solr Statistics Input: an initial input component queries the internal DSpace Solr statistics core for new (later than a given/stored timestamp) usage events (item views/ item downloads). This initial event contains fields for timestamp, item id, user agent, IP address, among others
COUNTER Robots Filter: this filter excludes events generated by internet robots and crawlers based on a list of user agent values provided by project COUNTER 
DSpace Database Filter: this stage queries the internal DSpace relational database (currently only Postgres supported) for complementary item information which is not stored in the Solr core but is required by OpenAire specifications. This filter adds item title, bitstream filename and oai_identifier as event fields
Matomo API Filter: this filter transforms previously gathered data into the set of parameters required by Matomo Tracking API 
Matomo Sender Output: this filter buffers and sends batches of events into the regional tracker using the bulk tracking feature of Matomo HTTP Tracking API 
The resulting pipeline runs from the main collector script that stores the last successfully sent timestamp as a state for future calls.
.. include:: ./INSTALLATION.rst
This component is part of an alternative DSpace Usage Statistics collector strategy developed by LA Referencia / CONCYTEC (Perú) / IBICT (Brasil) / OpenAIRE as part of OpenAIRE Advance project - WP5 - Subtask 5.2.2. "Pilot common methods for usage statistics across Europe & Latin America"
 Schirrwagen, Jochen, Pierrakos, Dimitris, MacIntyre, Ross, Needham, Paul, Simeonov, Georgi, Príncipe, Pedro, & Dazy, André. (2017).
 OpenAIRE2020 - Usage Statistics Services - D8.5. doi: https://doi.org/10.5281/zenodo.1034164
 Python generators https://wiki.python.org/moin/Generators
 Project COUNTER https://www.projectcounter.org/
 Matomo tracking API, https://developer.matomo.org/api-reference/tracking-api
 DSpace Statistics https://wiki.lyrasis.org/display/DSDOC3x/DSpace+Statistics
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size dspace_stats_collector-0.4.4-py2.py3-none-any.whl (22.2 kB)||File type Wheel||Python version py2.py3||Upload date||Hashes View|
|Filename, size dspace_stats_collector-0.4.4.tar.gz (93.0 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for dspace_stats_collector-0.4.4-py2.py3-none-any.whl
Hashes for dspace_stats_collector-0.4.4.tar.gz