eea.daviz 9.5

EEA DaViz is a plone product which uses Exhibit and Google Charts API to easily create data visualizations based on data from csv/tsv, JSON, SPARQL endpoints and more.

Introduction

EEA Daviz is a web tool developed by the European Environment Agency which helps creating interactive data visualizations easily through the web browser, no extra tools are necessary. It is free and open source.

You can generate attractive and interactive charts and combine them in a dashboard with facets/filters which updates the charts simultaneously. Data can be uploaded as CSV/TSV or you can specify SPARQL to query online Linked open data servers (aka sparql endpoints).

Daviz is the first Semantic web data visualisation tool for Plone CMS, entirely web-based!

At the moment Simile Exhibit and Google Charts visualizations are supported. The architecture allows to extend Daviz with more visualisation libraries (visualisations plugins).

Contents

• Introduction

• Main features

• Video Tutorials

• Live demos

• Architecture overview

• Installation

  • Google Maps API Key

    • Plone

    • Zope

• Dependencies

• Source code

• Copyright and license

• More documentation

• Links

• Funding and project management

• Changelog

  • 9.5 - (2015-05-15)

  • 9.4 - (2015-05-05)

  • 9.3 - (2015-04-02)

  • 9.2 - (2015-03-17)

  • 9.1 - (2014-11-20)

  • 9.0 - (2014-10-01)

  • 8.9 - (2014-08-29)

  • 8.8 - (2014-08-27)

  • 8.7 - (2014-06-30)

  • 8.6 - (2014-06-06)

  • 8.5 - (2014-05-13)

  • 8.4 - (2014-04-29)

  • 8.3 - (2014-03-10)

  • 8.2 - (2014-02-03)

  • 8.1 - (2014-01-22)

  • 8.0 - (2014-01-14)

  • 7.9 - (2013-12-09)

  • 7.8 - (2013-10-04)

  • 7.7 - (2013-09-06)

  • 7.6 - (2013-08-30)

  • 7.5 - (2013-06-17)

  • 7.4 - (2013-04-17)

  • 7.3 - (2013-03-20)

  • 7.2 - (2013-03-18)

  • 7.1 - (2013-03-15)

  • 7.0 - (2013-02-25)

  • 6.4 - (2013-02-04)

  • 6.3 - (2013-01-15)

  • 6.2 - (2013-01-10)

  • 6.1 - (2012-11-09)

  • 6.0 - (2012-10-08)

  • 4.9 - (2012-08-10)

  • 4.8 - (2012-07-13)

  • 4.7 - (2012-07-04)

  • 4.6 - (2012-06-12)

  • 4.5 - (2012-04-26)

  • 4.4 - (2012-03-30)

  • 4.3 - (2012-03-05)

  • 4.2 - (2012-02-06)

  • 4.1 - (2012-02-06)

  • 4.0 - (2011-11-07)

  • 1.5 - (2011-09-21)

  • 1.4 - (2011-09-07)

  • 1.3 - (2011-07-29)

  • 1.2 - (2011-04-07)

  • 1.1 - (2011-03-16)

  • 1.0 - (2011-03-16)

Main features

1. No desktop tools needed to make visualizations. all web based.

2. Data input: easy copy and paste of data table from any webpage or excel sheet

3. Data input from URL (CSV/TSV/JSON)

4. Data input: drag-and-drop your data from CSV/TSV files, and we do the rest

5. Data input advanced: retrieve data from any SPARQL endpoint on the fly

6. Intuitive visualization editor to create interactive charts.

7. Large amount of visualizations available: Area, Bar, Bubble, Candlestick, Column, Combo Chart, Gauge, Geo Intensity Maps, Line, Pie, Radar, Scatter, Stepped Area, Table, Tree Map, Motion Charts, Faceted search table, Faceted tiles, Faceted timeline, Faceted map and more…

8. Dashboard. Group charts into one or more dashboards.

9. Share any chart. Embeddable visualization in any webpage.

10. Customizable chart options and colors

11. Data table manipulation via drag and drop, preparing table for chart

12. Pivot table (Transpose), transform row values into columns

13. Modular framework for extending it with more visualizations

14. Branding: add your own logo to each chart + QR code

15. And much more…

It is simple to use, needs no desktop application, everything is done through the web by uploading an “excel file”, CSV, TSV. You can also query the “web of data” via public available sparql endpoints.

You can easily make visualizations like:

1. Simile Exhibit

2. Google Charts

See also initial project wiki page for the reasoning behind this project.

Video Tutorials

Most of DaViz features are presented throughout a series of video tutorials available under the EEA Web Systems Training YouTube channel. The screencasts are divided in two playlists: Basic and Advanced.

Live demos

• Basic tutorials

• Advanced tutorials

• Daviz showroom

• Google charts demos

• MIT Simile Exhibit demos

Architecture overview

At the moment Simile Exhibit and Google Charts visualizations are supported. The architecture allows to extend Daviz with more visualisation libraries (visualisations plugins).

Installation

zc.buildout

If you are using zc.buildout and the plone.recipe.zope2instance recipe to manage your project, you can do this:

• Update your buildout.cfg file:

  • Add eea.daviz to the list of eggs to install

  [instance]
  ...
  eggs =
    ...
    eea.daviz

• Re-run buildout, e.g. with:

  $ ./bin/buildout

Google Maps API Key

Plone

Within ZMI -> Plone Site -> portal_properties add a plone property sheet called geographical_properties and inside it add a new string property called google_key.

Zope

Within ZMI -> Top Folder -> manage_propertiesForm add a string property called google_key

In this property you have to paste the Google maps API KEY, follow instruction https://developers.google.com/maps/documentation/javascript/tutorial#api_key

The Google account you use to generate the key has to be owner of the site, this is done by verification via Google webmaster tools.

Dependencies

EEA Daviz has the following dependencies:

• Plone 4.x

• eea.app.visualization

• eea.sparql

• eea.forms

• eea.googlecharts

• eea.exhibit

• collective.js.jqueryui < 1.9 (Plone 4.0, 4.1, 4.2)

• collective.js.jqueryui > 1.9 (Plone 4.3+)

The following package are optional. Still they can improve the user experience with this tool:

• eea.relations

• eea.cache (Check eea.cache documentation for more about memcache configuration)

• eea.depiction

[instance]
...
eggs =
  ...
  eea.daviz [full]

zcml =
  ...
  eea.daviz-overrides
  eea.daviz-full

Source code

Latest source code (Plone 4 compatible):

• Plone Collective on Github

• EEA on Github

Plone 2 and 3 compatible (Simile Exhibit visualisations only):

https://github.com/collective/eea.daviz/tree/plone25

Copyright and license

The Initial Owner of the Original Code is European Environment Agency (EEA). All Rights Reserved.

The EEA Daviz (the Original Code) is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

More details under eea.daviz/docs/License.txt

More documentation

• Daviz technical documentation on github

• Daviz plone product summary

• Data input examples

• How to prepare your data

Links

1. Simile Wiki - Exhibit 2.0: http://simile.mit.edu/wiki/Exhibit

2. Simile widgets: http://www.simile-widgets.org/exhibit

3. EEA Daviz how-to: http://taskman.eionet.europa.eu/projects/zope/wiki/HowToDaViz

4. EEA Daviz backlog wiki: http://taskman.eionet.europa.eu/projects/zope/wiki/DaViz

5. Google charts: http://code.google.com/apis/chart/

Funding and project management

EEA - European Environment Agency (EU)

Changelog

9.5 - (2015-05-15)

• Change: Mark data provenance fields as required [szabozo0 refs #25341]

9.4 - (2015-05-05)

• Change: Made all data provenance fields required: title, link, owner [szabozo0 refs #24412]

9.3 - (2015-04-02)

• Change: Made data provenance required for published [szabozo0 refs #24412]

• Upgrade step: Within “Plone > Site setup > Add-ons” click on upgrade button available for eea.daviz

• Feature: Added IDavizLayer browser layer [olimpiurob refs #24121]

• Change: Made “MultiDataProvenanceSchemaExtender” browser layer aware [olimpiurob refs #24121]

9.2 - (2015-03-17)

• Change: Switched to curl in the jenkins build install script [olimpiurob refs #22402]

• Change: Changed fetch url for jenkins build install script to the install script from the main EEA CPB repository [olimpiurob refs #22402]

9.1 - (2014-11-20)

• Feature: formated_provenance now contain the acronym information extracted from the title of the data source if found within parenthesis [ichim-david refs #21467]

9.0 - (2014-10-01)

• Feature: removed “Caching” action. We are now using the more complex “Cache” action registered by eea.cache [ghicaale refs #21143]

• Feature: added upgrade step to remove absolete “Caching” action [ghicaale refs #21143]

8.9 - (2014-08-29)

• Feature: On the daviz edit page ask for confirmation before allowing users to modify data sources [szabozo0 refs #20869]

• Bug fix: Exclude relations where related item has link to current daviz in it’s provenance info [szabozo0 refs #20869]

8.8 - (2014-08-27)

• Change: Also include eea.cache within full.zcml [voineali refs #20678]

8.7 - (2014-06-30)

• Change: Added documentation for end users [szabozo0 refs #18400]

• Change: Read the tags for tutorial screencasts from the description of the video [szabozo0 refs #18400]

8.6 - (2014-06-06)

• Feature: Added tutorials [szabozo0 refs #18400]

8.5 - (2014-05-13)

• Change: Added support for null value for slickgrid boolean fields [olimpiurob refs #19206]

8.4 - (2014-04-29)

• Change: Made Daviz Visualization plone_lock_info aware [voineali refs #19172]

• Bug fix: Disabled rename on spreadsheet when editing an already existing visualization [szabozo0 refs #19149]

• Feature: Added file type to allowed content types list, required for saving svg versions of the charts [szabozo0 refs #18910]

• Change: Added tutorials link on edit page [szabozo0 refs #18400]

8.3 - (2014-03-10)

• Change: Always display the tutorials and example links on edit [szabozo0 refs #18400]

• Change: Link video tutorials within documentation [avoinea]

8.2 - (2014-02-03)

• Bug fix: Fixed path handling of internal links [ghicaale refs #16666]

8.1 - (2014-01-22)

• Bug fix: Fixed the case then provenance URL points to a view or template [ghicaale refs #16666]

• Feature: adding Sphinx-generated documentation in Page Template format [batradav refs #9502]

8.0 - (2014-01-14)

• Bug fix: Don’t reload edit page after a file was uploaded [szabozo0]

• Feature: When adding internal link for data provenance add the object as a related item [ghicaale refs #16666]

7.9 - (2013-12-09)

• Feature: Save chart’s customized size in the DAVIZ_CHARTS annotation [szabozo0 refs #17453]

7.8 - (2013-10-04)

• Change: Updated package dependencies to reference the newly prefered Zope locations that require minimum Plone 4.1 [ichimdav refs #15651]

7.7 - (2013-09-06)

• No changes

7.6 - (2013-08-30)

• Bug fix: Display data owner title in provenance info [szabozo0 #16585]

7.5 - (2013-06-17)

• Feature: Changed data provenance to allow multiple data provenances; and precedence of provenances [szabozo0 #9561]

• Bug fix: add an override for manage_beforeDelete to trigger a DavizWillBeRemovedEvent; intercept this event to delete the annotation on the backreferenced object that involves the deleted daviz [tiberich #14341]

• Bug fix: pylint fixes [ghicaale refs #14554]

• Feature: locales improved [lepri]

7.4 - (2013-04-17)

• Bug fix: fix migration in eea.relations integration [tiberich #14267]

7.3 - (2013-03-20)

• Bug fix: make DavizVisualization inherit from Products.ATContentTypes instead of plone.app.folder to avoid problem with LinguaPlone generated mutators [tiberich #14050]

7.2 - (2013-03-18)

• Bug fix: Fixed loss of data settings customizations if one would have changed any other metadata than ‘spreadsheet (Copy and paste a data table from a file/webpage)’within Edit tab [voineali refs #14172]

7.1 - (2013-03-15)

• Change: make chart selection for relations sortable [tiberich #13993]

• Upgrade step: go to /www/@@migrate_daviz_relations to migrate to sortable relations for daviz/ims integration [tiberich #13993]

• Feature: added information for contributors [ciobabog refs #13892]

• Change: Added description to controlpanel fields [szabozo0]

• Bug fix: fixed small bug in code that broke get_daviz when daviz was deleted [tiberich #13993]

7.0 - (2013-02-25)

• Change: removed redundant and unnecessary data_source provenance code, which is already better implemented with @@data.info [demarant refs #9561 and #9571]

6.4 - (2013-02-04)

• Feature: Support data annotations [voineali refs #9558]

6.3 - (2013-01-15)

• Upgrade step: Within “Plone > Site setup > Add-ons” run upgrades (if available) for the following packages in the same order as bellow: - EEA Versions (eea.forms) - EEA Forms (eea.forms) - EEA Sparql (eea.sparql) - EEA Visualization API (eea.app.visualisation); - EEA Google Charts (eea.googlecharts); - EEA Exhibit (eea.exhibit); - EEA Daviz (eea.daviz)

• Feature: Added italian translations [simahawk]

6.2 - (2013-01-10)

• Upgrade Step: if you have eea.relations installed in your system, in portal_setup, Import tab, select the profile “EEA Daviz: eea.relations integration” and import the Javascripts Registry step.

• Upgrade step: Within “Plone > Site setup > Add-ons” click on upgrade button available for eea.daviz

• Feature: Added WYSIWYG table editor (SlickGrid) when one directly copy&paste dataset within free text field [voineali refs #9610]

• Feature: added integration with eea.relations [tiberich refs #9571]

6.1 - (2012-11-09)

• Upgrade step: Within “Plone > Site setup > Add-ons” click on upgrade button available for eea.daviz

• Feature: Added i18n translations [avoinea]

• Feature: Possibility to add data source from a external URL [voineali refs #9576]

6.0 - (2012-10-08)

• Upgrade step: ZMI > portal_setup > profile “EEA Daviz” > import Action Providers settings

• Feature: Added Visualization data source provenance adapter for DavizVisualization objects [voineali refs #5490]

• Feature: On “Create Visualization” create the visualization and redirect to “Edit visualization” tab [szabozo0 refs #5449]

• Feature: Use portal_daviz utility for default folder settings [szabozo0 refs #5449]

• Change: Added daviz.json adapter for IDavizVisualization content-type [voineali refs #5449]

• Change: Added IVisualizationData adapter for DavizVisualization content-type [voineali refs #5511]

4.9 - (2012-08-10)

• Upgrade step: Within “Plone > Site setup > Add-ons” click on upgrade button available for eea.daviz [szabozo0 refs #5347]

• Bugfix: Fixed columns order [szabozo0 refs #5137]

• Feature: Added new action on data visualization “Create another visualization”; Added action on Sparql “Create visualization with this dataset”; Added action on all object which provides IVisualizationEnabled “Create another visualization” [szabozo0 refs #5347]

• Feature: use title of data source as default title for visualisation [szabozo0 refs #5138]

4.8 - (2012-07-13)

• Change: updated readme with info about google map API KEY configuration. [demarant refs #5338]

4.7 - (2012-07-04)

• Bug fix: Fixed possible_relations.xml when used togheter with eea.relations on a non-EEA Plone site [voineali refs #5330]

4.6 - (2012-06-12)

• Bug fix: fixed markup of HISTORY.txt file [ciobabog refs #5231]

• Feature: Warn user about changing data risks [voineali fixes #4973]

• Feature: Added @@data.info adapter for DavizVisualization content-type in order to easily get data source and data owner metadata [voineali refs #5072]

4.5 - (2012-04-26)

• Feature: Possibility to add data provenance/source information when one copy&paste data tables or upload them via QuickUpload Widget [voineali refs #5072]

4.4 - (2012-03-30)

• Feature: added documentation and links on how to examples and how prepare data for visualizations. [demarant #4995]

• Change: added default workflow mapping for daviz visualization. [demarant #5002] Upgrade step: if you have already been using daviz visualization you would need to migrate the instances to the new workflow states. Plone > Site setup > Types > choose Daviz Visualization and the new worklow to folder_workflow. Define the states mapping as you prefer.

• Upgrade step: Within “Plone > Site setup > Add-ons” click on upgrade button available for eea.daviz

• Feature: Updated Daviz Visualization fields help to provide examples and tutorials [voineali #4995]

• Change: do not show duplicates of sparql and json data when adding existing data. show only last version. Upgrade step (not needed for EEA production): ZMI > portal_setup > profile “EEA Daviz” > import Possible relations settings [demarant #5003]

• Feature: Refactoring to be able to use eea.googlecharts or eea.exhibit as standalone packages or as part of eea.daviz visualization bundle. [voineali #4996]

4.3 - (2012-03-05)

• Bug fix: restricted data sources relations to exhibit json or sparql queries. Upgrade step: ZMI > portal_setup > profile “EEA Daviz” > import Possible relations settings [demarant #4919]

• Change: The new UI expects eea.googlecharts and eea.sparql, therefore they are now an integral part of this product and declared as dependencies. [demarant]

• Change: Renamed DavizPresentation to DavizVisualization. [demarant]

• Change: removed label widget for data input, no need for it, we have already many headings. [demarant]

• Change: Re-group DavizPresentation AT Schema [voineali #4980]

• Change: Reordered the data input fields. Most common scenarion on top. rephrasing. [demarant #4980]

• Upgrade step: Within “Plone > Site setup > Add-ons” click on upgrade button available for eea.daviz [voineali]

• Fixed: Support case insensitive column types in CSV files [voineali #4975]

• Feature: Also support CSV or any other SV file by auto-detecting dialect [voineali #4977]

• Feature: Added SpreadSheet field to DavizPresentation Schema in order to easily copy & paste TAB separated text as data source [voineali #4967]

• Feature: Added QuickUpload field to DavizPresentation Schema in order to easily upload new TAB separated files as data sources [voineali #4965]

• Change: changed texts for visualizations, changed Data Sources to be mandatory [szabozo0 #4948]

• Bug fix: properly link images in css files [tiberich #4956]

• Change: corrected daviz presentation description [demarant]

4.2 - (2012-02-06)

• no changes

4.1 - (2012-02-06)

• Upgrade step: Within “Plone > Site setup > Add-ons” click on upgrade button available for eea.daviz [voineali #4866]

• Feature: Moved facets in global settings tab in order to have more space for views configuration [voineali #4914]

• Feature: Added a CSS class on section tabs [voineali #4880]

• Feature: Redirects to daviz-edit if there is no DaViz view/chart configured [voineali #4882]

• Feature: Added a way to register additional charts as tabs in view mode. [voineali #4873]

• Feature: Added DavizPresentation content-type in order to easy aggregate internal and external exhibit sources [voineali #4866]

4.0 - (2011-11-07)

• Feature: Added upgrade steps [voineali #4392]

• Feature: Plone 4.x compatible [ichimdav, voineali #4234]

• Feature: Added exhibit ‘properties’ dict in generated JSON [voineali 4528]

1.5 - (2011-09-21)

• Feature: Added exhibit ‘properties’ dict in generated JSON [voineali 4528]

1.4 - (2011-09-07)

• Feature: Enabled daviz for eea.dataservice EEAFigureFile [voineali #4492]

1.3 - (2011-07-29)

• Feature: Added babel translator API and a custom translator: rdf-xml to exhibit-json [voineali #4416]

• Feature: Possibility to add external exhibit data sources like “Google Spreadsheet”, “RDF/XML” and “External JSON”. [voineali #4416]

• Feature: Added editable ‘Lens’ property to Tiles view in order to easily create custom Exhibit Lens Templates [voineali #4414]

• Feature: Added a way to reorder Exhibit Views [voineali #4177]

• Feature: Added a way to reorder or hide Exhibit Facets [voineali #4417]

• Feature: Possibility to add/remove facets [voineali #4416]

• Feature: Made Exibit edit page more user friendly (WYSIWYG) [voineali #4417]

• Feature: JSON Converter handles column types if specified in CSV columns header (ex: Title:lable, Year:date, etc) [voineali #4414]

• Feature: Added ‘Details’ column for ‘Tabular View’. This can be enabled/disabled in ‘Tabular View’ settings. [voineali #4419]

1.2 - (2011-04-07)

• Feature: Added edit form for generated JSON [voineali #4199]

• Feature: If column name contains “:list” exported data in JSON will pe an array [ghicaale #4199]

• Bug fix: Made exibit available for ATBlob files [voineali #4135]

• Cleanup: of pyflakes warnings [ichimdav #4141]

• Cleanup: of sensible pylint violations [ichimdav #4140]

• Feature: Added live search field as default on top [demarant, #3023]

• Change: Moved facets to the right [demarant]

• Bug fix: Fixed Google Map Key alert [voineali #4196]

• Change: Disabled Thumbnails View as it’s bugous on IE [voineali #4196]

1.1 - (2011-03-16)

• Bug fix: Removed jQuery and jQuery UI from daviz-edit.css, daviz-edit.js as they generate errors if already installed. You’ll have to manually install jQuery and jQuery UI in portal_css and portal_javascripts [voineali #4196]

1.0 - (2011-03-16)

• Initial release [voineali #2773]