cmem-plugin-querycatalog

Use queries from the Corporate Memory query catalog in workflows.

This is a plugin for eccenca Corporate Memory. You can install it with the cmemc command line client like this:

cmemc admin workspace python install cmem-plugin-querycatalog

Tasks

This plugin adds three workflow tasks, all of which work on a query catalog rather than on a query pasted into the workflow.

• List Catalog Queries emits one entity per query a catalog holds, describing it: the catalog graph, the query IRI, label, description, type, the query text as stored and the names of the placeholders it contains.
• Get Catalog Query emits the same description for a single query, chosen on the task or named by an incoming entity.
• Execute Catalog Query runs a stored query. A SELECT produces one entity per result row, an ASK a single answer, a CONSTRUCT or DESCRIBE its statements as quad entities, and an update query produces nothing and reports what it wrote.

A query that contains {{placeholder}} holes turns their names into input fields on Execute Catalog Query, and runs once per incoming entity.

The catalog graph is picked from a list that groups the registered query catalogs first, then the shape catalogs, then any other graph that happens to hold queries.

Known limitations

The whole result of a query is held in memory. Corporate Memory sends it in a single response, so a query returning very many rows is limited by what the DataIntegration process can hold.

Execute Catalog Query derives its input port from the placeholders of the stored query, and DataIntegration caches that port on the task. After upgrading this plugin to a version that changes the shape of a port, reload the projects that use it - cmemc project reload <project>, or the reload action in the workspace. Until then an existing task keeps the ports it was created with, and a workflow built on it can fail with an error naming a schema the current version no longer uses. Saving the task again has the same effect, and tasks created after the upgrade are unaffected.