Skip to main content

Vocabulary library Plone. Central, Pluggable, TTW, with IMS VDEX Support

Project description

ATVocabularyManager: a vocabulary managing portal tool for Plone

.. contents::


- ATVocabularyManager offers central through the Plone management of
dynamic vocabularies.

- This product is based on Archetypes and made to work with Archetypes as well
as with other Products. It is intended use is within Archetypes Fields.
Using it as a vocabulary provider for CMFMetadata worked out fine too.
Integration with different other products will work as well.

- to use a managed vocabulary simply add the term
``vocabulary = NamedVocabulary("myvocabulary")`` to the fields of your
Archetypes Schema, import NamedVocabulary from this Product and create
your vocabulary with id myvocabulary in 'portal_vocabularies' tool
(available through Plone Site-Setup).

- ATVocabularyManager supports:

simple flat key
value vocabularies,

tree like
hierachical vocabularies (see Limitations),

IMS Vocabulary Definition Exchange Format (VDEX) aware vocabularies
with XML Import and Export. VDEX is i18n-aware by its nature and does
not need LinguaPlone!

Vocabularies are pluggable types.
ATVocabularyManager is prepared for extension with your special
vocabulary type. ArchGenXML will help you here.
Each vocabulary term needs to be an CMF aware content type. Reuse normal
rich content as a vocabularies.

ArchGenXML 1.4+ code-generator does full integration of ATVocabularyManager:
via tagged value defined named vocabularies are registered transparently,
VDEX-XML files are imported at install-time, stub vocabularies are created
at install time and custom types are registered by just providing appropriate

ATVM is Linguaplone compatible (only tested with SimpleVocabulary, and
TreeVocabulary). Add a simple vocabulary with some items, install and configure
Linguaplone, translate the vocabulary to the language(s) of your choice,
translate every item to the language(s) of your choice. NamedVocabulary() will
return the vocabulary as usual, the keys will stay the same disregarding
language settings, the values will show in the currently selected language.
VDEX vocabularies are not using LinguaPlone, but are i18n-aware (imo much
better than everything else).

You can do hierachy-aware searches on treevocabularies (for more information on
this see doc/search_treevocabulary.txt) attention: curently certain changes in
the term hierachy require a catalog rebuild (see Limitations).


See docs/INSTALL.txt.

To speed up ATVocabularyManager you might want to associate it with a Cache-Manager.


- Archetypes 1.5.x and above.

- Targets with Plone 3.1+

- imsvdex (usally it get fetched automatically as dependency, otherwise
easy_install imsvdex - )

- Linguaplone (optional: for tests and multilingual vocabularies)


If you're using a buildout run the tests using

bin/instance test -s Products.ATVocabularyManager

To run a certain testmodule run

bin/instance test -s Products.ATVocabularyManager -m testSimpleVocabulary


TreeVocabulary catalog updates
If you are using the hierachy-aware catalog search support of
tree-vocabularies you need to rebuild the uid_catalog and portal_catalog
if you move a vocabularyterm that has other terms below it.

Creation/Edit of VDEX TTW is difficult.
Take an editor of your choice, create the vocab and upload it or help
writing an UI.

Known Bugs

- Check the issue-tracker:
Feel free to add bugs in here!

Usally the authors are offering professional support. The classical well-working
community support is found at the mailing-lists and IRC-channels announced at

Several parts code was created for the ZUCCARO project.
ZUCCARO (Zope-based Universally Configurable Classes for Academic
Research Online) is a database framework for the Humanities developed
by the Bibliotheca Hertziana, Max Planck Institute for Art History
For further information: "":

Copyright / License

Products.ATVocabularyManager is under a BSD-like Licence.

Products.ATVocabularyManager is copyright by

- Jens Klein <> and the respective authors

- 2010-2011 by BlueDynamics Alliance, Austria, Germany, Switzerland

- 2007-2008 by BlueDynamics Alliance, Klein & Partner KEG, Austria

- 2004-2009 by BlueDynamics Alliance, Klein & Partner KG, Austria
and parts eduplone Open Source Business Network EEIG, Austria

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are

* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.
* Neither the name of ATVocabularyManager nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.



- Jens Klein <>

- Major contributions by Harald Friessnegger 'frisi': i18n, cleanup,
more cleanup of hierachical vocabularies.


Several people committed smaller fixes and translations, thank you all!

- andrewburkhalter - <>
- do3cc
- fRiSi <>
- gotcha <>
- jensens <>
- naro <>
- rocky <>
- thet <>
- Wouter Vanden Hove, <>



- create an issue: simplevocabs do not get alphabetically ordered,
nore do they use the order defined by getObjectPositionInParent
(see issue #4)

- drop the try
from Products.Archetypes.interfaces.vocabulary import IVocabularyTerm

- we should add the parameter 'lang' to IVocabularyTerm.getTermValue
[jens] agrees, but we should make it optional (normally translations should happen transparently)

- move getTermPath to separate hierarchicalvocabulary interface?
[hf] this would allow us to differentiate hierarchical vocabularies by their interface
we'd should to make namedvocbaulary.getKeyPathForTerms aware of this too then.

- (missing in alias) mark methods getVocabularyValue and getVocabularyKey as deprecated
and use the methods getTermValue and getTermKey instead
[jens] XXX

- use zope3 interfaces for terms
useful for example in NamedVocabulary.getKeyPathForTerms (see comment there)

- resolve TODOs (XXX, FIXME) marked in the code ;-)

Ideas for features


basic idea:
fields can specify selection depth (0 unlimited, 1 first level, 2 seconde level, etc)
that gets displayed in the Displaylist.

why do we need that:
if a certain field does not need the fine-grained selection possibilities
a vocabulary offers, no new vocabulary has to be defined.

variable displaylists

basic idea:
field can specify which attribute of the vocabularyterms is used for the

why could we need that:
in certain cases the same vocabulary is needed, but another field or field
combination should be used for display in the selection.

more complicated handling of cached displaylists (different dictionaries have
to be handeled, not just one)

caching displaylists

basic idea:
caching of displaylists and vocabularydicts using ZCacheable (the only thing we'd
have to care about is invalidating the cache when a term gets moved or changed)

why do we need that:
fetching vocabulary displaylists costs a lot.
at the moment the displaylist for the whole vocab gets fetched
everytime a content type that uses the vocabulary is edited and even VIEWED!

[jens] will implement this for the vdex vocabulary in the near future: DONE
open tasks: Invalidate cache after upload of a vdex.
(Auto?)-Assign to a ram-cache-manager.

generic setup support

basic idea
vdex already can be ex and imported to xml. we shoud realize it for simple
and term too
[jens] decision needed: use vdex, some own xml or better atxml from marshall
or just simple Excel importable CSV?)
Jens decision is: vdex (implemented import) or csv (todo).

common hierarchic functionality of vdex and tree and possibly other types
should be coded in a mixin.
code and test once, use everywhere

catalog updates

current limitation:

every change on the structure of a vocabulary
(eg: moving a term up the hierarchy)
has to trigger not only the invalidation of the cache but also
the reindexing of all terms down the hierarchy
(uid_catalog search on getTermUidPath,

currently this is a limitation of treevocabulary.
vdex dont need to get cataloged

need to reindex objects in uid_catalog and portal_catalog:
Which method do we need to hook?

in portal_catalog for indexes used by content types
the worse problem is to reindex the objects that define custom indexes on their path-methods.
we don't know which catalog they use (could be portal_catalog, but also any other)


- store catalog and index in vocab or the atvmtool
- provide a hook somewhere that can be implemented
- maybe use references instead of text-linefields to be able to find out which content uses a certain term.

Simple Vocabulary

Simple vocabularies are flat vocabularies.

Creating a simple vocabulary

SimpleVocabularies can be created in the vocabulary library:

>>> atvm = self.portal.portal_vocabularies
>>> self.setRoles(['Manager'])
>>> _ = atvm.invokeFactory('SimpleVocabulary', 'testvocab', title="test vocabulary")

You can fetch a vocabulary calling ``getVocabularyByName`` on
the vocabulary library

>>> simple = atvm.getVocabularyByName('testvocab')
>>> simple
<SimpleVocabulary at /plone/portal_vocabularies/testvocab>
>>> simple.Title()
'test vocabulary'

Creating simple vocabulary terms

Simple vocabularies can only contain SimpleVocabularyTerms.

>>> simple.allowedContentTypes()
[<FactoryTypeInformation at /plone/portal_types/SimpleVocabularyTerm>]

You can add terms using ``invokeFactory`` or the method

>>> _ = simple.invokeFactory('SimpleVocabularyTerm', 'term1')
>>> simple.addTerm('term2', 'first time')

addTerm can ignore duplicate keys, and returns whether adding
has been successfull.

>>> simple.addTerm('term2', 'second time', silentignore=True)
>>> simple.getDisplayList(self)
<DisplayList [('term1', ''), ('term2', 'first time')] at ...>

Batch creation

For creating simple vocabularies in python code you can use
a convenience method atvm provides for you:

>>> from Products.ATVocabularyManager.utils.vocabs import createSimpleVocabs

This needs to be fed with a dictionary in the following format::

>>> testvocabs = {}
>>> testvocabs['sorting'] = (
... ('c', u'Alpha'),
... ('a', u'Zeppelin'),
... ('y', u'Charly'))
>>> createSimpleVocabs(atvm, testvocabs)
>>> sorting = atvm.getVocabularyByName('sorting')
>>> sorting.contentIds()
['c', 'a', 'y']


You can define the sort order of the vocabularyterms within a
simplevocabulary by choosing one of the values in `Sort method`.

The default sort order is alphabetically by Vales (Title)::

>>> sorting.getField('sortMethod').vocabulary.keys()
['getObjPositionInParent', 'lexicographic_values', 'lexicographic_keys']
>>> sorting.getSortMethod()

Sorting by Values

>>> sorting.getDisplayList(self)
<DisplayList [('c', 'Alpha'), ('y', 'Charly'), ('a', 'Zeppelin')] at ...>

Sorting by keys

>>> from Products.ATVocabularyManager.config import SORT_METHOD_LEXICO_KEYS
>>> sorting.setSortMethod(SORT_METHOD_LEXICO_KEYS)
>>> sorting.getDisplayList(self)
<DisplayList [('a', 'Zeppelin'), ('c', 'Alpha'), ('y', 'Charly')] at ...>

Sorting by folder position

>>> sorting.listFolderContents()
[<SimpleVocabularyTerm .../sorting/c>, <SimpleVocabularyTerm .../sorting/a>, <SimpleVocabularyTerm .../sorting/y>]
>>> from Products.ATVocabularyManager.config import SORT_METHOD_FOLDER_ORDER
>>> sorting.setSortMethod(SORT_METHOD_FOLDER_ORDER)
>>> sorting.getDisplayList(self)
<DisplayList [('c', 'Alpha'), ('a', 'Zeppelin'), ('y', 'Charly')] ...>

Linguaplone Support

Simple vocabularies have full Linguaplone Support, but also work
w/o having Linguaplone installed::

>>> self.portal.portal_quickinstaller.isProductInstalled('LinguaPlone')

If Linguaplone is not installed, our vocabulary works
without causing troubles

>>> self._createTestVocabulary()
>>> vocab = self.atvm.teststates
>>> vocab.getVocabularyDict(vocab)
{'ger': 'Germany', 'fin': 'Finland', 'aut': 'Austria', 'nor': 'Norway'}

If Linguaplone is installed, the simplevocabulary is fully
supporting translated vocabularyterms:

XXX go on porting the tests/testSimpleVocabulary testTranslations here


Also sorting on translated vocabularies works as excepted:
XXX write tests for the following

* keys are the same for every language, so sorting by key should result in the same order in every language

* sorting on title might result in completely differnt order

* sorting on position in folder might be differnt to, since translations have their separate order

Searches on hierarchical vocabularies

This tests demonstrates how ATVM enables you to make your contenttype
searchable using a treevocabulary field.

imagine the following vocabulary:


* electrical
* mechanical

* otto engine
* diesel engine

a search for engine==mechanical should return all entries
for otto and diesel engines

You can skip this tutorial if you are no developer and simply
want to learn how to do hierarchy-aware searches on your content type.
This is demonstrated in the Product ATVocabularyManagerExample_

.. _ATVocabularyManagerExample:

Setup Vocabulary

let's create our vocabulary using the utility methods
provided by ATVM::

>>> from Products.CMFCore.utils import getToolByName
>>> atvm = getToolByName(self.portal, 'portal_vocabularies')
>>> from Products.ATVocabularyManager.utils.vocabs import createHierarchicalVocabs

>>> dictionary = {
... ('electrical', 'electrical engines'): {},
... ('mechanical', 'mechanial engines'): {
... ('otto', 'ottomotor'): {},
... ('diesel', 'diesel engine'): {}
... }
... }
>>> hierarchicalVocabs = {}
>>> hierarchicalVocabs[('enginetypes', 'Diffetent types of engines')] = dictionary
>>> createHierarchicalVocabs(atvm, hierarchicalVocabs)

now we've got our vocabulary available::

>>> engines = atvm.getVocabularyByName('enginetypes')
>>> engines
<TreeVocabulary at /plone/portal_vocabularies/enginetypes>

Background Information

This small section tells you how ATMV achieves that an ottomotor is
recognized as a mechanical engine too.

The key of a vocabulary term is the UID of the canonical object
(in case the vocabulary is translated into different languages)
and can be obtained using the method ``getTermKey``

>>> ottomotor = engines.mechanical.otto
>>> ottomotor.getTermKey() == ottomotor.UID()

A ``TreeVocabularyTerm`` implements the method ``getTermKeyPath``
that returns a list of vocabularyterm-keys of the term itself
and all the terms in the hierarchy above it.

>>> ottoPath = ottomotor.getTermKeyPath()
>>> engines.mechanical.getTermKey() in ottoPath

Each VocabularyTerm is indexed using a KeywordIndex on
``getTermKeyPath`` in uid_catalog.

The keypath is is available as a metadata-column in uid_catalog,
so it can be fetched for via a catalog query without invoking real objects.

our ottomotor term can be found searching for the ottomotor uid::

>>> uid_catalog = getToolByName(self.portal, 'uid_catalog')
>>> uid_catalog(getTermKeyPath=ottomotor.UID())[0].getObject()
<TreeVocabularyTerm at .../enginetypes/mechanical/otto>

a search for a machanical engine returns 3 vocabularies:
the ottomotor, the dieselengine and the term for mechanicalengine too::

>>> result = uid_catalog(getTermKeyPath=engines.mechanical.UID())
>>> [ for brain in result]
['mechanical', 'diesel', 'otto']

a change in the hierarchy of our vocabulary will reindex all vocabularies
below the changed term.

XXX make new term below engines: 'fossil'' and move
mechanical there. after that the otto and diesel
should be available under fossil too

Make your content type searchable

To be able to search for content types assiciated to terms
not only directly but also down the hierarchy we need to index
our object under the TermKeyPath(s) of the associated vocabularyterm(s).

ATVM provides a fast implemented utility method for this purpose
that is working only on the catalog.

You can utilize ``getKeyPathForTerms`` in ``NamedVocabulary``::

>>> from Products.ATVocabularyManager import NamedVocabulary
>>> nv = NamedVocabulary('engines')
>>> keyPath = nv.getKeyPathForTerms(self.portal, ottomotor.getTermKey())
>>> engines.mechanical.getTermKey() in keyPath
>>> ottomotor.getTermKey() in keyPath

All you need to do is to implement a method that uses the
``getKeyPathForTerms`` method and define it as index method::

# .. somewhere in the schemadefinition


# .. somewhere in your content type definition

define someIndexMethod(self):
"""used to index myfield
# all we have to know is the field name of the field this
# index method belongs to
fieldName = 'myfield'

# the following code need not be touched
termUID = self.getField(fieldName).get(self)
return self.getField(fieldName).vocabulary.getKeyPathForTerms(self, termUID)

See ATVocabularyManagerExample_ for a more detailed explanation how
to use ATVocabularyManager in your content type.



1.6.2 (2012-03-22)

- Avoid breaking the import of *.zexp-files when an vocabularies don't exist
anymore, but instantiated object still exist in the import-file.

- Added field 'showTermPath' for vdex vocabularies, to manually turn of the
term path. [jensens] [hpeteragitator]

- Added field 'showLeafsOnly' for vdex vocabularies, to change if only the leaf
of the vocabulary tree can be selected. [jensens] [hpeteragitator]

- Refactored getDisplayList for vdex vocabularies [jensens] [hpeteragitator]

- Run i18ndude and updated german translations [hpeteragitator]

- Let createSimpleVocabs accept dictionary keys as tuples of (id, title) (like
createHierarchicalVocabs already does), to enable one to give the vocabulary
a title.

1.6.1 (2011-05-02)

- Avoid deprecation warnings [WouterVH]

- Update metadata-files and folder-structure to current conventions. [WouterVH]

- Add `z3c.autoinclude` entry point for automatic ZCML loading in Plone 3.3+.

- Add to include top-level \*.txt-files. [WouterVH]

1.6 - 2011-02-22

- Don't fetch version and shortdesc from metadata.xml - this is wrong. [jensens]

1.6.0a2 - 2011-02-18

- Changed method of dermining if LinguaPlone is installed. Asking QI does not
work in Plone4. [naro]

- Added some missing dependencies, and a new extra section to
This helps to get a simple buildout cfg for testing against plone4. [do3cc]

- Added some sleeps between installs in the tests, else genericsetup
creates ids twice, and fails. [do3cc]

- Temporary deactivated two checks in the tests. With
Products.CMFQuickInstaller>=3.0.3, the portal_vocabularies don't get
deleted during uninstall. [do3cc]

- Restructured test setup, since call semantics changed. [do3cc]

1.6.0a1 - 2010-07-16

- Fixed deprecated API call to registerType without Package Name. Works now for
Plone4. [thet]

- Added Products.LinguaPlone as test dependency to extras_require and begun
fixing tests. [thet]

- Removed old slow VDEX implementation; added stub for import/export of vocabs
with generixsetup; added support for vdex import with genericsetup. [jensens]

1.4.2 - 2008-06-04

- Added spanish translations provided by Diego Municio

- Merged in Duke's patch for multilingual csv import for simple vocabularies.
Cfr. [fRiSi]

- Fixed test setup so ATVM is set up using the new GS profile added by naro.
(tested with plone 3.1.1 buildout)
this fixes

- Got rid of tests.framework. [fRiSi]

- Added imsvdex as a required dependency in which makes it unnecessary
to ``easy_install imsvdex`` for people using buildout. [fRiSi]

- Added GenericSetup profile, removed [naro]

before 1.4

- Workaround for #46 Upload from Windows/IE creates wrong ID. [jensens]

- Made new VdexFileVocabulary as default for upload form. [jensens]

- Add alternative vdex vocabulary, much faster. [jensens]

- Added convinience method to vocabulary library (tool) which returns
a dictionary with key=vocbaulary id and value=title of vocabulary (language
aware. [jensens]

- Added convinience function to make it easier to fetch a value from a
vocabulary dictionary (utils.vocabs.fetchValueByKeyFromVocabularyDict). [jensens]

- Adding css handles to various elements of portal_vocabulary user
interface, so that if desired, one can easily dumb down the user
interface for less technical clients with css, but not override the template

- Language negotation to make sure the cache is language aware. [jensens]

- In types/simple/, applied patch of issue #4
(sorting of simple vocabularies) [fRiSi]

- In doc/simplevocabulary.txt, add doctest testing the sorting of simplevocabularies
sorting might not yet work with translated vocabularies. I added a section to
test this, but did not yet write the tests there. [fRiSi]

-, configure.zcml,, types/simple/ [rocky]

- "renamed events" are now fired when the title of any term is updated. [rocky]

- "deleted events" are now fired when a term is deleted from a
vocabulary; this particular feature will only work when running on
Zope 2.9 or higher (note: this doesn't make ATVM dependent on Zope 2.9)

- types/tree/, types/simple/
replaced ``if not instance is None:`` with a try-except.
if you are on a type that is not linguaplone enabled,
and using a vocabulary you got an error when creating an
object of this type because getLanguage raised an AttributeError. [fRiSi]

- types/tree/
to fix
in the meantime i solved the problem with the "hack solution"
to take away the IVocabulary interface from the term that is
extending the vocabulary. [fRiSi]
(see the issue above for more information.)

- Makefile
added a new target `clean` that removes stale pyc files and ``-~`` backup
files [fRiSi]

- used to not add the keypath of a term if the search for a
term uid in uid-catalog returned more than one value. proceeding and just taking
the first result is better than silently ignoring the path for this item. [fRiSi]

- In Extensions/, did a cleaner check for BadRequest Error in case the tool already
existed at install time (also added basic tests for installing/uninstalling/and
reinstalling the product. [fRiSi]

- In Extensions/, self.portal.portal_properties.navtree_properties.metaTypesNotToList
is a tuple by default (on a freshly created plonesite)
ATVM creates a list at installation time and removes the tool from this list at
uninstallation. If other products get installed between installing
and uninstalling, they possibly change metaTypesNotToList back to a tuple again.
changed the installer and uninstaller to operate on and return a tuple here.
(should fix [fRiSi]

- tests/
tests for the installation and uninstallation of the product.
agreed with jens that we could do a zexp of the vocabularies at uninstallation
in case this was done by accident.
( [fRiSi]

- types/simple/ removed bug, which stops me from adding new terms. [jensens]

- In types/simple/, SimpleVocabularyTerms now return the canoncial's id
as key. This differs from the current behaviour but is essential for translated
vocabularies. [fRiSi]

- In types/simple/,getTermKeyPath now returns a list containing the key of
the term, not it's uid. This is consistent with the interface. [fRiSi]

- types/simple/, types/tree/
GetVocabularyDict now uses instance.getLanguage in favour of
portal_lanuages.getPreferredLanguage (otherwhise passing instance would be
useless in most cases)

- tests/ Added a test that checks if translations return
the same key and if dictionaries are translated.


- Improved the docstrings to be more explicit.

- getTermKey now states that all translations of a term
have the same key.

- getTermValue: now has an optional ``lang`` parameter
making ``--kwargs`` needless.

- Add sorted simple vocabulary [gotcha]

- Run i18ndude and update fr and nl [gotcha]

- Merged branch supporting catalog searches on hierarchical vocabularies to trunk

- Merged current trunk into this branch. [fRiSi]

- Utils/
utility methods for batch creating vocabularies.[fRiSi]

- types/simple/ types/tree/
Marked methods getVocabularyValue and getVocabularyKey as deprecated and used those
defined in IVocabularyTerm to make terms implement their interface. [fRiSi]

- types/tree/ types/simple/
Improved linguaplone support and also added a testcase. [fRiSi]

- tests/
Testing if treevocabulary is linguaplone aware. [fRiSi]

- doc/search_treevocabulary.txt: A doctest explaining how ATVM performs searches
on hierarchical vocabularies, and how to enable them for your content type.

- Added dutch translations, thanks to Atopia. [jladage]

- AllowedTypesByIface is broken, it seems - adding manual override to fix this
in ATVM. [optilude]

- Removed workflows from ATVM types on installation. [ferri]

- Added brazilian portuguese translation and portal_vocabularies title i18n
support. [ferri]

- Added i18n and Czech translation. [naro]

- Uncommented code in to make portal_vocabularylib not show in the
navigation tree. [panjunyong]

- Use StringWidget instead of IDWidget for Simple vocaublary term. [panjunyong]

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution (146.0 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page