Products.ZCatalog 3.0.1

Zope 2's indexing and search solution.

Overview

The ZCatalog is Zope’s built in search engine. It allows you to categorize and search all kinds of Zope objects.

It comes with a variety of indexes for different types of data.

Changelog

3.0.1 (2013-10-15)

• Fix BooleanIndex when index inversion occurs as a result of reindexing and existing document with the opposite value. Fixes LP #1236354.

3.0 (2013-02-24)

• Strip white space from name when adding a column or index.

• Forward compatibility for Zope 4 removal of RequestContainer.

• Optimize brain instantiation, by creating underlying record items in a single step, instead of creation and three update calls.

3.0b1 (2012-07-19)

• LP #727981: Fix DateIndex ZMI browsing for dates in the first month of a year.

• Unify Unindex and DateIndex search logic (_apply_index) adding not support to DateIndexes.

3.0a2 (2012-04-26)

• Fixed another issue with preserving score values, when a custom index was queried first which was neither ILimitedResultIndex aware nor return scores, and a later index was of the default ZCTextIndex type.

3.0a1 (2012-04-22)

• Expand query report, to cover details on sort indexes, order and limits.

• As part of each progress handler report, also do an automatic transaction savepoint, to give the ZODB cache a chance to do garbage collection.

• Added a threshold argument to the catalog’s addColumn and delColumn methods and used it for a progress handler. Also optimized some of their internals.

• Added support for sort_on queries with any number of sort indexes and differing sort_order values. For example: {‘foo’: ‘a’, ‘sort_on’: (‘foo’, ‘bar’)} {‘foo’: ‘a’, ‘sort_on’: (‘foo’, ‘bar’), ‘sort_order’: (‘’, ‘reverse’)} {‘foo’: ‘a’, ‘sort_on’: (‘foo’, ‘bar’, ‘baz’)}

• Added support for not queries in field and keyword indexes. Both restrictions of normal queries and range queries are supported, as well as purely exclusive queries. For example: {‘foo’: {‘query’: [‘a’, ‘ab’], ‘not’: ‘a’}} {‘foo’: {‘query’: ‘a’, ‘range’: ‘min’, ‘not’: [‘a’, ‘e’, ‘f’]}} {‘foo’: {‘not’: [‘a’, ‘b’]}}. Note that negative filtering on an index still restricts items to those having a value in the index. So with 10 documents, 5 of them in the foo index with a value of 1, a query for not 1 will return no items instead of the 5 items without a value. You need to index a dummy/default value if you want to consider all items for a particular index.

• Updated deprecation warnings to point to Zope 4 instead of 2.14.

2.13.22 (2011-11-17)

• Added a new load_from_path class method to the PriorityMap, which allows one to load a plan from a file, instead of a module via an environment var.

2.13.21 (2011-10-20)

• Refactored value index logic. Determine value indexes per catalog instead of globally. Store value index set in the priority map, so it can be seen in the ZMI and stored in the module level storage.

• Added support for using ZCatalog as local utility. This feature requires the optional five.globalrequest dependency.

2.13.20 (2011-08-23)

• Fixed incorrect calculation of batches in the second half of the result set in sortResults.

2.13.19 (2011-08-20)

• Increase plan precision to 4 digits in its string representation.

2.13.18 (2011-07-29)

• In the string representation of a catalog plan, round the times to at most two digits after the comma.

2.13.17 (2011-07-29)

• Put back the weightedIntersection optimization but guard against results with values and do the appropriate fallback to the weighted version.

2.13.16 (2011-07-24)

• Restored preserving score values from ZCTextIndex indices. https://bugs.launchpad.net/zope2/+bug/815469

2.13.15 (2011-06-30)

• Fixed undefined variables in BooleanIndex inline migration code.

• Fixed BooleanIndex’ items method so the ZMI browse view works.

2.13.14 (2011-05-19)

• Fixed addition of two LazyCat’s if any of them was already flattened.

• Extend BooleanIndex by making the indexed value variable instead of hardcoding it to True. The indexed value will determine the smaller set automatically and choose its best value. An inline switch is done once the indexed value set grows larger than 60% of the total length. 60% was chosen to avoid constant switching for indexes that have an almost equal distribution of True/False.

• Substitute catalog entry in UUIDIndex error message.

2.13.13 (2011-05-04)

• Optimize Catalog.updateMetadata avoiding a self.uids lookup and removing inline migration code for converting self.data from non-IOBTree types.

• In the path index, don’t update data if the value hasn’t changed.

2.13.12 (2011-05-02)

• Optimize DateRangeIndex for better conflict resolution handling. It always starts out with storing an IITreeSet of the value instead of special casing storing an int for a single value. The single value as int optimization should be provided via a separate API to be called periodically outside the context of a normal request.

• Replaced weightedIntersection and weightedUnion calls with their non-weighted version, as we didn’t pass in weights.

2.13.11 (2011-05-02)

• Fix possible TypeError in sortResults method if only b_start but not b_size has been provided.

• Prevent the new UUIDIndex from acquiring attributes via Acquisition.

2.13.10 (2011-04-21)

• Handle TypeErrors in the KeywordIndex if an indexed attribute is a method with required arguments.

• Added reporting of the intersection time of each index’ result with the result set of the other indexes and consider this time to be part of each index time for prioritizing the index.

• Removed tracking of result length from the query plan. The calculation of the length of an intermediate index result can itself be expensive.

2.13.9 (2011-04-10)

• Added a floor and ceiling value to the date range index. Values outside the specified range will be interpreted the same way as passing None, i.e. since the beginning of time and until the end of it. This allows the index to apply its optimizations, while objects with values outside this range can still be stored in a normal date index, which omits explicitly passed in None values.

2.13.8 (2011-04-01)

• Fixed bug in date range index, which would omit objects exactly matching the query term if a resultset was provided.

• Fixed the BooleanIndex to not index objects without the cataloged attribute.

2.13.7 (2011-02-15)

• Fixed the DateIndex._unindex to be of type IIBTree instead of OIBTree. It stores document ids as keys, which can only be ints.

2.13.6 (2011-02-10)

• Remove docstrings from various methods, as they shouldn’t be web-publishable.

2.13.5 (2011-02-05)

• Fixed test failures introduced in 2.13.4.

2.13.4 (2011-02-05)

• Added a new UUIDIndex, based on the common UnIndex. It behaves like a FieldIndex, but can only store one document id per value, so there’s a 1:1 mapping from value to document id. An error is logged if a different document id is indexed for an already taken value. The internal data structures are optimized for this and avoid storing one IITreeSet per value.

• Optimize sorting in presence of batching arguments. If a batch from the end of the result set is requested, we internally reverse the sorting order and at the end reverse the lazy sequence again. In a sequence with 100 entries, if we request the batch with items 80 to 90, we now reverse sort 20 items (100 to 80), slice of the first ten items and then reverse them. Before we would had to sort the first 90 items and then slice of the last 10.

• If batching arguments are provided, limit the returned lazy sequence to the items in the required batch instead of returning leading items falling outside of the requested batch.

• Fixed inline IISet to IITreeSet conversion code inside DateRangeIndex’ _insertForwardIndexEntry method.

2.13.3 (2011-01-01)

• Avoid locale-dependent test condition in test_length_with_filter.

2.13.2 (2010-12-31)

• Preserve actual_result_count on flattening nested LazyCat’s.

• Preserve the actual_result_count on all lazy return values. This allows to get proper batching information from catalog results which have been restricted by sort_limit.

• Made sure actual_result_count is available on all lazy classes and falls back to __len__ if not explicitly provided.

• Optimized length calculation of Lazy classes.

2.13.1 (2010-12-25)

• Added automatic sorting limit calculation based on batch arguments. If the query contains a b_start and b_size argument and no explicit sort_limit is provided, the sort limit will be calculated as b_start + b_size.

• Avoid pre-allocation of marker items in LazyMap.

2.13.0 (2010-12-25)

• Fix LazyMap to avoid unnecessary function calls.

• Released as separate distribution.