Fast and memory efficient DAWG for Python
This package provides DAWG(DAFSA)-based dictionary-like read-only objects for Python (2.x and 3.x).
String data in a DAWG may take 200x less memory than in a standard Python dict and the raw lookup speed is comparable; it also provides fast advanced methods like prefix search.
Issue tracker: https://github.com/kmike/DAWG/issues
- Fixed a bug in DAWG building: input should be sorted according to its binary representation.
- Wrapper is rebuilt with Cython 0.21dev;
- Python 3.4 compatibility is verified.
- has_keys_with_prefix(prefix) method (thanks Matt Hickford)
- Extension is rebuilt with Cython 0.19.1;
- fixed segfault that happened on lookup from incorrectly loaded DAWG (thanks Alex Moiseenko).
- Installation issues in environments with LC_ALL=C are fixed;
- PyPy is officially unsupported now (use DAWG-Python with PyPy).
- many thread-safety bugs are fixed (at the cost of slowing library down).
- fix installation under PyPy (note: DAWG is slow under PyPy and may have bugs).
- small tweaks for docstrings;
- the extension is rebuilt using Cython 0.18.
- small improvements to .compile_replaces method;
- benchmarks for .similar_items method;
- the extension is rebuilt with Cython pre-0.18; this made .prefixes and .iterprefixes methods faster (up to 6x in some cases).
- tests are included in source distribution;
- benchmark results in README was nonrepresentative because of my broken (slow) Python 3.2 install;
- installation is fixed under Python 3.x with LC_ALL=C (thanks Jakub Wilk).
- better error reporting while building DAWGs;
- __contains__ is fixed for keys with zero bytes;
- dawg.Error exception class;
- building of BytesDAWG and RecordDAWG fails instead of producing incorrect results if some of the keys has unsupported characters.
The storage scheme of BytesDAWG and RecordDAWG is changed in this release in order to provide the alphabetical ordering of items.
This is a backwards-incompatible release. In order to read BytesDAWG or RecordDAWG created with previous versions of DAWG use payload_separator constructor argument:
- Segfaults with empty DAWGs are fixed by updating dawgdic to latest svn.
- iterkeys, iteritems and iterprefixes methods (thanks Dan Blanchard).
- prefixes method for finding all prefixes of a given key.
- bundled dawgdic C++ library is updated to the latest version.
- similar_keys, similar_items and similar_item_values methods for more permissive lookups (they may be useful e.g. for umlaut handling);
- load method returns self;
- Python 3.3 support.
Greatly improved memory usage for DAWGs loaded with load method.
There is currently a bug somewhere in a wrapper so DAWGs loaded with read() method or unpickled DAWGs uses 3x-4x memory compared to DAWGs loaded with load() method. load() is fixed in this release but other methods are not.