Skip to main content

indexer addons: sort, cjksplitter, and so on

Project description

zapian: schemaless python interface to Xapian
===============================================

作为一个pythoner,我们有理由爱xapian...

但xappy已年久失修过于陈旧了... 喜欢elasticsearch的api,但是憎恨Luence的java架构,不愿引入新的服务进程?

那么zapian,可能是你需要的....

欢迎拍砖: http://weibo.com/panjunyong

特性
-------------------------------------

- 为xapian提供更友好的schemaless的api
- 支持分区索引:可单独指定分区搜索,或合并搜索

- 历史数据存放在不同的索引分区
- 根据数据存放区域进行分区

Schemaless API
-------------------------------------

首先需要初始化数据库:

db = Zapian(path='/tmp/test_zapian_db')

添加一个分区:

db.add_part('2001-02')

添加索引:

db.add_document(part='2001-02',
uid='1111',
index = { '+title' : u'我们很好.doc',
'searchable_text' : u'',
'modified' : datetime.datetime(),
'crated' : datetime.datetime()
},
data = {}
)

修改索引:

db.replace_document(part, uid, doc)

删除索引:

db.delete_document(part, uid)

搜索:

db.search(parts, ["and",
{ "filters":
"exclude":
},

[ "or",
{"filters":
"exclude": },
{ "filters":
"exclude": }
]
]
)


doc和索引的关系
-------------------------------------

xapian内部对数据有三种用途:term索引、排序字段、返回data;系统自动对数据类型进行处理:

- set/list/tuple:对每个包含数据,完整匹配搜索(term索引)
- string/unicode: 用于全文搜索(term索引)
- datetime/int/float: 用于排序和范围比较(排序字段)
- 如果字符串类型的字段以 + 开头,表示除了全文索引,也会用于排序

数据库的结构
-------------------------------------

数据库存放在文件夹结构:

schema.yaml # 库结构信息
20120112/ # 某个分区,标准xapian数据库
20120512/ # 另外一个分区,标准xapian数据库

其中schema.json, 由系统自动维护, 记录了2个信息:

1. PREFIX和字段的映射:

prefixes:{title':"NC", 'created':"LL"}

2. attribute存放的slot位置:

slots:{'modified':1, 'created':2}

安装使用
-------------------------------------

1. 需要先安装xapian:http://xapian.org/download
2. 直接在pypi上安装这个包: https://pypi.python.org/pypi/zapian

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

zapian-0.3.0dev.tar.gz (17.2 kB view details)

Uploaded Source

File details

Details for the file zapian-0.3.0dev.tar.gz.

File metadata

  • Download URL: zapian-0.3.0dev.tar.gz
  • Upload date:
  • Size: 17.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for zapian-0.3.0dev.tar.gz
Algorithm Hash digest
SHA256 442b0d6fad347a78a45b60075600eaf95189eb586cf5caecd2c420c698deef90
MD5 c24fdcee096aa655a5e77500ba3cefe0
BLAKE2b-256 7454341b769ec71ffc75fd7ccfbc3c054ee7dcd1d71b158e81cc355d9a889288

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page