A tiny NoSQL database with in-memory caching
Project description
Tinyble
========
Introduction
--------
Tinyble is a super lightweight database inheriting `TinyDB`_. It is a document-oriented NoSQL but can also be used as a SQL database. It is most suitable for small applications where MongoDB is too way overkill.
The features are:
.. _TinyDB: https://github.com/msiemens/tinydb
- Written in pure Python and works well with most Python versions
- Very small, stored in JSON format and requires no external server
- Different from TinyDB, flexible auto-paging is provided to enable fast write-in and read-out
- Combine in-memory caching with disk storage to trade-off speed and reliability
Difference from TinyDB
---------
TinyDB stores each table in a single JSON file. For every IO operation, the JSON file will be wholly read and rewritten. As the file size gets larger or when there are many write-in operations, the speed can be dramatically slow.
Tinyble uses a separate thread to manage all the write/update operations. Specifically, all newly-updated data will first be stored in memory and an internal timer expires, they will be batch-written in the JSON file.
This reduces the number of IO operations. Moreover, a limit on the JSON file size is imposed when the size exceeds the limit, a new JSON file is created. This limits the volume of each read operation.
The internal timer and the file size can be set by users.
Version
--------
The latest version is v0.1.4.3. The current version is still on experimental status. Please use it with caution. Please contact me if you find any bugs/problems.
v0.1.4.3(07.03.2015)
^^^^^^
- added the close() function
- added the projection (partial search) function
- added the rst file
v0.1.3(04.03.2015)
^^^^^
- fixed the print bug
- fixed the 'get' method bug
How to install
--------
The easiest way to install is to use
::
(sudo) pip install tinyble
in the command line tool.
Working with Django
-------
A very simple and ugly example showing the use of Tinyble with Django can be found in `Tinyble with Django`_
.. _Tinyble with Django: https://github.com/StevenSLXie/django_with_tinyble
Example
-------
Create a new database and a new collection
^^^^^
::
db = Tinyble('data')
collection = db.collection('example')
Insert some data
^^^^^
::
col = db.collection('example')
col.insert({'type': 'apple', 'number': 1})
col.insert({'type': 'pineapple', 'number': 2})
col.insert({'type': 'blueberry', 'number': 3})
Update some data
^^^^^
::
col.update({'number':10}, cond= where('type')=='pineapple')
Delete some data
^^^^^
::
col.remove(eids=[3])
col.remove(cond=where('number')<4)
Other usages are very similar to TinyDB.
Set the parameters
^^^^^
::
col.setting(file_size=100, query_cache_size=10, write_freq=5)
The above setting means each JSON file is limited to 100 entries and the database will store the latest 10 query results in memory and write-to-disk frequency is 5s.
Select some data
^^^^^^
::
col.search(where('number')>2)
Select part of data in a document
^^^^^^^
::
col.search(where('type')!= 'pineapple', ['number'])
This will only show the 'number' key, not the whole dictionary.
========
Introduction
--------
Tinyble is a super lightweight database inheriting `TinyDB`_. It is a document-oriented NoSQL but can also be used as a SQL database. It is most suitable for small applications where MongoDB is too way overkill.
The features are:
.. _TinyDB: https://github.com/msiemens/tinydb
- Written in pure Python and works well with most Python versions
- Very small, stored in JSON format and requires no external server
- Different from TinyDB, flexible auto-paging is provided to enable fast write-in and read-out
- Combine in-memory caching with disk storage to trade-off speed and reliability
Difference from TinyDB
---------
TinyDB stores each table in a single JSON file. For every IO operation, the JSON file will be wholly read and rewritten. As the file size gets larger or when there are many write-in operations, the speed can be dramatically slow.
Tinyble uses a separate thread to manage all the write/update operations. Specifically, all newly-updated data will first be stored in memory and an internal timer expires, they will be batch-written in the JSON file.
This reduces the number of IO operations. Moreover, a limit on the JSON file size is imposed when the size exceeds the limit, a new JSON file is created. This limits the volume of each read operation.
The internal timer and the file size can be set by users.
Version
--------
The latest version is v0.1.4.3. The current version is still on experimental status. Please use it with caution. Please contact me if you find any bugs/problems.
v0.1.4.3(07.03.2015)
^^^^^^
- added the close() function
- added the projection (partial search) function
- added the rst file
v0.1.3(04.03.2015)
^^^^^
- fixed the print bug
- fixed the 'get' method bug
How to install
--------
The easiest way to install is to use
::
(sudo) pip install tinyble
in the command line tool.
Working with Django
-------
A very simple and ugly example showing the use of Tinyble with Django can be found in `Tinyble with Django`_
.. _Tinyble with Django: https://github.com/StevenSLXie/django_with_tinyble
Example
-------
Create a new database and a new collection
^^^^^
::
db = Tinyble('data')
collection = db.collection('example')
Insert some data
^^^^^
::
col = db.collection('example')
col.insert({'type': 'apple', 'number': 1})
col.insert({'type': 'pineapple', 'number': 2})
col.insert({'type': 'blueberry', 'number': 3})
Update some data
^^^^^
::
col.update({'number':10}, cond= where('type')=='pineapple')
Delete some data
^^^^^
::
col.remove(eids=[3])
col.remove(cond=where('number')<4)
Other usages are very similar to TinyDB.
Set the parameters
^^^^^
::
col.setting(file_size=100, query_cache_size=10, write_freq=5)
The above setting means each JSON file is limited to 100 entries and the database will store the latest 10 query results in memory and write-to-disk frequency is 5s.
Select some data
^^^^^^
::
col.search(where('number')>2)
Select part of data in a document
^^^^^^^
::
col.search(where('type')!= 'pineapple', ['number'])
This will only show the 'number' key, not the whole dictionary.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tinyble-0.1.4.4.tar.gz
(10.9 kB
view hashes)