Skip to main content

Easy to use 👋 fast & lightweight ☄️ secure 🔐 JSON based database for python! 🐍.

Project description

logo

Datagoose

Datagoose is an easy to use JSON based database for python.

Version Downloads Stars Commits

With Datagoose:

  • Best performance. Datagoose is a lightweight database.
  • Methods that makes your job easier.
  • Regex supports.
    • You must enable USE_REGEX option.
  • Safe to use.
  • Auto or manual save, for who wants better performance.
  • Easy to use database. Created for everyone.
  • Rich options. includes hash keys, database path, regexp and more.
  • Auto backup
  • Can be dumped, also can load a data from JSON file.
    • <Datagoose>.load()
    • <Datagoose>.dump()

Update (1.6.0)

  • Added PIN option for encryption.
    • If you used 1.5.0 encryption, use PIN 2 or dump your database and make a new one.
  • Added .replace, .replace_one methods.
  • Added 3 new event.
  • Added ENCRYPTED option for auto-backup.
  • Removed .save_with_indent method.

Download

You can download with pip install -U datagoose (PyPi Page) or, you can use with source code.

Note: Current stable version is 1.6.1. You can download stable version with pip install datagoose==1.6.1 (PyPi).

Quick Documentation

Before Start

You should know these things before using datagoose;

  • Datagoose keeps data in memory, not in a file.

    • You can save with <Datagoose>.save() for remember the database next time.
    • Also now you can enable AUTO_SAVE option for auto-saving.
  • Datagoose is not a professional database.


Performance

Test Result (Auto-Save Enabled):

  • 100 Data with insert many:
    • Starting Insert...
      Inserting Finished in  0:00:00.007002
      
  • 1,000 Data with insert many:
    • Starting Insert...
      Inserting Finished in  0:00:00.032004
      
  • 10,000 Data with insert many:
    • Starting Insert...
      Inserting Finished in  0:00:00.278020
      
  • 100,000 Data with insert many:
    • Starting Insert...
      Inserting Finished in  0:00:02.808687
      
  • 1,000,000 Data with insert many:
    • Starting Insert...
      Inserting Finished in  0:00:31.908481
      

Methods

Quick Start

[] - Optinal
[argument_name:argument_type:argument_default_value] - Optinal Argument Name | Optional Argument Type | Optional Argument Default Value

{} - Normal
{argument_name:argument_type} - Argument Name | Argument Type

* - Args
from datagoose import Datagoose

database = Datagoose("example")

Options

# Options must be a dict. lets get informations about current options.

# PATH:
  # Type: String
  # Description: Custom path for JSON file.
  # Note: please add path like: "databases/datagoose" not "./databases/datagoose/"
  # Default: datagoose_files
# AUTO_SAVE:
  # Type: Bool
  # Description: When enabled, auto-saves the database when an action performed.
  # Default: False
# HASHING:
  # Type: List
  # Description: Key list for replace data with sha256 hash when inserted.
  # Default: []
# SAFE_MODE:
  # Type: Bool
  # Description: Enable/Disable safe mode. do not recommending to disable this option. if you know what are you doing, then good luck. 
  # Default: True
# USE_REGEX:
  # Type: Bool
  # Description: Enable/Disable regex option. 
  # Default: False
# ENCRYPTED:
  # Type: Bool
  # Description: Enable/Disable encrypting data. 
  # Default: False
# PIN:
  # Type: Int
  # Description: Decryption key for encryption. you should not expose this key.
  # Default: 2

# Example:
database = Datagoose("example", {
    "AUTO_SAVE": True,
    "USE_REGEX": True,
    "ENCRYPTED": True,
    "PIN": 69 * 420,
    "HASHING": [
        "PASSWORD"
    ]
})
# <Datagoose>.read() -> Returns the entire data
  # Return Type: List
  # Example(s):
    full_database = database.read()

# <Datagoose>.info -> Returns database info
  # Return Type: Dict
  # Example(s):
    print(database.info)

# <Datagoose>.length -> Returns total data count
  # Return Type: Integer
  # Example(s):
    print(database.length)

# <Datagoose>.uptime -> Returns database uptime
  # Return Type: datetime.timedelta
  # Example(s):
    print(database.uptime)

# <Datagoose>.save() -> Saves the database
  # Return Type: Bool
  # Example(s): 
    database.save()

# <Datagoose>.clear() -> Clears the entire database
  # Return Type: Bool
  # Example(s): 
    database.clear()

Inserting Data

# <Datagoose>.insert_one({data:dict}) -> Insert one data to database
  # Return Type: Dict
  # Argument: data
    # Description: The data will be inserted into database.
  # Example(s):
    database.insert_one({
        "name": "John",
        "lastname": "Doe",
        "age": 25,
        "city": "New York",
        "country": "USA",
        "job": "Doctor"
    })

# <Datagoose>.insert_many(*{data:dict}) -> Insert many data to database
  # Return Type: Bool
  # Args:
    # Description: The data(s) will be inserted into database.
  # Example(s):
    database.insert_many({
        "user": 1,
        "nickname": "normal_guy_100"
    }, {
        "user": 2,
        "nickname": "user_555"
    })

Note: Datagoose adds unique hash ("_id") to every document that inserted. Make sure do not add same "_id" to documents.

# Example

database.insert_one({ "_id": 1, "name": "an_user" })
database.insert_one({ "_id": 1, "name": "another_user" })
# This example will not give an error. But when you use delete or update, datagoose may confuse while find the object.

Finding Data

# <Datagoose>.find({data:dict}) -> Find data from database
  # Return Type: Generator? (Dict Yield)
  # Argument: data
    # Description: The data will find from database.
  # Example(s):
    found = database.find({
        "age": 25
    }) # -> {'name': 'test', 'age': 25}, {'name': 'test2', 'age': 25} ...

    for result in found:
      print(result)

# <Datagoose>.query({check:function}) -> Query data from database with function. *New in 1.4.1*
  # Return Type: Generator? (Dict Yield)
  # Argument: check
    # Description: Functions for check, must return bool.
  # Example(s):
    for i in database.query(lambda data: 'POINT' in data and data['POINT'] >= 1337):
      print(i)

# <Datagoose>.find_one({data:dict}) -> Find one data from database
  # Return Type: Dict
  # Argument: data
    # Description: The data will find from database.
  # Example(s):
    result = database.find_one({
        "user_id": 295818619
    })
    print(result["nickname"])

Updating Data

# <Datagoose>.update({data:dict}, {new_data:dict}) -> Update data from database
  # Return Type: List
  # Argument: data
    # Description: The data will find from database.
  # Argument: new_data
    # Description: The data will be changed with found data.
  # Example(s):
    database.update({
        "age": 25
    }, {
        "age": 26
    }) # Now every data has 'age' and 'age' key value is 25, changed with 'age' = 26 
    
# <Datagoose>.update_one({data:dict}, {new_data:dict}) -> Update one data from database
  # Return Type: Dict
  # Argument: data
    # Description: The data will find from database.
  # Argument: new_data
    # Description: The data will be changed with found data.
  # Example(s):
    database.update_one({
        "user_id": 2486718956
    }, {
        "banned": True,
        "ban_time": "30d"
    }) # Now only one data updated.

Note: When you add new key and value to new_data dictionary, it will insert into data.

# Example

database.insert_one({ "_id": 1, "name": "an_user" })
database.update_one({
    "_id": 1
}, {
    "total_kill": 16
})
# Now our data is:
{
    "_id": 1,
    "name": "an_user",
    "total_kill": 16
}

Replace Data

# <Datagoose>.replace({data:dict}, {new_data:dict}) -> Replace data from database
  # Return Type: List
  # Argument: data
    # Description: The data will find from database.
  # Argument: new_data
    # Description: The data will be replaced with found data.
  # Example(s):
    database.replace({
        "age": 25
    }, {
        "thing": True, 
        "age": 26
    }) # Now every data has 'age' and 'age' key value is 25, changed with new data.
    
# <Datagoose>.replace_one({data:dict}, {new_data:dict}) -> Replace one data from database
  # Return Type: Dict
  # Argument: data
    # Description: The data will find from database.
  # Argument: new_data
    # Description: The data will be replaced with found data.
  # Example(s):
    database.replace_one({
        "user_id": 2486718956
    }, {
        "banned": True,
        "ban_time": "30d",
        "user_id": 2486718956
    }) # Now only one data replaced.

Note: The difference between update and replace, replace changes the entire data with new one. update only updates the key that given.

Deleting Data

# <Datagoose>.delete({data:dict}) -> Delete data from database
  # Return Type: List
  # Argument: data
    # Description: The data will be deleted from database.
  # Example(s):
    database.delete({
        "total_hours": 1
    }) # Now datagoose deleted all data contains key 'total_hours' and key value is 1

# <Datagoose>.delete_one({data:dict}) -> Delete one data from database
  # Return Type: Dict
  # Argument: data
    # Description: The data will be deleted from database.
  # Example(s):
    database.delete_one({
        "user_id": 6811
    })

Count & Exists

# <Datagoose>.count({data:dict}) -> Count all data contains data argument
  # Return Type: Integer
  # Argument: data
    # Description: The data will count.
  # Example(s):
    database.count({
        "name": "John",
        "age": 40
    }) # -> 6157

# <Datagoose>.exists({data:dict}) -> Checks data exists
  # Return Type: Bool
  # Argument: data
    # Description: The data will be checked.
  # Example(s):
    result = database.exists({
        "name": "a_guy_11"
    })

    if result:
        database.delete_one({ "name": "a_guy_11" })

Note: "has" is an alias for exists.

# Example

result = database.has({
    "name": "a_guy_11"
})

if result:
    database.delete_one({ "name": "a_guy_11" })

# is same with:

result = database.exists({
    "name": "a_guy_11"
})

if result:
    database.delete_one({ "name": "a_guy_11" })

Using Regex

# Quick example for using regex in datagoose.
# You must enable USE_REGEX option for using regex in datagoose.

for i in db.find({"ANSWER": r"yes|y"}):
    print(i)

Load & Dump

# <Datagoose>.dump({location:str}, [encoding:str:utf-8]) -> Dumps the database to JSON file.
  # Return Type: None
  # Argument: location
    # Description: The location that will dump.
  # Argument: encoding
    # Description: Encoding while write.
  # Example(s):
    database.dump("./dump.json")

# <Datagoose>.load({location:str}, [encoding:str:utf-8], [overwrite:bool:True]) -> Load database from JSON file.
  # Return Type: Dict
  # Argument: location
    # Description: The location that will dump.
  # Argument: encoding
    # Description: Encoding while write.
  # Argument: overwrite
    # Description: If True, it will delete old data and paste loaded one. Else, it will extend data with loaded JSON.
  # Example(s):
    # 1.
      database.load("./old_database.json", overwrite=False)
    # 2.
      database.load("./old_database.json")

Note: for .load(), the JSON will loaded must have 'database' key and value must be a list. Also in list, values must be dict.

Note: "export" is an aliases for .dump().

# Example

database.export("./dump.json", indent=4)

# is same with:

database.dump("./dump.json", indent=4)

Copying Data

# <Datagoose>.copy({data:dict}, [repeat:int:1]) -> Copy data to database
  # Return Type: Bool
  # Argument: data
    # Description: The data will be copied to database.
  # Argument: repeat
    # Description: Repeat number for copy. Must be between 1 - 100000
  # Example(s):
    database.copy({
        "total_hours": 1
    }) # Now datagoose copied all data contains these informations.

# <Datagoose>.repeat_one({data:dict}) -> Copy one data to database
  # Return Type: Dict
  # Argument: data
    # Description: The data will be copied to database.
  # Example(s):
    database.copy_one({
        "user_id": 6811
    })

Collecting Garbage Data

# <Datagoose>.collect_garbage() -> Returns all garbage data in database
  # Return Type: Generator
  # Example(s):
    getgc = database.collect_garbage()
    for garbage in getgc:
      print("Found Garbage Data: ", garbage)

# <Datagoose>.clear_garbage() -> Clear all garbage data in database
  # Return Type: Bool
  # Example(s):
    database.clear_garbage()

Sorting Database

# <Datagoose>.sort_for_key({key:str}, [reverse:bool:False]) -> Sort database for key.
  # Return Type: List
  # Argument: key
    # Description: The key for sort.
  # Argument: reverse
    # Description: Reverse the result.
  # Example(s):
    point_list = database.sort_for_key("point", reverse=True)
    winner = point_list[0]

    print(f"Congrats, {winner['name']}. You won the game!")

# <Datagoose>.find_and_sort({data:dict}, {key:str}, [reverse:bool:False]) -> Find and sort database for key.
  # Return Type: List
  # Argument: data
    # Description: The data for find.
  # Argument: key
    # Description: The key for sort.
  # Argument: reverse
    # Description: Reverse the result.
  # Example(s):
    finished = database.find_and_sort({"finished": True}, "time", reverse=True)
    winner = finished[0]

    print(f"Congrats, {winner['name']}. You won the game!")

Note: .sort_for_key() not changes the database, just returns sorted version of database.

Auto-Backup

# <Datagoose>.start_backup([options:dict:{}]) -> Starts backup for every x second. Raises error when already started.
  # Return Type: None
  # Argument: options
    # Description: Options for auto-backup
  # Example(s):
    database.start_backup({
      "TIME": 60, # Second for repeat time. Minimum 30 second, Maximum 31,557,600 (1 year) second.
      "PATH": "database/backups", # Path for backup files.
      "ENCRYPTED": True # Encryption for database.
    })

# <Datagoose>.stop_backup() -> Stops backup loop. Will not effect if already stopped.
  # Return Type: Bool
  # Example(s):
    database.stop_backup()

# <Datagoose>.is_backup_open -> Returns backup status.
  # Return Type: Bool
  # Example(s):
    if not database.is_backup_open:
      print("Backup Disabled.")

Using Events

# You can use an event with .on(event_name, function).
# Example:

database.on("before_insert", lambda v: print("Starting to insert data: {0}".format(v)))
# Or:
def before_insert_function(value):
    # ...
    print("Starting to insert value: {0}".format(value))

database.on("before_insert", before_insert_function)

Event List

All events and quick descriptions;

{
  "before_insert": lambda value: None, # Runs before insert.
  "should_insert": lambda value: True, # Check method for should value insert.
  "after_insert": lambda value: None, # Runs after insert.

  "before_update": lambda now, changes: None, # Runs before update.
  "should_update": lambda now, changes: True, # Check method for should data update.
  "after_update": lambda now, old: None, # Runs after update.

  "before_replace": lambda now, new: None, # Runs before replace.
  "should_replace": lambda now, new: True, # Check method for should data replace.
  "after_replace": lambda now, old: None, # Runs after replace.

  "before_delete": lambda value: None, # Runs before delete.
  "should_delete": lambda value: True, # Check method for should data delete.
  "after_delete": lambda value: None, # Runs after delete.

  "before_copy": lambda value: None, # Runs before copy.
  "should_copy": lambda value: True, # Check method for should data copy.
  "after_copy": lambda value: None, # Runs after copy.

  "before_save": lambda: None, # Runs before save.
  "after_save": lambda: None, # Runs after save.

  "before_export": lambda: None, # Runs before using .dump() (export)
  "after_export": lambda: None, # Runs after using .dump() (export)

  "before_import": lambda: None, # Runs before using .load() (import)
  "after_import": lambda: None, # Runs after using .load() (import)

  "backup_start": lambda: None, # Runs when auto-backup starts.
  "backup": lambda: None, # Runs when backup data.
  "backup_finish": lambda: None, # Runs when auto-backup finish

  "before_garbage_clear": lambda: None, # Runs before garbage clear.
  "after_garbage_clear": lambda: None # Runs after garbage clear.
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datagoose-1.6.1.tar.gz (16.3 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page