Import and export metadata about mongodb databases and collections
Project description
Import/Export Mongo Schema
===================
Import and export mongodb schemas **without** copying all the data. It will extract the following meta data about a mongo database:
* Collections
* Indexes
* Cap sizes
* Schema validators
The primary use case is when you have developed an application that uses mongodb, and want to setup a new instance with
the appopriate database layout. You can then provide a `config.json` file with your application and have this script
setup the database for you without the need for extra `ensureIndex` calls!
This data will be stored in a `json` file for a database that looks something like this:
```json
{
"databases": {
"test": {
"location": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"name": "pos_2dsphere",
"2dsphereIndexVersion": 3,
"keys": [
[
"pos",
"2dsphere"
]
]
},
{
"name": "device_1_timestamp_1",
"keys": [
[
"device",
1
],
[
"timestamp",
1
]
]
}
],
"options": {}
},
"users": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"unique": true,
"name": "username_idx",
"keys": [
[
"username",
1
]
]
}
],
"options": {
"validator": {
"$jsonSchema": {
"bsonType": "object",
"required": [
"username",
"password",
"level"
],
"properties": {
"username": {
"bsonType": "string",
"description": "must be a string and is required"
},
"level": {
"bsonType": "string",
"enum": [
"user",
"admin",
"moderator"
],
"description": "must be a string"
},
"password": {
"bsonType": "string",
"description": "must be a bcrypt password",
"pattern": "^\\$2b\\$\\d{1,2}\\$[A-Za-z0-9\\.\\/]{53}$"
}
}
}
},
"validationLevel": "strict",
"validationAction": "error"
}
},
"capped": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"unique": true,
"name": "key_1",
"keys": [
[
"key",
1
]
]
}
],
"options": {
"capped": true,
"size": 64000,
"max": 5000,
"validator": {
"$jsonSchema": {
"bsonType": "object",
"description": "Simple key value store",
"required": [
"key",
"value"
],
"properties": {
"key": {
"bsonType": "string",
"maxLength": 64.0,
"description": "the key value"
},
"value": {
"bsonType": "string",
"description": "the associated value"
}
}
}
},
"validationLevel": "strict",
"validationAction": "error"
}
}
}
},
"exported": "2018-07-18T17:12:43.460992"
}
```
Installation
------------
```bash
pip install MongoSchemaImportExport
```
Usage
-----
Make sure the user you are using to import/export has the appropriate privileges, they'll probably need to have the
`root` role or `dbOwner` on the source and destination.
To export your data run:
```bash
mongo-schema-export.py --uri mongodb://user:password@database.host1.com:27017/admin --databases test2,testIgnore
```
To import your schema run as bellow. Use `--delete-col` to delete collections before creating them (**WARNING:** this
will delete your data, you cannot change
an existing collection into a capped one, although, you can set a validator after creation):
```bash
mongo-schema-import.py --uri mongodb://user:password@database.host2.com:27017/admin --databases db_1,db_2 --verbose --delete-col
```
You will get an output like this:
```
Skipping: testIgnore
Creating database: test2
Dropping collection location
Creating collection: location
Options {}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'name': 'pos_2dsphere', '2dsphereIndexVersion': 3, 'keys': [['pos', '2dsphere']]}
Creating index: {'name': 'device_1_timestamp_1', 'keys': [['device', 1], ['timestamp', 1]]}
Dropping collection users
Creating collection: users
Options {'validator': {'$jsonSchema': {'bsonType': 'object', 'required': ['username', 'password', 'level'], 'properties': {'username': {'bsonType': 'string', 'description': 'must be a string and is required'}, 'level': {'bsonType': 'string', 'enum': ['user', 'admin', 'moderator'], 'description': 'must be a string'}, 'password': {'bsonType': 'string', 'description': 'must be a bcrypt password', 'pattern': '^\\$2b\\$\\d{1,2}\\$[A-Za-z0-9\\.\\/]{53}$'}}}}, 'validationLevel': 'strict', 'validationAction': 'error'}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'unique': True, 'name': 'username_idx', 'keys': [['username', 1]]}
Dropping collection capped
Creating collection: capped
Options {'capped': True, 'size': 64000, 'max': 5000, 'validator': {'$jsonSchema': {'bsonType': 'object', 'description': 'Simple key value store', 'required': ['key', 'value'], 'properties': {'key': {'bsonType': 'string', 'maxLength': 64.0, 'description': 'the key value'}, 'value': {'bsonType': 'string', 'description': 'the associated value'}}}}, 'validationLevel': 'strict', 'validationAction': 'error'}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'unique': True, 'name': 'key_1', 'keys': [['key', 1]]}
```
If you get permission errors, make sure your user has the right roles to read and write databases and collections.
===================
Import and export mongodb schemas **without** copying all the data. It will extract the following meta data about a mongo database:
* Collections
* Indexes
* Cap sizes
* Schema validators
The primary use case is when you have developed an application that uses mongodb, and want to setup a new instance with
the appopriate database layout. You can then provide a `config.json` file with your application and have this script
setup the database for you without the need for extra `ensureIndex` calls!
This data will be stored in a `json` file for a database that looks something like this:
```json
{
"databases": {
"test": {
"location": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"name": "pos_2dsphere",
"2dsphereIndexVersion": 3,
"keys": [
[
"pos",
"2dsphere"
]
]
},
{
"name": "device_1_timestamp_1",
"keys": [
[
"device",
1
],
[
"timestamp",
1
]
]
}
],
"options": {}
},
"users": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"unique": true,
"name": "username_idx",
"keys": [
[
"username",
1
]
]
}
],
"options": {
"validator": {
"$jsonSchema": {
"bsonType": "object",
"required": [
"username",
"password",
"level"
],
"properties": {
"username": {
"bsonType": "string",
"description": "must be a string and is required"
},
"level": {
"bsonType": "string",
"enum": [
"user",
"admin",
"moderator"
],
"description": "must be a string"
},
"password": {
"bsonType": "string",
"description": "must be a bcrypt password",
"pattern": "^\\$2b\\$\\d{1,2}\\$[A-Za-z0-9\\.\\/]{53}$"
}
}
}
},
"validationLevel": "strict",
"validationAction": "error"
}
},
"capped": {
"indexes": [
{
"name": "_id_",
"keys": [
[
"_id",
1
]
]
},
{
"unique": true,
"name": "key_1",
"keys": [
[
"key",
1
]
]
}
],
"options": {
"capped": true,
"size": 64000,
"max": 5000,
"validator": {
"$jsonSchema": {
"bsonType": "object",
"description": "Simple key value store",
"required": [
"key",
"value"
],
"properties": {
"key": {
"bsonType": "string",
"maxLength": 64.0,
"description": "the key value"
},
"value": {
"bsonType": "string",
"description": "the associated value"
}
}
}
},
"validationLevel": "strict",
"validationAction": "error"
}
}
}
},
"exported": "2018-07-18T17:12:43.460992"
}
```
Installation
------------
```bash
pip install MongoSchemaImportExport
```
Usage
-----
Make sure the user you are using to import/export has the appropriate privileges, they'll probably need to have the
`root` role or `dbOwner` on the source and destination.
To export your data run:
```bash
mongo-schema-export.py --uri mongodb://user:password@database.host1.com:27017/admin --databases test2,testIgnore
```
To import your schema run as bellow. Use `--delete-col` to delete collections before creating them (**WARNING:** this
will delete your data, you cannot change
an existing collection into a capped one, although, you can set a validator after creation):
```bash
mongo-schema-import.py --uri mongodb://user:password@database.host2.com:27017/admin --databases db_1,db_2 --verbose --delete-col
```
You will get an output like this:
```
Skipping: testIgnore
Creating database: test2
Dropping collection location
Creating collection: location
Options {}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'name': 'pos_2dsphere', '2dsphereIndexVersion': 3, 'keys': [['pos', '2dsphere']]}
Creating index: {'name': 'device_1_timestamp_1', 'keys': [['device', 1], ['timestamp', 1]]}
Dropping collection users
Creating collection: users
Options {'validator': {'$jsonSchema': {'bsonType': 'object', 'required': ['username', 'password', 'level'], 'properties': {'username': {'bsonType': 'string', 'description': 'must be a string and is required'}, 'level': {'bsonType': 'string', 'enum': ['user', 'admin', 'moderator'], 'description': 'must be a string'}, 'password': {'bsonType': 'string', 'description': 'must be a bcrypt password', 'pattern': '^\\$2b\\$\\d{1,2}\\$[A-Za-z0-9\\.\\/]{53}$'}}}}, 'validationLevel': 'strict', 'validationAction': 'error'}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'unique': True, 'name': 'username_idx', 'keys': [['username', 1]]}
Dropping collection capped
Creating collection: capped
Options {'capped': True, 'size': 64000, 'max': 5000, 'validator': {'$jsonSchema': {'bsonType': 'object', 'description': 'Simple key value store', 'required': ['key', 'value'], 'properties': {'key': {'bsonType': 'string', 'maxLength': 64.0, 'description': 'the key value'}, 'value': {'bsonType': 'string', 'description': 'the associated value'}}}}, 'validationLevel': 'strict', 'validationAction': 'error'}
Creating index: {'name': '_id_', 'keys': [['_id', 1]]}
Creating index: {'unique': True, 'name': 'key_1', 'keys': [['key', 1]]}
```
If you get permission errors, make sure your user has the right roles to read and write databases and collections.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file MongoSchemaImportExport-0.2.1.tar.gz
.
File metadata
- Download URL: MongoSchemaImportExport-0.2.1.tar.gz
- Upload date:
- Size: 5.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.18.4 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.23.0 CPython/3.6.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 01aaef2ea78a1f80e5005584e75e81300d54135f9571c75557bed631bd8f8e65 |
|
MD5 | ef1e4772ed9c86612a16d76ee02bdf77 |
|
BLAKE2b-256 | e6aa0ceb4e88b27cd8a7ebdb8acf0da99f1ff1bf218f8fd53ee74f523b3cf67b |
Provenance
File details
Details for the file MongoSchemaImportExport-0.2.1-py3-none-any.whl
.
File metadata
- Download URL: MongoSchemaImportExport-0.2.1-py3-none-any.whl
- Upload date:
- Size: 9.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.18.4 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.23.0 CPython/3.6.4
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0768db1950bc125f9c20cfe116342e2b72c845248bac8aa3de7d750ab767eb28 |
|
MD5 | df927290b78cbfd047e529f4b44562e5 |
|
BLAKE2b-256 | d6280ac4d1390039a2fb64c0deda2d851ffbcb2db8a5919a0800cfaa3853d4e1 |