An OPAL fetch provider to bring authorization state from MongoDB.
Project description
OPAL Fetcher for MongoDB
Made with ❤️ at
Check out OPAL main repo here.
What's in this repo?
An OPAL custom fetch provider to bring authorization state from MongoDB.
How to try this custom fetcher in one command? (Example docker-compose configuration)
You can test this fetcher with the example docker compose file in this repository root. Clone this repo, cd
into the cloned repo, and then run:
docker compose up
this docker compose configuration already correctly configures OPAL to load the MongoDB Fetch Provider, and correctly configures OPAL_DATA_CONFIG_SOURCES
to include an entry that uses this fetcher.
✏️ How to use this fetcher in your OPAL Setup
1) Build a custom opal-client Dockerfile
The official docker image only contains the built-in fetch providers. You need to create your own Dockerfile
(that is based on the official docker image), that includes this fetcher's pip package.
Your Dockerfile
should look like this:
FROM permitio/opal-client:latest
RUN pip install --no-cache-dir --user opal-fetcher-mongodb-treedom
2) Build your custom opal-client container
Say your special Dockerfile from step one is called custom_client.Dockerfile
.
You must build a customized OPAL container from this Dockerfile, like so:
docker build -t yourcompany/opal-client -f custom_client.Dockerfile .
3) When running OPAL, set OPAL_FETCH_PROVIDER_MODULES
Pass the following environment variable to the OPAL client docker container (comma-separated provider modules):
OPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_mongodb.provider
Notice that OPAL receives a list from where to search for fetch providers.
The list in our case includes the built-in providers (opal_common.fetcher.providers
) and our custom MongoDB provider.
4) Using the custom provider in your DataSourceEntry objects
Your DataSourceEntry objects (either in OPAL_DATA_CONFIG_SOURCES
or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.
Example value of OPAL_DATA_CONFIG_SOURCES
(formatted nicely, but in env var you should pack this to one-line and no-spaces):
{
"config": {
"entries": [
{
"url": "mongodb://user:password@mongodb/test_database?authSource=admin",
"config": {
"fetcher": "MongoDBFetchProvider",
"database": "opal_fetcher_mongodb",
"collection": "cities_collection",
"find": { "query": {} }
},
"topics": ["policy_data"],
"dst_path": "cities"
}
]
}
}
Notice how config
is an instance of MongoDBFetchProvider
(code is in opal_fetcher_mongodb/provider.py
).
Values for this fetcher config:
- The
url
is actually a MongoDB uri. - Your
config
must include thefetcher
key to indicate to OPAL that you use a custom fetcher. - Your
config
must include thecollection
key to indicate what collection to query in MongoDB. - Your
config
may include thedatabase
key to indicate what database to query in MongoDB. If not specified, the default database will be used. - Your
config
must include one offindOne
,find
oraggregate
keys to indicate what query to run against MongoDB. - Your
config
may include thetransform
key to transform the results from thefind
oraggregate
queries.
Query methods
All the three available query methods accept the same input parameters as defined in the MongoDB documentation.
findOne
findOne
- MongoDB docs
Example configuration
{
"config": {
"entries": [
{
...
"config": {
...
"findOne": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
}
}
}
]
}
}
find
find
- MongoDB docs
Example configuration
{
"config": {
"entries": [
{
...
"config": {
...
"find": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": "",
"merge": true
}
}
}
]
}
}
aggregate
aggregate
- MongoDB docs
Example configuration
{
"config": {
"entries": [
{
...
"config": {
...
"aggregate": {
"pipeline": [
...
],
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": ""
}
}
}
]
}
}
Query transform
transform
allows you to transform the results from the find
or aggregate
queries.
first
transform.first
allows you to return only the first result from the query.
Equivalent to the following Python code:
result = query_result[0]
mapKey
transform.mapKey
allows you to map the original list-like result to a dictionary-like result using the property specified in the mapKey
as the key for the dictionary.
Equivalent to the following Python code:
result = {}
for item in query_result:
result[item['key']] = item
Only properties in the root of the document can be used as the key for the dictionary.
merge
transform.merge
allows you to merge the results from the query into a single document. Duplicate keys will be overwritten by the last document in the list.
Equivalent to the following Python code:
result = {}
for item in query_result:
for key, value in item.items():
result[key] = value
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for opal_fetcher_mongodb_treedom-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a4456f0e3f1a84f0d7739450295a0ae29a5875beaab6c1fed55d2790321a9468 |
|
MD5 | 5d087e2eee1df787e7798319b8255c96 |
|
BLAKE2b-256 | cb2c69a6760297d2ce596f6a205472ff78da380b60a142784deea3fc428a2ddf |
Hashes for opal_fetcher_mongodb_treedom-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 17159a3cadaea7db8f8245f84da6e67118ffdb21854c289f21c7f3e9fa9ddb78 |
|
MD5 | ec0681cbf06427a08f24101027eb9c46 |
|
BLAKE2b-256 | 914b4a22105a74fa5b7bc92f6f793562faa5aa1689307466b487d862f733c588 |