minio-manager 0.1.0b1

Declare what MinIO buckets, IAM policies, ILM policies you want, and let MinIO Manager do the work.

minio-manager

Declare what MinIO buckets, IAM policies, ILM policies you want, and let MinIO Manager do the work.

• GitHub repository: https://github.com/alveel/minio-manager/
• Documentation https://alveel.github.io/minio-manager/

Description

The concept for management is to have so-called "bucket groups".

Each bucket group is managed by an account that only has access to buckets in that group.

It should be noted that this is explicitly intended for the creation and updating of resources in MinIO. It does not delete any resources anywhere.

Requirements

• Python (3.9 or newer)
• PDM

Getting started with your project

Install the environment and the pre-commit hooks with

make install

You are now ready to start development on your project! The CI/CD pipeline will be triggered when you open a pull request, merge to main, or when you create a new release.

To finalize the set-up for publishing to PyPi or Artifactory, see here. For activating the automatic documentation with MkDocs, see here. To enable the code coverage reports, see here.

Releasing a new version

Set up

An admin user should be used for these steps.

1. Create the bucket for the secret backend minio-manager-secrets

   mc mb $ALIAS/minio-manager-secrets

2. Create a user (either in MinIO or your identity provider)

   You can use mc admin user add $ALIAS minio-manager for a MinIO user

3. Create a policy that gives read/write access to the bucket for the secret backend

   You can use the example provided in the examples directory:

   mc admin policy create $ALIAS minio-manager examples/minio-manager-secrets-policy.json

4. Attach the policy to the user:

   • For MinIO: mc admin policy attach $ALIAS minio-manager --user=minio-manager
   • For LDAP: mc idp ldap policy attach $ALIAS minio-manager --user='uid=minio-manager,cn=users,dc=your,dc=domain'

5. Upload your secret backend (e.g. secrets.kdbx) to the bucket root

6. Create a MinIO service account/access key with either option:

   • mc admin user svcacct add $ALIAS minio-manager and note down the access and secret keys

7. Copy .env.example to .env and set the following variables to the obtained keys

   • MINIO_MANAGER_SECRET_BACKEND_S3_ACCESS_KEY
   • MINIO_MANAGER_SECRET_BACKEND_S3_SECRET_KEY

8. Configure the other variables in the .env file. Descriptions of each variable can be found in the Environment variables section

9. Each "bucket group" manager user must get its own policy.

   1. You can find an example in examples/bucket-group-user-policy.json
   2. mc admin policy create $ALIAS infra-test-manager examples/bucket-group-user-policy.json
   3. mc idp ldap policy attach $ALIAS infra-test-manager --user='uid=infra-test-manager,cn=users,dc=your,dc=domain'

10. You can then log in to the web console with this user to create an access key exactly like how we did it previously

MinIO

At least two users are required in MinIO. One with access to a single bucket containing the secret backend, all other users are to be used as "bucket group" managers. For each bucket created under this manager user a service account (or access key in S3/MinIO terms) will be created.

Secret backend

Keepass

The Keepass database's root group must be named "Passwords".

You must have a group called "s3" and subgroups with the name of each MinIO cluster.

Entry names must be unique.

Entries are found by way of the title of the entry, the username is not considered when searching.

Environment variables

Required

• MINIO_MANAGER_CLUSTER_NAME The name of the cluster, used for example in the secret backend
• MINIO_MANAGER_MINIO_ENDPOINT What host:port to use as MinIO/S3 endpoint
• MINIO_MANAGER_MINIO_CONTROLLER_USER The entry of the MinIO controller user in the secret backend that contains its access and secret keys
• MINIO_MANAGER_CLUSTER_RESOURCES_FILE The YAML file with the MinIO resource configuration (buckets, policies, etc.)
• MINIO_MANAGER_SECRET_BACKEND_TYPE What secret backend to use. Currently only keepass is supported
• MINIO_MANAGER_SECRET_BACKEND_S3_ACCESS_KEY The access key to the S3 bucket where the secret database is stored
• MINIO_MANAGER_SECRET_BACKEND_S3_SECRET_KEY The secret key to the S3 bucket where the secret database is stored
• MINIO_MANAGER_KEEPASS_PASSWORD Keepass database password

Optional

• MINIO_MANAGER_MINIO_ENDPOINT_SECURE Whether to use HTTPS for the endpoint. Defaults to True
• MINIO_MANAGER_SECRET_BACKEND_S3_BUCKET The name of the bucket where the secret backend is kept. Defaults to minio-manager-secrets
• MINIO_MANAGER_KEEPASS_FILE The name of the database file in the S3 bucket. Defaults to secrets.kdbx
• MINIO_MANAGER_LOG_LEVEL The log level of the application. Defaults to INFO, may also use DEBUG
• MINIO_MANAGER_DEFAULT_BUCKET_VERSIONING What bucket versioning level to use for all buckets by default if not specified on the bucket level. Defaults to "Disabled", can also configure "Enabled" or "Suspended"

To do features

• Check if policies are already attached to a user before running the "attach" command.
• Automatically generate keepass database when it is configured as the secret backend while not present in the bucket.
• Also sort policy Principals to prevent unnecessary policy updates.
• Re-use ServiceAccount object instead of MinioCredentials object which is effectively the same.
• Allow cleaning up of removed resources, e.g. service account that doesn't have a related bucket.
• Improve logging not to show stack trace when log level is not DEBUG.
• Add colours to different log levels.
• Create container image of project.
• Add lifecycle management to created buckets.
• Consider implementing object locking.

---

Repository initiated with fpgmaas/cookiecutter-pdm.