DevSecOps command line asset inventory
Project description
Clinv
clinv
is a command line inventory for DevSecOps resources in AWS.
Features
- Manage a dynamic inventory of risk management resources (Projects, Services, Information, People) and infrastructure resources (EC2, RDS, S3, Route53, IAM users, IAM groups...).
- Add risk management metadata to your AWS resources.
- Monitor if there are resources that are not inside your inventory.
- Perform regular expression searches on all your resources.
- Get all your resources information.
- Works from the command line.
Install
pip install clinv
clinv
will use your AWS cli credentials, therefore you must have them
configured.
Usage
clinv
will make all the operations on your local inventory. To sync what's
in your AWS account with your local inventory, execute clinv generate
.
Generate
clinv generate
uses boto3
to update what is in your AWS inventory, right now
it is able to import the following resources from all the regions:
- Autoscaling Groups.
- EC2.
- IAM users and groups.
- RDS.
- Route53.
- S3.
- Security Groups.
- VPCs.
As it's querying all regions, it can take long to fetch all the data. If you
only want to update a resource type, pass it as an argument to generate. For
example, clinv generate ec2
.
It will automatically mark the EC2 instances as monitored if they have the tag
monitor
with value "True"
.
It will automatically mark the EC2 instances as monitored if they have the tag
monitor
with value "True"
.
List
clinv list resource_type
will show a list of id and names of the selected
resource type.
In the case of services, it will only display the ones that doesn't have a state == terminated
.
Search
clinv search search_string
will case insensitively match:
- Project names, aliases and descriptions.
- Service names, aliases and descriptions.
- Information names, aliases and descriptions.
- EC2, RDS and Route53 properties (id, name, instance_type, private ips, public ips, descriptions, regions and security group ids).
- People names and emails.
- IAM users, groups and it's attached policies.
And will print the matching information.
The search_string can be a regular expression.
Unassigned
clinv unassigned resource_type
will show a list of id and names of elements
that are unassigned.
If resource_type
is ec2
, rds
, s3
, route53
, it will search for
instances that aren't assigned in a service
.
If resource_type
is people
, service
or information
, it will search for instances
that aren't assigned to a project
.
Monitor
clinv monitor monitor_status
will show a list of id and names of elements
that have the specified monitor_status
. The monitor_status
can be one of:
true
, false
and unknown
.
Export
clinv export
will create an ods file with the inventory in ~/.local/share/clinv/inventory.ods
Data Files
All data used by clinv
is saved into yaml files in the data path directory (by
default ~/.local/share/clinv
).
- source_data.yaml: Raw information of the AWS account, generally with stripped dictionaries from boto3 resources.
- user_data.yaml: Raw clinv additional data explained below.
The user_data.yaml file
The user_data file contains information of the following types of resources:
- Projects
- Informations
- Services
- AWS resources
clinv
still doesn't support the creation of resources, therefore I suggest
that you configure your editor to add templates (for example Ultisnip for
neovim).
Projects
Projects represent the reason for a group of service and information assets to exist.
An example of a project could be:
projects:
pro-01:
name: Clinv project
aliases: Other name for the project
description: Project to develope clinv
informations:
- inf-01
- inf-02
links:
homepage: https://git.digitales.cslabrecha.org/lyz/clinv
docs: https://git.digitales.cslabrecha.org/lyz/clinv
people:
- peo-01
services:
- ser-01
- ser-02
- ser-03
state: active
Informations
Represent an information asset.
An example of an information asset could be:
informations:
inf-01:
name: Clinv source code
description: Clinv source code
personal_data: false
responsible: peo-1
state: active
Services
Represent a service asset.
An example of a service asset could be:
services:
ser-01:
name: Clint homepage
description: Clint homepage
access: public
aws:
ec2:
- i-xxxxxxxxxxxx
route53:
- XXXXXXX-blabla.clinv.org-cname
rds:
s3:
iam_group:
endpoints:
- https://www.thispagedoesnot.exist
responsible: peo-01
informations:
- inf-01
links:
agile board: tbd
ci: tbd
docs:
deploy: tbd
internal: tbd
ops: tbd
public: tbd
issues: tbd
source code: https://git.cloud.icij.org/staff/cassandra-prophecies
authentication:
method: sso
2fa: true
users:
- admins
- partners
- internet
People
Represent a person.
An example of a service asset could be:
people:
peo-01:
name: Lyz
description: Administrator
state: active
email: lyz@clinv.org
iam_user: 'iamuser_lyz'
AWS Resources
Represents AWS resources, we'll show a typical resource template, though
a default one is filled up when clinv generate
is issued.
EC2
ec2:
i-xxxxxxxxxxxxxxxxx:
description: 'Clinv homepage'
to_destroy: false
environment: 'production'
RDS
rds:
db-xxxxxxxxxxxxxxxxxxxxxxxxxx:
description: Clinv backend database
environment: production
region: eu-east-1
to_destroy: false
Route53
route53:
zone_id-clinv.com-cname:
description: 'Public endpoint to the clinv frontened'
to_destroy: true
S3
s3:
clinv-bucket:
description: Webpage static storage bucket
desired_permissions:
read: public
write: private
environment: production
state: active
to_destroy: false
IAM Groups
iam_groups:
arn:aws:iam::xxxxxxxxxxxx:group/admin:
description: AWS administrators.
desired_users:
- arn:aws:iam::xxxxxxxxxxxx:user/lyz
- arn:aws:iam::xxxxxxxxxxxx:user/user_1
name: Administrator
state: active
to_destroy: false
IAM Users
iam_users:
arn:aws:iam::xxxxxxxxxxxx:user/lyz:
description: Lyz AWS IAM User
name: Lyz
state: active
to_destroy: false
Contributing
If you want to contribute with the project, first read the project guidelines and then contribute :).
You can also see the roadmap of the project in the todo.
If you want to create a new source or resource go to this file.
If you want to create a new report go to this file.
Test
To run the tests first install tox
pip3 install tox
And then run the tests
tox
Collaborators
This project is being developed with the help of ICIJ
Authors
lyz
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.