multicloud-diagrams 0.3.11

Library to generate DRAW.IO compatible diagrams to represent Cloud infrastructure. AWS Cloud supported.

MultiCloud Diagrams

Generate and render Diagrams-As-a-Code (DAG) in popular drawio vector format for multiple Cloud providers. This is graph not static, so you can move, edit, position nodes like you want.

• Source code: https://github.com/tsypuk/multicloud-diagrams
• PyPI multicloud-diagrams: https://pypi.org/project/multicloud-diagrams/
• Sponsor project: https://patreon.com/tsypuk
• CHANGELOG: https://github.com/tsypuk/multicloud-diagrams/blob/main/CHANGELOG.MD
• Bug reports: https://github.com/tsypuk/multicloud-diagrams/issues
• Samples and examples: https://github.com/tsypuk/multicloud-diagrams/tree/main/samples/samples

Supported Nodes Landscape:

Cloud Provides:

Supported:

Planned to be added:

Features:

• allows to generate drawio diagram with predefined styles for popular aws services, support graph-based connection with named edges
• supports single and batch elements append to diagram
• duplicates detection to prevent ball of mud in file format and diagram
• verification that both vertices present on diagram when adding edge connection between them
• if the node is not present, fallback to default icon when rendering
• read previous version of drawio file and reuses existing vertices coordinates when generating a new version
• generate diagram from YAML definition
• mix and augment Diagram-as-code, real infra crawler, static yaml-based content to diagram

Library usage samples:

1. Generating AWS DynamoDB Details drawio diagram based on library, http, awscli, etc. Example based on boto3 DynamoDB response

{
        'AttributeDefinitions': [
            {
                'AttributeName': 'first',
                'AttributeType': 'S'
            },
            {
                'AttributeName': 'second',
                'AttributeType': 'N'
            },
            {
                'AttributeName': 'third',
                'AttributeType': 'B'
            },
        ],
        'TableName': 'prod-dynamo-table',
        'KeySchema': [
            {
                'AttributeName': 'string',
                'KeyType': 'HASH'
            },
        ],
        'TableSizeBytes': 123,
        'ItemCount': 123,
        'TableArn': 'arn:aws:dynamodb:eu-west-1:123456789:table/prod-dynamo-table',
        'LocalSecondaryIndexes': [{
...       
...
...
          
          'SSEDescription': {
            'Status': 'ENABLED',
            'SSEType': 'KMS',
            'KMSMasterKeyArn': 'string',
          },
          'DeletionProtectionEnabled': True
        }

• Source aws_dynamo_boto3_response.py
• Output compiled output.prod.dynamo.drawio:

2. Diagrams as a Code. DAG. Generating AWS IAM Graph from the code based on DAG declaration:

    producer_func_arn = 'arn:aws:lambda:eu-west-1:123456789:function:producer-lambda'
metadata = {
    "CodeSize": 1234,
    "Handler": "main",
    "Layers": 0,
    "Memory": 128,
    "PackageType": "Zip",
    "Runtime": "go1.x",
    "Timeout": 30,
    "TracingConfig": "{'Mode': 'Active'}",
    "Version": "$LATEST"
}
mcd.add_vertex(node_id=producer_func_arn, node_name='producer-lambda', arn=producer_func_arn, node_type='lambda_function', metadata=metadata)

consumer_func_arn = 'arn:aws:lambda:eu-west-1:123456789:function:consumer-lambda'
mcd.add_vertex(node_id=consumer_func_arn, node_name='consumer-lambda', arn=consumer_func_arn, node_type='lambda_function')

dynamo_arn = 'arn:aws:dynamodb:eu-west-1:123456789:table/table1'
mcd.add_vertex(node_id=dynamo_arn, node_name='table1', arn=dynamo_arn, node_type='dynamo')

sns_arn = 'arn:aws:sns:eu-west-1:123456789:internal.fifo'
metadata = {
    "Owner": 123456789,
    "SubscriptionsConfirmed": 3,
    "SubscriptionsPending": 0
}
mcd.add_vertex(node_id=sns_arn, node_name='internal', arn=sns_arn, node_type='sns', metadata=metadata)

sqs_arn = 'arn:aws:sqs:eu-west-1:123456789:int-eu-live-events.fifo'
metadata = {
    "DelaySeconds": 0,
    "FifoQueue": "TRUE",
    "ReceiveMessageWaitTimeSeconds": 0,
    "SqsManagedSseEnabled": "false",
    "VisibilityTimeout": 30
}
mcd.add_vertex(node_id=sqs_arn, node_name='events', arn=sqs_arn, node_type='sqs', metadata=metadata)

sqs_arn2 = 'arn:aws:sqs:eu-west-1:123456789:int-eu-live-events2.fifo'
mcd.add_vertex(node_id=sqs_arn2, node_name='events2', arn=sqs_arn2, node_type='sqs')

s3_arn = 'arn:aws:s3:::bucket_name/key_name'
mcd.add_vertex(node_id=s3_arn, node_name='key_name', arn=s3_arn, node_type='s3')

mcd.add_link(src_node_id=f'lambda_function:{producer_func_arn}', dst_node_id=f'sns:{sns_arn}')
mcd.add_link(src_node_id=f'lambda_function:{producer_func_arn}', dst_node_id=f's3:{s3_arn}')
mcd.add_link(src_node_id=f'sns:{sns_arn}', dst_node_id=f'sqs:{sqs_arn}')
mcd.add_link(src_node_id=f'sns:{sns_arn}', dst_node_id=f'sqs:{sqs_arn2}')
mcd.add_link(src_node_id=f'sqs:{sqs_arn}', dst_node_id=f'lambda_function:{consumer_func_arn}')
mcd.add_link(src_node_id=f'lambda_function:{consumer_func_arn}', dst_node_id=f'dynamo:{dynamo_arn}')

• Source aws_service_end_2_end
• Output compiled output.prod.end2end.drawio

3. Generating Diagram from YAML source

vertices:
  - name: prod-lambda-name
    type: lambda_function
    arn: arn:aws:lambda:eu-west-1:123456789:function:prod-lambda-name
  - name: role-lambda-name
    type: iam_role
    arn: arn:aws:iam::123456789:role/prod-lambda-name
  - name: prod-cloudwatch-policy
    icon: broker
    type: iam_policy
    arn: arn:aws:iam::123456789:policy/prod-cloudwatch-policy
  - name: prod-s3-policy
    type: iam_policy
    arn: arn:aws:iam::123456789:policy/prod-s3-policy
  - name: prod-dynamodb-policy
    type: iam_policy
    arn: arn:aws:iam::123456789:policy/prod-dynamo-policy
edges:
#  1st approach src/dst are linked to yaml vertices by name
  - { src: prod-lambda-name, dst: role-lambda-name, label: HasRole, link_type: none }
  - { src: role-lambda-name, dst: prod-cloudwatch-policy, label: Allow CloudWatch logs, link_type: none }
#  2nd approach src/dst are linked by ARN (ARN can be present in same yaml, or loaded programmatically)
  - {
    src_arn: arn:aws:iam::123456789:role/prod-lambda-name,
    src_type: iam_role,
    dst_arn: arn:aws:iam::123456789:policy/prod-s3-policy,
    dst_type: iam_policy,
    label: Allow S3 access,
    link_type: none }
# 3rd approach is mixed of 1st and 2nd
  - {
    src_arn: arn:aws:iam::123456789:role/prod-lambda-name,
    src_type: iam_role,
    dst: prod-dynamodb-policy,
    label: Allow DynamoDB read access,
    link_type: none }

• Source aws_iam_from_yaml.py

• Output compiled output.prod.iam-roles-from-yaml.drawio

• 

Publishing

poetry build
export PYPI_USERNAME=
export PYPI_PASSWORD=
poetry publish --build --username $PYPI_USERNAME --password $PYPI_PASSWORD

cd samples/samples
poetry install
poetry run python aws_iam_roles_from_code_with_enum.py

poetry run flakehell plugins
NAME        | VERSION  | CODES            | RULES
mccabe      | 0.6.1    | C90              | +*
pycodestyle | 2.7.0    | E, W             | +*, -E501
pyflakes    | 2.3.1    | F                | +*
pylint      | 0.0.0    | C, E, F, I, R, W | +*

poetry run flakehell lint

poetry run git-changelog -c angular -s docs,feat,test --output CHANGELOG.MD
git push --tags

Contribution Guide:

OpenSource Guide, How to contribute to opensource