apollog 0.1.1

A monitoring solution for AWS services with log aggregation and error event summarization

Apollog

What is Apollog?

Apollog is an open source solution for monitoring your existing services on AWS. There are 2 key offerings of this project:

• Log Aggregation over services: Collect and centralize logs from multiple AWS services
• Error Event summarization powered by LLMs: Automatically analyze and summarize error events

With this project, the aim is to allow users to simply deploy a side car stack on AWS that can keep track of your services and provide detailed analysis of any error events.

Installation

Option 1: Install from PyPI (Recommended)

pip install apollog

Option 2: Install from Source

git clone https://github.com/Bhavinrathava/Apollog.git
cd Apollog
pip install -e .

Quick Start

1. Initialize a New Project

# Create a new project in the current directory
apollog init

# Or specify a different directory
apollog init --destination my-apollog-project

2. Configure Your AWS Credentials

Ensure your AWS credentials are properly configured. You can do this by:

• Using the AWS CLI: aws configure
• Setting environment variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
• Using a credentials file in ~/.aws/credentials

3. Edit the Configuration File

Edit the examples/config.yaml file to specify the AWS services you want to monitor:

# Services to monitor
services:
  - serviceName: "MyAPIGateway"
    namespace: "/aws/APIGateway"
    region: "us-west-2"
  - serviceName: "MyLambda"
    namespace: "/aws/Lambda"
    region: "us-west-2"

# Architecture configuration
architecture: 
  logTableName: "Apollog-TimeBlock-Logs"
  summaryTableName: "Apollog-event-summary"
  tableRegion: "us-west-2"

# Stack name for CloudFormation
stack_name: "Apollog-Monitoring-Stack"

4. Deploy the Stack

apollog deploy --config examples/config.yaml

This will:

1. Create Lambda deployment packages
2. Upload them to an S3 bucket
3. Deploy a CloudFormation stack with all necessary resources
4. Set up the frontend for accessing the monitoring dashboard

Architecture

Apollog deploys the following AWS resources:

• API Gateway: Serves the frontend interface
• Lambda Functions:
  • Log Ingestion Lambda: Collects logs from specified services
  • Summary Lambda: Analyzes logs and generates summaries
  • Frontend Lambda: Serves the web interface
• DynamoDB Tables:
  • Logs Table: Stores collected logs
  • Summary Table: Stores error event summaries
• CloudWatch Events: Triggers the log ingestion process on a schedule

Advanced Usage

Custom Project Directory

If you've organized your project files in a different directory structure, you can specify the project directory when deploying:

apollog deploy --config examples/config.yaml --project-dir /path/to/project

Updating an Existing Stack

When you run the deploy command and a stack with the same name already exists, you'll be prompted to update it:

Stack Apollog-Monitoring-Stack already exists. Do you want to update it? (Y/N):

Enter Y to update the existing stack with your new configuration.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.