robotmonitor 0.1.0

A tool to monitor and notify robot log errors.

robotmonitor

Maintain datalake to monitor robot real time situation

Project Setup Instructions

Project Overview

This project aims to connect robot logs to the internet, ensuring comprehensive log management and instant error notification. The objectives include:

• Saving all logs in a NoSQL data store.
• Generating instant messages upon detecting errors.

We achieve this through four main steps:

1. Configure Slack for notifications.
2. Configure folder sharing from the robot to a Linux system.
3. Install and configure the Elastic Stack (ELK).
4. Run the Python service to handle logs and notifications.

Step-by-Step Instructions

##########################

1. Configure Slack

##########################

Slack Configuration for Notifications

This section guides you through setting up Slack to receive instant messages when errors are detected in robot logs. Follow the steps below to create and configure a Slack app, which will allow you to send notifications directly to your chosen Slack channel.

Step 1.1: Create and Configure a Slack App

1. Create a Slack App:

   • Navigate to Slack API and click on "Create New App." (you need to sign in first).
   • Select "From scratch," name your app, and choose the Slack workspace where notifications will be sent.
   • Click "Create App."

2. Set Permissions:

   • Go to "OAuth & Permissions" (left side, under Features)
   • Scroll down to "Scopes" and click "Add an OAuth Scope."
   • Add the following scopes to enable message sending and channel reading:
     • chat:write
     • channels:read
     • groups:read
     • im:read
     • mpim:read
     • channels:history
     • conversations.connect:read
     • incoming-webhook
   • Scroll back up to "Install to Workspace" (green) to generate and obtain the OAuth Access Token, which will be used to authenticate API requests.
   • Create a configuration file named .env with the following line: SECONDS_TO_STAY_SILENCE=3 SLACK_TOKEN=xoxb-******

3. Incoming Webhook Setup (Optional) You can skip this step. It allows another method to post to slack, called webhook. Webhook is simpler, but with a limited functionality.

   • In the app settings, select "Incoming Webhooks" from the sidebar.
   • Toggle the activation switch to "On."
   • Click "Add New Webhook to Workspace" and select the channel where you want to receive notifications.
   • Copy the generated Webhook URL, which will be used in your Python script to send messages.

Step 1.2: Find Your Slack Channel ID

1. Locate Channel ID:

• Open your Slack workspace in a browser.
• Navigate to the channel where notifications should appear.
• The URL will include the Channel ID at the end, e.g., https://app.slack.com/client/T00000000/C00000000 — C00000000 is your Channel ID.
• Add to .env the following line: SLACK_CHANNEL_ID=C***

2. Invite the bot to the channel. Go to your Slack Channel and type the following as a message. /invite @bot-name

3. Test pip install -r requirements.txt python tests/test_slack_bot.py

##########################

2. Folder Sharing

##########################

On the Computer connected to the robot:

1. Navigate to the Logs Folder:

   • Go to c:\Program Files (x86)\Hamilton\

2. Configure Sharing:

   • Right-click the Logs folder and select Properties.
   • Switch to the Sharing tab and click on Advanced Sharing.
   • Check Share this folder. Set the share name, for instance, RobotLogs.
   • Click on Permissions and ensure that you grant 'Read' or 'Full Control' permissions, depending on the required level of access.
   • Confirm all settings by clicking OK.

3. Note the Network Path:

   • The shared folder's network path will typically look like \\YOUR-PC-NAME\RobotLogs. Ensure to note this path as it will be used for mounting the folder on Ubuntu.

On Ubuntu:

1. install cifs sudo apt-get install cifs-utils

2. Create the folder for sharing sudo mkdir /var/log/shared_logs

3. Mount temporarily: Use the following command to mount the Windows shared folder to the mount point you created:

   sudo mount -t cifs -o username=YOUR_WINDOWS_USERNAME,password=YOUR_WINDOWS_PASSWORD,iocharset=utf8,noperm //YOUR-PC-NAME/RobotLogs /var/log/shared_logs

4. Configure Persistent Mounting Add the following line to /etc/fstab (you can use emacs/nano/vi) //YOUR-PC-NAME/RobotLogs /var/log/shared_logs cifs username=YOUR_WINDOWS_USERNAME,password=YOUR_WINDOWS_PASSWORD,iocharset=utf8,noperm 0 0

5. remount: sudo mount -a

#######################

3. Install ELK

####################### Installtion of the ElasticSearch,Kibana,Logstash is out of the scope of this repo. You can use paid services such as https://logz.io/, https://www.elastic.co/, https://aws.amazon.com/opensearch-service/ or dare to install it yourself ! See e.g. here for the last option: https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-22-04

I have provided in this repo a logstash configution file scripts/logstash.conf

To load all log files into Elasticsearch. Make sure to update the user and password in the relevant section.

##############################

4. run the notifier script

############################## nohup python3 scripts/elasticsearch_error_notifier.py >> elasticsearch_error_notifier.log 2>&1 &

References

• For a comprehensive guide on Slack APIs and further configuration details, visit the Slack API Documentation.
• For specifics on message sending methods, see Sending Messages on Slack.

Note

With the above setup, your project can send real-time notifications to a Slack channel, enhancing the monitoring system's responsiveness to issues. Depending on your preference and requirement, choose between using the API for complex interactions or Webhooks for straightforward message sending.