mycloudctl 1.0.0

Official CLI for myCloud — your personal cloud storage

myCloud

A full-stack personal cloud platform built with Flask + MySQL that supports file storage, folder management, sharing (direct invite + token links + batch sharing), previews, stash/recovery workflows, monitoring dashboards, and optional storage-node workflows.

Table of Contents

• Overview
• Core Features
• Project Structure
• Architecture
• Infrastructure and Runtime Topology
• Data and Schema Evolution
• Environment Configuration
• Local Development Setup
• Production Deployment (Linux VPS)
• Operations and Monitoring
• Security Notes
• Troubleshooting
• Release/Push Checklist

Overview

myCloud is a web application for managing personal cloud files with:

• User auth (username/password and social auth)
• Upload/download and folder operations
• Advanced sharing:
  • direct user invites
  • token-based share links
  • batch share links for mixed files/folders
• Shared library actions (preview, copy to MyCloud, leave share, favorites)
• Stash workflow for temporary removal/recovery
• Node-aware storage metadata and optional node-side GUI tooling
• Monitoring endpoints for system, services, app activity, and product analytics

Core Features

Account and Auth

• Registration, login, logout, password reset, email verification
• Two-factor code flow endpoints
• OAuth providers:
  • Google
  • GitHub

File and Folder Management

• Upload files, create folders
• Rename, move, delete, stash, unstash
• Bulk operations: move/delete/download/share/stash/unstash/remove-share/add-to-mycloud
• Preview support for images/video/audio/PDF/code/docx (web UI behavior)

Sharing

• Direct item sharing with permission updates
• Share links for file/folder
• Batch share link generation and acceptance
• Unified share removal APIs
• Invite response APIs (accept/decline + optional mute sender)
• Muted user management for invite control

UX and Settings

• Preferences APIs and sync endpoints
• Theme/view mode handling in frontend assets
• Profile picture upload/remove
• Notifications API + mark-read/hide

Monitoring and Analytics

• System metrics API (CPU/RAM/disk/network/load)
• Service status/log APIs (nginx/gunicorn/ssh where available)
• Application activity monitor (recent events/endpoints)
• Product analytics overview/trends

Project Structure

.
|- app.py                        # Main Flask + Socket.IO server
|- services.py                   # Linux service/log monitoring helpers
|- monitor.py                    # System monitor abstraction + local implementation
|- app_monitor.py                # App-level activity monitoring
|- data_monitor.py               # Data/storage monitoring
|- product_analytics.py          # Product usage analytics
|- db_update_nodes.py            # DB schema updates for storage node support
|- migrate_shares_table.py       # Migration to unified shares table
|- setup_batch_sharing_db.py     # Batch share schema bootstrap
|- mycloud_node_gui.py           # Optional node desktop GUI client
|- deploy_changed_files.ps1      # Incremental deployment helper (scp/ssh)
|- templates/                    # Flask templates
|- static/                       # Frontend JS/CSS/assets
|- uploads/                      # Runtime user uploads (ignored from git)
|- requirements.txt              # Python dependencies
|- .env.example                  # Environment variable template

Architecture

Application Layer

• Framework: Flask
• Real-time channel: Flask-SocketIO
• Reverse-proxy awareness: Werkzeug ProxyFix
• Session + auth logic + routing in a single main server file (app.py)

Real-time / Socket Layer

Socket handlers are registered in app.py for:

• node registration
• node disconnect
• chunked download transfer

Persistence Layer

• Primary DB: MySQL (mysql-connector-python)
• Schema support scripts:
  • migrate_shares_table.py creates/migrates a unified shares model
  • setup_batch_sharing_db.py provisions batch_share_links and batch_share_items
  • db_update_nodes.py provisions storage_nodes and extends files with node metadata

Frontend Layer

• Server-rendered Jinja templates in templates/
• Static JS/CSS in static/
• Rich shared-files interactions include:
  • list/grid sort/filter/view
  • direct invite modal and muted-users modal
  • multi-select action toolbar

Infrastructure and Runtime Topology

Typical production topology:

• Browser clients -> Nginx reverse proxy -> Gunicorn/Eventlet Flask-SocketIO app
• App server -> MySQL database
• Optional storage nodes -> connect/register via Socket.IO events

services.py is designed for Linux hosts with optional systemctl / journalctl access and can inspect:

• nginx
• gunicorn (unit configurable via MONITOR_GUNICORN_SERVICE)
• ssh/sshd

Data and Schema Evolution

Node/Storage schema updates

Run once when enabling node-aware storage metadata:

python db_update_nodes.py

What it does:

• Creates storage_nodes
• Adds files.storage_mode (default central) if missing
• Adds files.node_id FK to storage_nodes(id) if missing

Shares migration

Run once to unify old sharing models:

python migrate_shares_table.py

What it does:

• Creates unified shares table if missing
• Migrates data from older share tables/links when present

Batch share schema bootstrap

Run once to enable batch share links:

python setup_batch_sharing_db.py

What it does:

• Creates batch_share_links
• Creates batch_share_items with FK cascade

Environment Configuration

Copy the template and fill values:

cp .env.example .env

On Windows PowerShell:

Copy-Item .env.example .env

Key variables

• Flask/runtime:
  • FLASK_SECRET_KEY
  • FLASK_HOST
  • FLASK_PORT
  • FLASK_DEBUG
  • APP_BASE_URL
• Session hardening:
  • SESSION_COOKIE_SECURE
  • SESSION_COOKIE_SAMESITE
  • SESSION_LIFETIME_HOURS
• Database:
  • DB_HOST
  • DB_USER
  • DB_PASSWORD
  • DB_NAME
• OAuth/API:
  • GOOGLE_CLIENT_ID
  • GOOGLE_CLIENT_SECRET
  • GOOGLE_API_KEY
  • GITHUB_CLIENT_ID
  • GITHUB_CLIENT_SECRET
  • MONITOR_API_KEY
• SMTP:
  • SMTP_SENDER_EMAIL
  • SMTP_SENDER_PASSWORD
• Storage and sockets:
  • UPLOAD_FOLDER
  • ACCOUNT_STORAGE_LIMIT_GB
  • SOCKETIO_CORS_ORIGINS
• Monitoring/services:
  • MONITOR_GUNICORN_SERVICE

Local Development Setup

1) Create virtual environment

python -m venv .venv

Windows PowerShell:

.\.venv\Scripts\Activate.ps1

2) Install dependencies

pip install -r requirements.txt

3) Configure .env

Set a strong FLASK_SECRET_KEY and valid DB credentials.

4) Optional DB setup/migrations

python db_update_nodes.py
python migrate_shares_table.py
python setup_batch_sharing_db.py

5) Run app

python app.py

Server binds to:

• host: FLASK_HOST (default 0.0.0.0)
• port: FLASK_PORT (default 5000)

Production Deployment (Linux VPS)

Recommended stack

• Nginx (reverse proxy)
• Gunicorn + eventlet worker class for Socket.IO compatibility
• systemd service for app process
• MySQL reachable from app host

Example command:

gunicorn -k eventlet -w 1 -b 0.0.0.0:5000 app:app

Note: For Flask-SocketIO apps, eventlet/gevent-compatible deployment is recommended.

Incremental deploy helper

deploy_changed_files.ps1 can:

• detect changed deployable files from git status
• fallback to changed_files.txt when needed
• upload files via scp and pre-create remote directories via ssh
• optionally restart systemd service

Example:

.\deploy_changed_files.ps1 -RestartService -ServiceName mycloud

Before using in another environment, adjust script parameters:

• KeyPath
• User
• ServerHost
• RemoteRoot
• ServiceName

Operations and Monitoring

Monitoring routes are exposed in the app for:

• system metrics
• service status/logs
• app-level statistics and recent endpoint activity
• product analytics summaries/trends

For protected monitoring APIs, provide configured MONITOR_API_KEY where required by your runtime policy.

Security Notes

• Never commit .env, private keys, or runtime upload content.
• Rotate any credentials that were ever hardcoded/shared.
• Keep FLASK_SECRET_KEY long and random.
• Enforce HTTPS in production and keep SESSION_COOKIE_SECURE=true.
• Restrict CORS origins via SOCKETIO_CORS_ORIGINS.

Troubleshooting

App fails at startup with secret key error

• Ensure FLASK_SECRET_KEY is set in .env.

DB connection/migration errors

• Verify DB_HOST, DB_USER, DB_PASSWORD, DB_NAME
• Ensure MySQL user has schema alter/create permissions

Socket issues behind proxy

• Ensure proxy supports websocket upgrade
• Use an eventlet-compatible Gunicorn setup

Monitoring service status unavailable

• Confirm Linux host has systemctl/journalctl
• Verify service names (MONITOR_GUNICORN_SERVICE)

Release/Push Checklist

• .env excluded from git
• uploads/, build/, dist/, __pycache__/, .venv/ excluded from git
• requirements.txt up to date
• .env.example present and current
• Run schema scripts on target environment before first production run

---