dcnr-discovery 2.1.0

Application based on self-discovery pattern.

DCNR Discovery

A DCNR Discovery is package supporting discovery architecture of the application. The principle is, that project folder contains app_discovery.yaml files on various places in the folder structure and this package will discover all those files, performs actions defined in them and thus creates whole application from its parts and connects them.

It enables to connect application infrastructure with its broader environment in which application operates. For this purposes are used for example environment variables containing reference to configuration data, or configuration data itself.

---

How It Works

At startup, the application factory walks the source tree starting from a given root directory, collecting every app_discovery.yaml file it encounters. Each file declares what its containing package contributes to the application: Flask blueprints, routes, health checks, context processors, and startup services. The factory processes all discovered files and assembles them into a single Flask application.

---

app_discovery.yaml Reference

An app_discovery.yaml file has two optional top-level sections: flask and app.

flask:
  # Flask-related declarations (blueprints, routes, health checks, context processors)

app:
  # Application lifecycle declarations (startup tasks, background threads)

Both sections are optional — a file may contain only flask, only app, or both.

---

The __self__ Placeholder

Any function value can use the special __self__ placeholder, which resolves at runtime to the dotted Python package path of the directory containing the app_discovery.yaml file.

Resolution uses sys.path to find the longest matching root, then computes the relative dotted path.

File location	__self__ resolves to
service/src/systems/adlib/app_discovery.yaml	systems.adlib
service/src/systems/alerts/app_discovery.yaml	systems.alerts
service/src/appdata/app_discovery.yaml	appdata

Examples:

function: "__self__.handlers.alert_conf_create"   # → systems.alerts.handlers.alert_conf_create

This keeps discovery files portable — if a package is moved, only its filesystem location changes; the __self__ references remain correct.

---

Environment Variable Substitution

All string values support ${ENV_VAR:default} syntax. The variable is resolved from the process environment at load time; if not set, the default (after the colon) is used. If no default is provided, the value resolves to an empty string.

host: "${PDB_HOST:localhost}"        # → value of PDB_HOST, or "localhost"
password: "${PDB_PWD:}"             # → value of PDB_PWD, or empty string

Resolved values are automatically coerced to int, float, or bool when the entire string is a substitution placeholder.

---

flask Section

flask.blueprints

Declares Flask blueprints with associated template and static folders. Templates in the declared folder are scanned for FLASK_PATH directives for automatic route registration.

flask:
  blueprints:
    - name: bp_main                          # Blueprint name (must be unique)
      folders:
        templates_folder: templates          # Relative to the YAML file's directory
        static_folder: static               # Relative to the YAML file's directory
        static_url_path: /monitor/static     # URL prefix for static files

Property	Type	Required	Description
name	string	No	Blueprint name. Auto-generated if omitted.
folders.templates_folder	string	No	Path to templates directory, relative to the YAML file. Default: templates
folders.static_folder	string	No	Path to static files directory. Default: static
folders.static_url_path	string	No	URL prefix for serving static files. Default: /monitor/static

Automatic Template Scanning (FLASK_PATH)

When a blueprint declares a templates_folder, all *.html files in that folder are scanned for a Jinja2 directive in the first 20 lines:

{% set FLASK_PATH = "/monitor/my_page" %}

If found, a GET route for that path is automatically registered, rendering that template. This allows templates to self-declare their route with zero YAML or Python configuration.

---

flask.routes

Declares HTTP route endpoints. Each route is registered on an auto-created blueprint scoped to the discovery file.

flask:
  routes:
    - path: "/monitor/some_endpoint"
      methods: [GET]
      type: json_proxy
      function: "__self__.api.get_data"

Property	Type	Required	Description
path	string	Yes	URL rule (Flask syntax, supports <variable> and <int:variable>)
methods	list	No	HTTP methods. Default: [GET]
type	string	Yes	One of: page, json_proxy, handler
function	string	Depends	Dotted path to a Python callable. Required for json_proxy and handler. Supports __self__.
template	string	Depends	Template filename. Required for page.
template_args	dict	No	Key-value arguments passed to render_template. Supports lpc: and url: prefixes (see below).

Route Types

page — Renders a Jinja2 template. No Python handler needed.

- path: "/monitor/dashboard"
  type: page
  template: "dashboard.html"
  template_args:
    report: "lpc:ui_get_report"     # call LPC command at render time
    job_id: "url:job_id"            # pull from URL parameter <job_id>

json_proxy — Calls a function and wraps the result in jsonify(). Ideal for data endpoints.

- path: "/monitor/api/data"
  methods: [GET]
  type: json_proxy
  function: "__self__.api.get_data"

handler — Delegates to a Python function that receives URL parameters as keyword arguments and must return a Flask response (or tuple).

- path: "/monitor/api/items/<int:item_id>"
  methods: [PUT, DELETE]
  type: handler
  function: "__self__.handlers.update_item"

Template Argument Prefixes

Prefix	Meaning	Example
lpc:	Execute an LPC command at render time	"lpc:ui_get_report"
url:	Extract from URL path parameters	"url:job_id"
(none)	Literal string value	"some text"

---

flask.health_checks

Declares HTTP health-check endpoints that return a static text response.

flask:
  health_checks:
    ready:
      path: "/ready"
      response: "I am ready!!!"
    live:
      path: "/live"
      response: "I am alive!!!"

Property	Type	Required	Description
(key)	string	Yes	Endpoint name (used as Flask endpoint identifier)
path	string	Yes	URL path
response	string	No	Response body text. Default: "OK"

---

flask.context_processors

Declares variables injected into every Jinja2 template rendered by the application.

flask:
  context_processors:
    - name: "adlib"
      type: module
      module: "systems.adlib"

    - name: "env_name"
      type: env
      value: "${APPLICATION_ENV:local}"

Property	Type	Required	Description
name	string	Yes	Variable name available in templates (e.g. {{ adlib }})
type	string	Yes	One of: module, env
module	string	Depends	Dotted import path. Required when type: module.
value	string	Depends	Literal or ${ENV_VAR:default} value. Required when type: env.

---

app Section

app.startup

Declares startup tasks and background services. Steps are executed asynchronously with dependency resolution — a step will wait for its depends_on target to complete before running.

app:
  startup:
    - step: "set_postgresql_conf"
      type: "startup"
      description: "Configure PostgreSQL connection"
      function: "dcnr.postgres.set_configuration"
      kwargs:
        host: "${PDB_HOST:localhost}"
        port: "${PDB_PORT:5432}"

    - step: "schema_deploy"
      type: "startup"
      function: "__self__.deploy_schema"
      depends_on: "set_postgresql_conf"

    - step: "alert_daemon"
      type: "thread"
      function: "__self__.check_alerts_daemon"
      daemon: true
      condition: "${APPLICATION_RUN_SERVICES:False} == True"
      depends_on: "set_postgresql_conf"

Property	Type	Required	Description
step	string	Yes	Unique identifier for this step (used in depends_on references)
type	string	Yes	startup (one-shot) or thread (long-running background thread)
function	string	Yes	Dotted module/function path. Supports __self__. Function name within the module to call
description	string	No	Human-readable description (logged at startup)
args	list	No	Positional arguments passed to the function
kwargs	dict	No	Keyword arguments passed to the function. Values support ${ENV_VAR} substitution.
depends_on	string	No	step name that must complete before this step runs
condition	string	No	Expression that must evaluate to true for the step to execute (see below)
daemon	bool	No	For type: thread only. Whether the thread is a daemon. Default: true

Conditions

Conditions are strings evaluated at runtime after environment variable substitution. They support simple equality expressions:

condition: "${APPLICATION_RUN_SERVICES:false} == true"

A condition prefixed with ! is inverted:

condition: "!${SKIP_SETUP:false}"

If no condition is specified, the step always executes.

Dependency Resolution

Steps with depends_on wait for the named step to complete. The runtime retries up to 12 times with 10-second intervals (total: 2 minutes). Steps whose dependencies are never satisfied are logged as unresolved.

- step: "schema_deploy"
  depends_on: "set_postgresql_conf"    # waits for this step to finish first

Dependencies work across discovery files — a step in systems/appdb/app_discovery.yaml can depend on a step defined in systems/emails/app_discovery.yaml, as long as the step name matches.

---

Complete Example

A minimal app_discovery.yaml for a subsystem package:

flask:
  routes:
    - path: "/monitor/api/widgets"
      methods: [GET]
      type: json_proxy
      function: "__self__.api.get_widgets"

    - path: "/monitor/api/widgets"
      methods: [POST]
      type: handler
      function: "__self__.handlers.create_widget"

app:
  startup:
    - step: "widget_db_config"
      type: "startup"
      function: "__self__.config.set_connection"
      kwargs:
        host: "${WIDGET_DB_HOST:localhost}"
        port: "${WIDGET_DB_PORT:5432}"

    - step: "widget_cache_warmup"
      type: "startup"
      function: "__self__.warm_cache"
      depends_on: "widget_db_config"

This file declares:

• Two REST endpoints (auto-registered on a blueprint)
• A database configuration step at startup
• A cache warmup step that runs after the database is configured

No changes to main.py, no central route table, no central startup sequence.

DCNR Introspection

In-memory message logging module for debugging purposes. Messages are stored per topic with a configurable size limit, keeping only the most recent entries.

Usage

Adding messages

from dcnr.introspection import add_message

add_message("my-topic", "Something happened")
add_message("my-topic", "An error occurred", level="error", details="Stack trace...")

Parameters:

• key – Topic/category name for the message.
• message – The log message text.
• level – Severity level (default: "info").
• details – Optional additional details.

Configuring topic limits

Each topic retains up to 50 messages by default. When the limit is reached, the oldest message is removed.

from dcnr.introspection.uimessages import set_topic_limit

set_topic_limit("my-topic", 100)

Retrieving data

from dcnr.introspection.uimessages import get_topics, get_messages_for_topic

topics = get_topics()          # List of all topics with message counts
msgs = get_messages_for_topic("my-topic")  # Messages for a specific topic

Integration with other modules

The module exposes two Flask handler functions that can be registered as routes to provide message data to a UI or external consumer:

• get_uimessages_topics() – Returns all topics as JSON.
• get_uimessages_messages() – Returns messages for a given ?topic= query parameter as JSON.

from dcnr.introspection import get_uimessages_topics, get_uimessages_messages

Thread Safety

All operations are protected by a threading lock, making it safe to call add_message from multiple threads.