Adaptive (Re)Utility Index — ARUI

ARUI is an experimental framework for estimating how suitable the learned

behaviour of an adaptive gating system may be for recreation using static gate

weights. ARUI can be used to measure how suitable an adaptive-model seed may be

for static recreation, while also providing the suggested fixed gate weights.

Status: Experimental / validation in progress.

ARUI v0.2.0 introduces a bounded, dimensionless index ranging from -1 to 1.

Its ability to predict successful static recreation is currently being

evaluated experimentally.

---

Motivation

Adaptive gating systems can dynamically change the contribution of different

information sources. This flexibility may improve predictive performance, but

it can also increase architectural and computational complexity.

In some cases, an adaptive gate may learn a useful allocation that changes only

slightly over time. If so, its behaviour may potentially be approximated using

a simpler static gate.

A seed with high adaptive gain and low gate displacement receives a higher

ARUI value.

ARUI is bounded between -1 and 1, allowing adaptive solutions evaluated on

different metric scales to be compared more meaningfully.

ARUI may also help avoid using large computational power whilst being uncertain

about the use of the adaptive weights for fixed models.

---

Methodology

Assume an adaptive gate contains $K$ data types and is evaluated across

$T$ observations.

At observation $t$, the gate produces:

$$ \mathbf{w}t = (w{t,1}, w_{t,2}, \ldots, w_{t,K}) $$

For a normalized gate:

$$ \sum_{j=1}^{K} w_{t,j} = 1 $$

---

1. Mean gate weight

For each data type $j$, ARUI calculates its mean gate weight:

$$ \bar{w}j = \frac{1}{T} \sum{t=1}^{T} w_{t,j} $$

The vector

$$ \bar{\mathbf{w}}

(\bar{w}_1,\bar{w}_2,\ldots,\bar{w}_K) $$

forms the proposed static recreation of the adaptive gate.

For normalized gate outputs, the mean weights also sum to 1, apart from

floating-point error.

---

2. Mean variation

For each data type, ARUI measures its mean absolute variation around its mean

gate weight:

$$ d_j = \frac{1}{T} \sum_{t=1}^{T} |w_{t,j}-\bar{w}_j| $$

This measures the typical distance between the dynamic gate allocation and the

constant weight that would be used during static recreation.

---

3. Seed displacement

The overall seed displacement is defined as the mean variation across all

$K$ data types:

$$ D_s = \frac{1}{K} \sum_{j=1}^{K} d_j $$

A lower $D_s$ indicates that the adaptive gate remains closer to its mean

configuration.

A perfectly static gate has:

$$ D_s = 0 $$

---

4. Bounded adaptive gain

ARUI defines a bounded signed adaptive gain so that a positive value always

indicates better adaptive performance.

For a metric where higher_is_better is True:

$$ G_s = \frac{A_s-F_s} {\max(A_s,F_s)} $$

For a metric where higher_is_better is False:

$$ G_s = \frac{F_s-A_s} {\max(A_s,F_s)} $$

where:

• $A_s$ is the adaptive performance for seed $s$

• $F_s$ is the corresponding fixed-model performance

For non-negative performance metrics:

$$ -1 \leq G_s \leq 1 $$

Through this calculation:

• $G_s > 0$: adaptive model performed better

• $G_s = 0$: equal performance

• $G_s < 0$: adaptive model underperformed

• $G_s = 1$: maximum possible adaptive advantage

• $G_s = -1$: maximum possible adaptive disadvantage

If both $A_s$ and $F_s$ are zero, ARUI defines:

$$ G_s = 0 $$

because neither model has a performance advantage over the other.

---

5. Adaptive (Re)Utility Index

ARUI combines bounded adaptive gain and gate displacement:

$$ \boxed{ ARUI_s = G_s - D_s(1+G_s) } $$

Equivalently:

$$ \boxed{ ARUI_s = (G_s+1)(1-D_s)-1 } $$

The displacement term moves the index away from the bounded adaptive gain as

the gate becomes less stable.

When:

$$ D_s=0 $$

the gate is perfectly static and:

$$ ARUI_s=G_s $$

For valid bounded gain and displacement values:

$$ \boxed{ -1 \leq ARUI_s \leq 1 } $$

A higher ARUI therefore favours solutions that combine:

• greater adaptive performance gain;

• lower average gate variation.

The theoretical maximum:

$$ ARUI_s=1 $$

requires:

$$ G_s=1 $$

and:

$$ D_s=0 $$

For a lower-is-better metric such as RMSE, this requires a perfect adaptive

performance value of zero, a non-zero fixed-model performance value, and no

variation in the learned gate weights.

An ARUI value of:

$$ ARUI_s=0 $$

with zero displacement corresponds to equal adaptive and fixed-model

performance.

Negative ARUI values indicate solutions whose combination of performance and

gate stability makes them unfavourable candidates for static recreation.