KPI

Overview

Key Performance Indicator (KPI) framework for scenario evaluation.

This module defines the foundational classes and enums used to define, compute, and present KPIs within the Algomancy framework. It integrates with the algomancy_utils.unit system to provide smart, unit-aware formatting for performance metrics.

Core concepts:

• BaseKPI: An abstract base class for all KPIs. It handles the metadata (name, improvement direction), unit scaling, and threshold comparisons.

• ImprovementDirection: An enum defining what “better” means for a given metric (e.g., HIGHER is better for throughput, LOWER is better for latency).

• Binary vs. Continuous KPIs: KPIs can be simple numeric trackers (continuous) or they can have a threshold (binary), where they are either “successful” or “failed” based on whether they met the threshold.

Why this exists:

Scenario results often contain raw data that needs to be distilled into meaningful metrics. This module provides a consistent way to define these metrics, automatically handle their units, and determine if they meet predefined success criteria, making it easy to generate summary reports.

Quick start:

1. Define a concrete KPI class by inheriting from BaseKPI and implementing the compute method.

2. Instantiate the KPI with a name, improvement direction, and base unit.

3. Call compute_and_check(result) to populate the KPI value from scenario results.

4. Use pretty() to get a human-readable string of the result.

Example

TODO Explain example

A basic KPI implementation

from algomancy_scenario import BaseKPI, ImprovementDirection
from algomancy_utils import QUANTITIES, BaseMeasurement

class DurationKPI(BaseKPI):
    def compute(self, result):
        # Logic to extract duration from result
        return 1250.5

time = QUANTITIES["time"]
# Expect duraction of at least 1000 seconds
kpi = DurationKPI(
    "System Duration",
    ImprovementDirection.AT_LEAST,
    BaseMeasurement(time["s"]),
    threshold=1000.0
)

kpi.compute_and_check(some_result_object)
print(kpi.pretty())
print(kpi.details())

✓
1.25 ks

Note that the KPI class is normally passed to the app configuration and a separate instance of the KPI is constructed every Scenario. Typical use of the DurationKPI class would therefore be:

Using KPIs in AppConfig

from algomancy_gui.configuration.appconfig import AppConfig
from algomancy_scenario.core_configuration import CoreConfig

config = AppConfig(
    core_config=CoreConfig(
        ...
        kpis={"duration": DurationKPI, ...},
        ...
    ),
    ...
)

Reference

class algomancy_scenario.keyperformanceindicator.ImprovementDirection(*values)

Bases: StrEnum

Defines the direction of improvement or success criteria for a KPI.

Members:

• HIGHER: The KPI is better when its value is higher (e.g., Accuracy).

• LOWER: The KPI is better when its value is lower (e.g., Latency).

• AT_LEAST: A binary success criterion; successful if value >= threshold.

• AT_MOST: A binary success criterion; successful if value <= threshold.

HIGHER = 'higher'

LOWER = 'lower'

AT_LEAST = 'at_least'

AT_MOST = 'at_most'

exception algomancy_scenario.keyperformanceindicator.KpiError(message)

Bases: Exception

Exception raised for errors during KPI computation or validation.

Parameters:

message (str) – Explanation of the error.

class algomancy_scenario.keyperformanceindicator.BaseKPI(name, better_when, base_measurement, threshold=None)

Bases: ABC

Abstract base class for all Key Performance Indicators.

A BaseKPI encapsulates the logic for computing a metric from scenario results and formatting it for display. It supports both continuous tracking and binary success/failure checks via thresholds.

Parameters:

• name (str) – The human-readable name of the KPI.

• better_when (ImprovementDirection) – The ImprovementDirection for this KPI.

• base_measurement (BaseMeasurement) – The BaseMeasurement defining the unit and formatting preferences.

• threshold (float | None) – An optional numeric threshold for binary KPIs. Required if better_when is AT_LEAST or AT_MOST.

Notes

• Subclasses MUST implement the compute method.

• The value of the KPI is typically set via compute_and_check.

property measurement: Measurement

Returns the underlying Measurement object for the KPI value.

property name: str

Returns the name of the KPI.

property better_when: ImprovementDirection

Returns the improvement direction of the KPI.

property is_binary_kpi: bool

Returns True if the KPI has a binary success/failure criterion.

A KPI is binary if its better_when is AT_LEAST or AT_MOST.

property success: bool

Returns True if the KPI meets its threshold criteria.

Returns:

Boolean indicating success.

Raises:

ValueError – If the KPI is not binary or if no threshold is defined.

property value: float | None

Returns the current numeric value of the KPI, or None if not yet computed.

get_threshold_str(unit=None)

Returns a formatted string of the KPI’s threshold.

Parameters:

unit (Unit | None) – Optional unit to scale the threshold to before formatting.

Returns:

Formatted threshold string.

Return type:

str

pretty(unit=None)

Returns a human-friendly string of the KPI result.

For binary KPIs, returns “✓” or “✗”. For continuous KPIs, returns the value with its unit.

Parameters:

unit (Unit | None) – Optional unit to scale the value to.

Returns:

A string representation of the KPI state or value.

Return type:

str

get_pretty_unit()

Returns the unit that would be used for pretty-printing the current value.

details(unit=None)

Returns a human-friendly string of the numeric KPI value and unit.

Unlike pretty(), this always returns the numeric value even for binary KPIs.

Parameters:

unit (Unit | None) – Optional unit to scale the value to.

Returns:

Formatted string of the value and unit.

Return type:

str | None

abstractmethod compute(result)

Abstract method to compute the KPI value from scenario results.

Parameters:

result (BASE_RESULT_BOUND) – The scenario result data.

Returns:

The computed numeric value.

Return type:

float

compute_and_check(result)

Computes the KPI value and updates the internal state.

This method calls compute(result), validates that the returned value is numeric, and updates the KPI’s value.

Parameters:

result (BASE_RESULT_BOUND) – The scenario result data.

Raises:

KpiError – If computation fails or returns a non-numeric value.

to_dict()

Returns a JSON-serializable dictionary representation of the KPI.

unit is the symbol of the KPI’s base unit (e.g. "s" for seconds); threshold is the raw numeric threshold (or None for non-binary KPIs). For pretty-printed values use pretty() / details().