Skip to content

Application Services

Application services coordinate framework behavior and depend on domain protocols instead of concrete infrastructure.

Benchmark Engine

Benchmark execution engine and supporting helpers.

BenchmarkLifecycleState

Bases: StrEnum

Lifecycle states for benchmark orchestration.

BenchmarkStep dataclass

One scheduled benchmark step.

ExecutionPlan dataclass

Ordered execution steps for a benchmark session.

measured_steps property

Return the measured steps only.

TimingSample dataclass

Bases: Generic[T]

A timed operation result with timestamps and a payload.

TimingEngine

Bases: Protocol

Measure a callable and return timestamps and duration.

measure(operation)

Execute one operation and measure its duration.

MonotonicTimingEngine

High-resolution timing engine backed by the system monotonic clock.

ScriptedTimingEngine

Deterministic timing engine for tests and smoke validation.

ExecutionScheduler

Build a deterministic execution plan for warmup and measured iterations.

StatisticsEngine

Compatibility facade for the canonical :class:StatisticalAggregator.

Historical callers receive duration-specific field names while all numeric calculations are delegated to the single metrics implementation.

summarize(durations)

Return duration statistics using the stable benchmark-engine schema.

BenchmarkRequest dataclass

Validated benchmark request accepted by the benchmark service.

BenchmarkOutcome dataclass

Result returned by the benchmark service.

LifecycleTracker

Validate benchmark lifecycle transitions.

state property

Return the current lifecycle state.

transition(new_state)

Move to a new lifecycle state if the transition is valid.

BenchmarkService

Coordinate benchmark execution over a configured backend.

run(request)

Run one benchmark session and return the final immutable session record.

Metrics Engine

Core metric computation for benchmark results and reporting summaries.

MetricsBundle dataclass

Computed metrics and report-ready summary data.

metrics property

Return all computed metrics in report order.

to_report_payload()

Return a stable payload for downstream report builders.

StatisticalAggregator

Compute deterministic statistical summaries for numeric samples.

summarize(values)

Return count, min, max, mean, median, standard deviation, and percentiles.

ResultSerializer

Serialize metric outputs for storage and report integration.

bundle_to_dict(bundle)

Serialize a metrics bundle with deterministic keys.

metrics_to_dicts(metrics)

Serialize metric records for JSON, Markdown, CSV, or HTML reporters.

CoreMetricsEngine

Compute core latency, throughput, resource, utilization, and FLOPS metrics.

compute(*, session_id, configuration, runs, hardware, workload, failure_message=None, latency_statistics=None)

Compute metrics from successful measured runs and backend observations.

Reporting Engine

Report generation services and report view models.

ReportMetadata dataclass

Metadata recorded for every generated report artifact.

to_dict()

Return a JSON-serializable metadata dictionary.

ReportView dataclass

Canonical report view consumed by all built-in reporters.

to_dict()

Return a JSON-serializable report view.

ReportRequest dataclass

Input accepted by the report service.

ReportViewBuilder

Build report-ready summaries from immutable benchmark sessions.

build(session, *, report_format)

Create a report view for one output format.

build_many(session, *, report_formats)

Build views for several formats while materializing shared content once.

ReportService

Validate report requests, manage export directories, and write artifacts.

generate(request)

Generate requested report artifacts and return their metadata.

Visualization Engine

Visualization models, services, and export helpers.

ChartTrace dataclass

One data series in a chart specification.

to_dict()

Return a Plotly-compatible trace dictionary.

ChartSpec dataclass

Dashboard-ready chart specification.

to_dict()

Return a JSON-serializable chart specification.

ChartComponent dataclass

Embeddable dashboard/report component.

to_dict()

Return a dashboard-ready component payload.

ChartArtifact dataclass

Generated chart artifact metadata.

ChartExportService

Export chart specifications to HTML, SVG, and PNG.

export(spec, output_dir, *, formats=('html', 'svg', 'png'))

Write chart artifacts and return checksum metadata.

render(spec, export_format)

Render a chart specification to one supported format.

render_html(spec)

Render an interactive Plotly-compatible HTML fragment.

render_svg(spec)

Render a deterministic static SVG fallback.

render_png(spec)

Render a small valid PNG preview for artifact pipelines.

VisualizationService

Build chart specifications and dashboard components from benchmark sessions.

build_specs(session, *, families=None)

Build chart specifications for the requested chart families.

build_dashboard_components(session, *, families=None)

Build HTML components suitable for reports or dashboard shells.

export(session, output_dir, *, chart_formats=('html', 'svg', 'png'), families=None)

Export chart artifacts for the requested session.

stable_chart_id(session_id, family)

Return a stable chart identifier for session and family.

numeric_metric(session, name)

Return a numeric metric value by name.

measured_runs(session)

Return successful measured execution results.

finite_or_zero(value)

Return a finite float or zero when a metric is unavailable.

Plugin Framework

Plugin discovery, validation, registration, and lifecycle management.

PluginProviderKind

Bases: StrEnum

Provider categories accepted by the AIHW-Bench plugin registry.

PluginLifecycleState

Bases: StrEnum

Lifecycle state tracked for each discovered plugin.

PluginManifest

Bases: BaseModel

Validated plugin manifest returned by simple plugin packages.

reject_self_dependency(value) classmethod

Normalize dependency names and reject blanks.

to_metadata()

Convert the manifest to the domain plugin metadata model.

PluginContext dataclass

Context passed to plugin lifecycle callbacks.

PluginRegistration dataclass

Registration payload returned by an AIHW-Bench plugin entry point.

from_manifest(manifest, *, providers=None, validate=None, activate=None, deactivate=None) classmethod

Create a registration from a manifest and optional provider objects.

PluginRegistry

Validate plugins and expose registered providers by category.

register(registration)

Validate and register one plugin registration.

register_many(registrations)

Register plugins after resolving same-process plugin dependencies.

activate_all(context=None)

Run validation and activation callbacks for registered plugins.

deactivate_all(context=None)

Run deactivation callbacks for active plugins in reverse registration order.

providers_for(kind)

Return registered providers for a provider category.

provider(kind, name)

Return one registered provider by category and provider name.

plugin_metadata()

Return plugin metadata in deterministic order.

PluginDiscovery

Discover and normalize plugin registrations from Python entry points.

discover_entry_points()

Load plugin registrations from installed package entry points.

discover_objects(objects)

Normalize in-memory plugin objects for tests and embedded composition roots.

registration_from_object(plugin_object)

Normalize a plugin object, factory, manifest, or registrar into a registration.

PluginManager

High-level plugin loader that combines discovery, registry, and lifecycle.

load(registrations=None, *, activate=True)

Discover, register, and optionally activate plugins.