Domain Models
Domain modules define immutable models, typed errors, and protocol boundaries.
Models
Pydantic data models for persisted AIHW-Bench records.
FrozenModel
Bases: BaseModel
Base class for immutable domain models.
SessionStatus
Bases: StrEnum
Lifecycle states persisted for benchmark sessions.
ExecutionPhase
Bases: StrEnum
Execution phases recorded in session data.
ExecutionStatus
Bases: StrEnum
Execution result status.
MetricKind
Bases: StrEnum
Metric provenance category.
DiagnosticSeverity
Bases: StrEnum
Severity for user-facing diagnostics.
ConfigurationSource
WorkloadConfig
BackendConfig
ExecutionConfig
ProfilingConfig
MetricsConfig
ReportsConfig
StorageConfig
PluginsConfig
Configuration
Bases: FrozenModel
Resolved AIHW-Bench configuration.
The model is immutable and records the sources used to build it.
explain_sources()
Return readable source descriptions in precedence order.
to_resolved_yaml()
Serialize the resolved configuration as YAML.
Diagnostic
Metric
Bases: FrozenModel
Measured, derived, estimated, or metadata metric.
format()
Format a metric for human-readable output.
compare_to(other)
Return numeric delta from another compatible metric.
HardwareProfile
Bases: FrozenModel
Host and target hardware metadata captured for a session.
summary()
Return a compact hardware summary.
capability_report()
Return normalized capabilities for backend selection and reports.
ModelMetadata
Bases: FrozenModel
Metadata for a benchmark workload or model.
fingerprint()
Return a stable fingerprint for comparison and caching.
to_dict()
Return a JSON-serializable model description.
summary()
Return summary fields suitable for reports.
ExecutionResult
Bases: FrozenModel
One backend execution event.
require_timezone(value)
classmethod
Require timezone-aware timestamps.
validate_time_order()
Ensure execution end time is not before start time.
Profile
BenchmarkResult
Bases: FrozenModel
Aggregate result for a workload/backend/device combination.
is_successful()
Return whether the benchmark result completed successfully.
primary_metric(name)
Return a primary metric by name.
RunHistory
PluginMetadata
Bases: FrozenModel
Validated plugin descriptor metadata.
is_compatible(core_api_version)
Return whether the plugin API version matches the core API version.
provider_names()
Return provider names declared by this plugin.
ExportArtifact
Bases: FrozenModel
Generated artifact metadata and integrity information.
require_created_at_timezone(value)
classmethod
Require timezone-aware artifact timestamps.
BenchmarkSession
Bases: FrozenModel
Immutable top-level benchmark session record.
require_session_timezone(value)
classmethod
Require timezone-aware session timestamps.
validate_completion_time()
Ensure completed sessions include completion time.
finalize(status, *, completed_at=None)
Return a finalized copy of the session.
add_artifact(artifact)
Return a copy of the session with a new artifact.
Errors
Typed exception hierarchy for AIHW-Bench.
ErrorContext
dataclass
Structured context attached to expected AIHW-Bench errors.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cause
|
str
|
Human-readable reason the operation failed. |
required |
suggestion
|
str
|
Recommended user or developer action. |
required |
documentation
|
str | None
|
Optional documentation reference. |
None
|
AihwBenchError
Bases: Exception
Base class for expected AIHW-Bench errors.
ConfigurationError
ValidationError
BackendError
RuntimeExecutionError
ModelError
ProfilerError
MetricError
PluginError
SessionError
ReportError
ExportError
SecurityError
InternalError
Ports
Domain port protocols for benchmark execution.
BenchmarkBackend
Bases: Protocol
Contract for a backend that can prepare, execute, and clean up a workload.
supports_device(device)
Return whether the backend can target the requested device string.
validate(configuration, hardware, workload)
Validate the backend against the selected configuration and hardware.
prepare(configuration, hardware, workload)
Prepare backend resources before the benchmark starts.
execute(phase, iteration, configuration, hardware, workload)
Run one warmup or measured iteration and return backend observations.
cleanup()
Release any resources acquired during preparation.
HardwareInspector
Bases: Protocol
Collect host and target hardware metadata for a benchmark session.
inspect()
Return a hardware profile for the current machine.
BackendRegistry
Bases: Protocol
Resolve and validate benchmark backends.
register(backend)
Register a backend implementation.
resolve(name)
Return a backend by registry name.
select(configuration, hardware, workload=None)
Select the best backend for the resolved configuration and hardware.
validate(backend, configuration, hardware, workload=None)
Validate a backend for the requested configuration and hardware.
MetricProvider
Bases: Protocol
Compute metrics from benchmark runs and captured observations.
compute(configuration, runs, hardware, workload)
Return metrics produced from the supplied benchmark session context.
Reporter
Bases: Protocol
Render a report view into one durable artifact format.
render(view)
Return the serialized report payload.
Visualizer
Bases: Protocol
Build a dashboard-ready chart specification from a benchmark session.
build(session)
Return a chart specification for the supplied session.
Model Support
Protocols and request/result models for model loading.
ModelLoadRequest
dataclass
Request describing a model source to be loaded.
LoadedModel
dataclass
Resolved model handle and extracted metadata.
ModelLoader
Bases: Protocol
Contract implemented by format-specific model loaders.
can_load(source)
Return whether the loader can handle the given file path.
load(request)
Load a model file and return metadata plus the resolved handle.
ModelLoaderCatalog
Bases: Protocol
Registry interface used by application services.
load(request)
Load a model from an explicit load request.
load_workload(workload)
Load a model from resolved configuration.