AIHW-Bench
AI Hardware Benchmark Suite
Measure AI workloads. Understand the hardware. Compare the results. Reproduce the experiment.
A reproducible Python framework for benchmarking, profiling, comparing, and reporting AI workloads across CPUs, GPUs, embedded systems, simulator-backed environments, and extensible accelerator backends.
Reproducible experiments
Resolved configuration, deterministic suite manifests, immutable sessions, checksums, and portable reports make a measurement reviewable.
Hardware-aware reporting
Capture CPU, GPU, embedded, accelerator, runtime, thermal, and power-policy context without overstating execution support.
Compare with evidence
Inspect latency, throughput, resource metrics, diagnostics, session deltas, reports, and dashboard history.
Designed to extend
Use backends, loaders, metrics, reporters, visualizers, and plugins to integrate supported runtimes and hardware.
From workload to evidence
Configuration + workload
│
▼
Benchmark engine ──► backend + hardware inspection
│
▼
Immutable benchmark session
│
├──► metrics and statistics
├──► reports and export artifacts
├──► dashboard history and comparisons
└──► local benchmark analysis assistant
Start in minutes
python -m pip install aihw-bench
aihw-bench benchmark --backend reference --warmup 1 --iterations 10 --report json
Continue with the Quick Start, explore the CLI, or materialize the official suite.
Hardware support, stated precisely
| Capability state | What it means |
|---|---|
| Detected | A host or runtime probe identified the hardware/software. |
| Reportable | AIHW-Bench can capture and display its metadata. |
| Runnable | An executable backend exists for the selected target. |
| Accelerated | The workload/runtime actually uses that acceleration path. |
Detection alone does not imply execution. For example, FPGA and RTL entries are reportable integration boundaries; the core package does not synthesize designs, program boards, or invoke vendor simulator binaries. Read the hardware support guide for the exact boundaries.
Project status
Stable v2.0.0 Python 3.12+ Local-first
The official suite uses deterministic synthetic inputs and reference fixtures / synthetic baselines. Real-device results require a compatible workload/runtime and a documented hardware test environment. The built-in assistant is a deterministic, metric-grounded local benchmark analysis assistant—not an LLM service.
Explore the documentation
| Start here | Build and extend | Operate and compare |
|---|---|---|
| Installation | Developer guide | Dashboard |
| Quick start | Plugin development | Reports |
| Official suite | API reference | Enterprise foundations |
The full engineering record—including architecture, security, storage, testing, and release specifications—remains available in Engineering.