Validation and performance

Use this page to select a check and understand the scope of published results. For interpreting an ordinary simulation, start with the scorecard and KPI reference.

Run checks from a source checkout

Install the development dependencies as described in the developer guide. Write generated reports to a separate output directory:

python scripts/audit/run_full_audit.py --strict --output-dir /tmp/citylearn-audit

This verifies physical scenarios, entity contracts, bounded probes of the public datasets and fixed-command regression references. Failed or missing checks fail verification; expected results are not regenerated by an ordinary run.

Investigate	Tool
Device physics and electrical limits	scripts/audit/audit_physics.py
Entity tables, actions and topology	scripts/audit/audit_entity_contract.py
Dataset loading and bounded execution	scripts/audit/audit_datasets.py
Independent energy and cash reconciliation	scripts/audit/audit_balances.py
Time axes and reproducibility	scripts/audit/audit_temporal.py
Continuous execution and resource use	scripts/audit/audit_long_run.py
Component and evaluation timings	Profiling commands in the developer guide

Use each script’s --help for its inputs and output options. Public datasets remain unchanged; no SoftCPS production datasets are needed for the default checks.

Published validation and measured improvements

The 3.1.0 release notes retain the functional changes, validation counts, benchmark protocol and measured before/after values. They distinguish the corrected functional baseline from the original 3.0.2 release. Older measurements remain in the release archive.

Bounded tests and integration benchmarks do not establish annual validation of every dataset, controller performance or calibration against real equipment. Reproduction requires matching inputs, parameters, runtime and validation scope. See the migration guide before comparing versions.