Testing Guide

This page describes how testing works for AMMM V2 and how to run tests reliably.

Test Stack

• pytest is the primary test runner.
• pytest-mock is required for many tests.
• nox runs a Python 3.10–3.12 test matrix (nox -s tests).

Canonical Local Command

Use this command for local runs to avoid plugin autoload conflicts:

PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest -q -p pytest_mock

Running Targeted Tests

Run specific tests for faster iteration:

PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest -q -p pytest_mock tests/driver/test_workflow.py

PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest -q -p pytest_mock -k convergence

Test Categories

Unit tests

Validate isolated functions/classes in src/ modules (for example diagnostics, preprocessing, optimisation helpers).

Integration tests

Validate interactions across the V2 workflow (MMMBaseDriverV2, workflow execution, stage artefact generation).

Documentation-aware checks

When workflow/output behaviour changes, verify:

• stage folders remain correct (00_... to 80_...)
• machine-readable diagnostics fields are still present:
  • 50_diagnostics/convergence_report.json -> converged
  • 50_diagnostics/calibration_report.json -> well_calibrated
  • 50_diagnostics/pareto_k_summary.json -> ok

Nox Matrix

Run the multi-version suite:

nox -s tests

The configured session installs editable package dependencies and executes:

pytest -q -p pytest_mock

with plugin autoload disabled.

Recommended Workflow for Changes

1. Run targeted tests for touched modules.
2. Run at least one integration path if workflow/output contracts changed.
3. Run the canonical local command before opening PR.
4. Use pytest --collect-only if you need to validate discovery quickly.

Notes

• Generated run artefacts (results/...) are not test fixtures and should not be committed.
• Diagnostics passing supports computational/predictive adequacy checks, not causal proof.