AMMM Troubleshooting Guide
This guide covers common AMMM V2 issues and practical fixes.
Quick Triage
Section titled “Quick Triage”- Confirm inputs were detected correctly (
data-config/or explicit--data/--config/--holidays). - Check stage outputs in your run folder, especially
50_diagnostics/. - If diagnostics failed, resolve those before consuming
40_decomposition/or70_optimisation/. - Re-run with explicit sampler overrides (
--draws,--tune,--chains,--target-accept) when needed.
Common Issues
Section titled “Common Issues”1. Multiple files auto-detected in data-config/
Section titled “1. Multiple files auto-detected in data-config/”Typical error:
ValueError: Multiple data files found in data-config/: [...]AMMM expects:
- one config file (
.ymlor.yaml) - one data file (
.csvor.xlsx) - optional one holidays file (
holidays.csvorholidays.xlsx)
Fix: keep only one active data/config file in data-config/, or pass explicit CLI paths.
2. GPU initialisation fails, fallback to CPU
Section titled “2. GPU initialisation fails, fallback to CPU”Typical message:
GPU initialization failed ... Falling back to CPU.Checks:
- verify compatible
jax/jaxlibversions - run with
--jax(and--gpuwhen CUDA is configured) - confirm CUDA drivers/toolkit are available in the current environment
CPU fallback is safe; it only affects speed.
3. Tensor/cache shape issues between runs
Section titled “3. Tensor/cache shape issues between runs”AMMM isolates PyTensor cache per run, but environment-level cache contamination can still occur in long-lived sessions.
Fixes:
- restart the Python process
- run via
python runme.pyfrom a fresh shell - use
ammm-cache infoandammm-cache pruneif cache buildup is severe
4. Convergence gate fails (diagnostics_gating: strict)
Section titled “4. Convergence gate fails (diagnostics_gating: strict)”Symptoms:
- pipeline stops after fitting
50_diagnostics/convergence_report.jsonhasconverged: false
Fixes:
- increase
--tune,--draws,--target-accept - simplify or tighten priors
- inspect
50_diagnostics/rank_trace.pngand50_diagnostics/energy_diagnostic.png
Behaviour by gate mode:
strict: halt on failed convergence gatewarn: continue with warningsoff: disable gate enforcement
5. Many divergences after tuning
Section titled “5. Many divergences after tuning”This often signals geometry issues in posterior sampling.
Try:
- higher
target_accept(for example0.9to0.95) - stronger priors for weakly identified parameters
- fewer highly collinear controls/media terms
6. Calibration diagnostics indicate poor calibration
Section titled “6. Calibration diagnostics indicate poor calibration”Check:
50_diagnostics/calibration_report.json(well_calibrated)50_diagnostics/calibration_pit_histogram.png50_diagnostics/calibration_pit_ecdf.png
If calibration is poor, review priors, feature design, and potential structural misspecification.
7. Results appear missing
Section titled “7. Results appear missing”AMMM V2 writes to stage folders, not legacy results/csv|png|json|other.
Expected layout:
00_run_metadata/10_pre_diagnostics/20_model_fit/30_model_assessment/40_decomposition/50_diagnostics/60_response_curves/70_optimisation/80_interpretation/
8. Large artefacts blocked by Git hosting
Section titled “8. Large artefacts blocked by Git hosting”Files such as model.dill and model.nc can exceed hosting limits.
Use .gitignore for generated artefacts (especially under results/), and avoid committing run outputs.
Testing and Reproduction
Section titled “Testing and Reproduction”Recommended test command:
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest -q -p pytest_mockFor quick checks, run targeted tests by module or feature area.
Logging Noise
Section titled “Logging Noise”If third-party debug logs are noisy (for example cmdstanpy), reduce logger verbosity in your runtime configuration before model fitting.
Interpretation Caveat
Section titled “Interpretation Caveat”Passing diagnostics indicates computational and predictive adequacy. It does not, by itself, establish causal validity.