Skip to main content

Reproducibility

For every production run retain:

  • GeoSC version and Git commit when run from source;
  • Python version and dependency environment;
  • exact YAML files and CLI command;
  • input file hash, extract timestamp, and geography version;
  • output artefacts and logs;
  • all random seeds and reported backend fields;
  • eligibility, exclusions, estimand, and window decisions.

Inference defaults sampled-placebo work to sparse_sc_placebo_seed: 110011. Power uses random_seed to spawn per-grid seeds and then per-simulation placebo seeds. Repeated seeded CPU runs with identical inputs and configuration are designed to reproduce their simulation rows. Parallel and sequential seeded runs share the configuration-seed strategy.

When a power seed is set, GPU DGP estimation and generation are disabled. An unseeded GPU run is performance-oriented and should not be treated as bitwise reproducible. Floating-point results can also vary across platforms and library versions; preserve the environment where exact replay matters.

Placebo sampling is reproducible only when the seed and eligible unit ordering are unchanged. Record possible_placebos, effective_max_n_pl, mode, and seed.

Documentation figures

The counterfactual, placebo and workflow SVGs share one illustrative panel: seed 29, one treated geography, 49 controls, 40 pre-period days, three cooldown days and 17 measured days. The treated geography’s untreated trajectory is a convex mix of three generated controls with added noise; four outcome units per day are injected after launch. Control outcomes include generated post-launch shocks. This deliberately constructed example is not a calibration study.

Rebuild the figures from a source checkout with the declared dependencies:

python scripts/build_docs_figures.py
python scripts/sync_docs_site.py
hugo --source docs-site --panicOnWarning

The script makes one native inference call after GeoSC resolves the cooldown. It retains honest placebo means from that call rather than reconstructing them from non-honest control predictions. It checks that the plotted measured gap averages to the reported ATT and that the displayed two-sided add-one p-value matches the native result. It performs no power grid or calibration study.

The numerical source records the seed, configuration summary and plotted values, with null for the excluded cooldown predictions. The native learner retains its documented deterministic CV seeds; no capped-placebo sampling is needed for this exhaustive single-treatment run. Numerical results can vary with dependency versions; regenerate all three figures together instead of mixing runs. Stable SVG settings omit timestamps and preserve text labels for browser zoom and accessibility.

Mathematical notation

Use a MyST math directive for display equations that contain Markdown-sensitive operators or subscripts:

```{math}
x_i + y_i = z_i
```

Sphinx reads the directive directly. The synchroniser converts its opening fence to math for Relearn’s native renderer, preserving the TeX body. The verified website build uses Hugo Extended 0.169.0 and Relearn 9.4.0, which renders math during the build. No custom client-side MathJax loader is needed.