Skip to main content

Subsections of Output Schemas

Donor Artefact Schemas

donor_eval_results.csv

Each row is a treated-candidate donor pair. Maintained columns include treatment and donor IDs; composite, fit, correlation, and seasonality scores; design-stage recommendation weight and role; rank; quality labels; warnings; metric failure reasons; correlation and its p-value; RMSE, percentage RMSE, MAPE, DTW; and metrics_complete.

Score normalisation and ranking occur within each treated geography. Cross- treatment score values are therefore not a common absolute scale.

donor_pool_quality.json

The shared JSON persistence contract applies: nested non-finite metrics become null, supported values are complete, and unsupported payloads fail explicitly. The JSON file is atomically replaced; the CSV and JSON pair is not a transaction.

Schema 1.0 records run configuration, an overall summary, and by_treatment summaries. Summary fields include recommendation counts, quality distribution, warning and metric-failure counts, weight role, maximum weight, dominant donor count, effective donor count, score and correlation summaries, strength score, and quality_assessment.

run_config.adaptive_scoring records scope, treatment_date, n_observations, outcome_mean, outcome_std (sample standard deviation), coefficient_of_variation, metric_weights, rmse_thresholds, and dtw_thresholds. The scope is all_geographies_pre_treatment; each observed row contributes equally to these statistics. The existing weight branch uses standard deviation divided by mean, with a value of one when the mean is zero. These adaptive thresholds are recorded settings, not calibrated acceptance tests; the separate explicit quality-screen thresholds retain their role.

quality_assessment is a deterministic screening label derived from configured thresholds. It is not a test of exchangeability and does not guarantee good SparseSC fit.

run_config.dtw_radius is the fixed approximation search radius (1), dtw_backend identifies the effective FastDTW callable module, and dtw_local_cost is absolute_difference. Pair-specific standardisation precedes approximate alignment and distance is divided by finite overlap count. The radius does not bound calendar lag. These fields are additive in schema 1.0.

Inference Result Schemas

JSON persistence contract

Analysis, validation and donor JSON artefacts use strict JSON: non-finite Python/NumPy numbers and pandas missing scalars become null, including inside arrays and nested containers. null means unavailable; it does not establish the reason or change the recorded result status. Finite estimates, interval bounds and other supported values retain their values. Finite extended-precision NumPy floats that overflow or underflow Python’s float range raise an error rather than becoming unavailable or zero.

Lists and NumPy arrays are written in full, without truncation or flattening. DataFrames retain split orientation (index, columns, data); Series retain a mapping from labels to values. Dates and times use ISO strings. JSON object keys are strings: supported scalar and date/time keys are converted explicitly. Duplicate Series labels, keys that collide after conversion (such as 1 and "1"), non-finite keys and unsupported values raise an error rather than silently discarding or stringifying data. Tuple values become JSON arrays; tuple keys and arbitrary Python objects are unsupported.

The diagnostics whitelist still omits raw fitted matrices and frames; their existing availability metadata remains. Accepted containers have no artificial size cap, so callers remain responsible for the size of explicitly supplied payloads. Historical files with non-standard NaN/Infinity tokens or truncated data must be regenerated from the original analysis inputs.

All analysis payloads are converted before any existing artefact is replaced. Each JSON file is written to a unique temporary file and atomically replaced; failed writes clean up their temporary file and preserve that destination. The artefact family is not a transaction: an I/O failure after an earlier successful replacement can leave files from different runs. JSON configuration saves reject non-finite settings rather than turning them into null, and serialisation errors leave the existing configuration file intact.

geolift_results.json

Schema 1.0 contains the primary unscaled result:

Field Contract
status ok, partial, or failed.
estimand att_unscaled.
att Average post-period treated-minus-counterfactual effect in outcome units.
p_value Two-sided in-space placebo p-value for the average effect.
p_value_method in_space_placebo.
n_placebos, max_n_placebos, possible_placebos, p_value_placebo_mode, placebo_seed, p_value_granularity Finite reference-set metadata.
ci_lower, ci_upper, ci, ci_source Unscaled SparseSC placebo interval or explicit unavailability.
se / se_available No estimator-provided standard error is claimed.
se_approx_from_ci Optional normal approximation from interval width; not a sampling SE.
significance_available, significant Detection availability and p_value < alpha flag.
analysis_window Launch, cooldown, and measurement metadata.
assumption_status, assumption_required, assumption_checks_completed Diagnostic/gate summary; advisory never establishes identification.
warnings, errors, caveats Interpretation controls.

Rendered inference contract

CLI, recipe and pipeline summaries lead with status, errors and warnings. A failed or unknown status, non-empty errors, missing unscaled estimand or non-finite ATT withholds primary inference. Otherwise, significance_available and the stored significant flag determine the displayed decision. An unavailable decision is not reported as non-significance; a retained finite p-value is explicitly diagnostic when significance is unavailable.

The nested ci object supplies canonical lower, upper and level values. Rendering requires ci.available: true, ci.source: sparsesc_unscaled_placebo, ci_source: unscaled, finite ordered bounds and a numeric level strictly in (0, 1). For ci.level: 0.8, the label is 80% CI for average ATT (unscaled). Top-level bounds alone do not establish availability or level, and scaled diagnostics never replace the primary interval. Unavailable intervals retain their recorded unavailable_reason when present.

plot_results() rejects failed or incomplete primary results. Partial plots show status, warnings and uncertainty/significance limitations. The horizontal band represents the interval for the average ATT, not pointwise or simultaneous coverage of the period-by-period effect path. Older incomplete dictionaries must be regenerated; these presentation checks do not change schema 1.0 or the estimator’s numerical output.

geolift_diagnostics.json

Contains pre-period RMS effect and p-value, scaled post-effect diagnostics, period effects, fitted donor-weight summaries, synthetic-control availability, assumption state, analysis window, and warnings/errors. Scaled effects are not interchangeable with the unscaled top-level ATT and interval.

Validation artefacts

data_validation.json records file and model panel construction. When imputation is used, imputation.method is linear_interpolate_within_analysis_windows and imputation.windows records affected-cell counts, period counts, and start/end dates for pre_treatment, cooldown, and measurement. Input missing counts remain available after preparation. When CSV covariates are supplied, covariates records method: pre_treatment_period_mean, the intervention date, retained pre-period count, missing pre-value count, and missing-value handling rule. These are additive fields in schema 1.0.

assumption_validation.json uses schema 1.2. Regression and visual pre-trend checks and the interference screen have gate_eligible: false, with signal, no_signal or indeterminate statuses. A regression signal concerns mean raw unit slopes, not the fitted synthetic trajectory. An unavailable or failed advisory diagnostic remains visible without changing estimator status or significance availability.

Only the experimental pre-period placebo method can have gate_eligible: true and pass, fail, error or skipped status. Its numerical procedure is unchanged; passing means only that its configured discrepancy criterion was met. It does not establish causal identification or calibrated error rates.

When checks ran without an eligible gate, overall_status is advisory and overall_valid is null. Otherwise the overall status reflects eligible placebo checks, or is skipped when none ran. overall_valid refers only to that diagnostic gate. all_checks_completed concerns eligible gate checks when present, otherwise advisory checks; an indeterminate check is incomplete. Remove required-gate requests from regression/visual configurations, or explicitly request parallel_trends_method: placebo. Regenerate schema 1.1 artefacts before current release validation.

Pipeline Report Schema

The pipeline writes geolift_pipeline_report.md and geolift_pipeline_report.html. Each identifies the three config paths and summarises available stage artefacts.

The power section reports grid-based MDE by duration at configured target_power and excludes rows where valid is false. The donor section lists the five highest composite-score rows per treatment from the CSV. The inference section reports available ATT, p-value, interval, and assumption status.

These reports are convenience summaries. They do not contain the full data validation, power failure diagnostics, donor pool quality JSON, assumption checks, or sensitivity evidence. Audit and interpret the source artefacts.

Power Analysis Results Schema

power_analysis_results.csv has one row per effect-size and duration pair.

Field Meaning
schema_version Row contract version (2.0); older rows are withheld from decision consumers.
estimator_settings Deterministic JSON recording effective native method/CV/defaults, match-space learner, cross-fitting folds/seed, T0/T1, placebo cap and effect-CI request. Additive schema 2.0 provenance.
target_power CLI planning threshold copied from YAML.
effect_size, duration, alpha Simulated grid and detection threshold.
power n_significant / n_simulations, available only when every draw succeeds.
conditional_power n_significant / n_successful; unavailable when no draws succeed.
power_all_attempt_lower_bound, power_all_attempt_upper_bound n_significant / n_simulations and (n_significant + n_failed) / n_simulations; completion bounds, not confidence intervals.
power_ci_lower, power_ci_upper Wilson interval for a complete run, using alpha as the tail setting; unavailable if any draw fails.
n_simulations, n_successful, n_significant, n_failed Monte Carlo counts.
n_invalid_p_values, n_estimation_failures Invalid estimator output versus generation/estimation failures; their sum equals n_failed. Failure categories retained from the previous schema.
failure_rate, valid Failure diagnostics; valid requires zero failures. It does not validate DGP plausibility.
random_seed, simulation_seed, placebo_seed_root, placebo_seed_strategy Reproducibility metadata.
dgp_rank, dgp_explained_var, dgp_factor_dynamics Fitted DGP metadata.
requested_max_n_pl, possible_placebos, effective_max_n_pl, p_value_placebo_mode Placebo support.
dgp_backend, generation_backend, seeded_reproducibility_mode Execution metadata.
effect_pattern, effect_baseline, effect_full_absolute, effect_baseline_definition Injection contract.
detection_rule, power_denominator Strict detection rule and primary denominator all_simulations.
warnings Semicolon-delimited row warnings.

Direct calls to SparseSCPowerCalculator.calculate_power() do not add the CLI’s target_power column; target power affects MDE selection, not simulation.

New rows use estimator_settings.model_type: retrospective, fast: true and match_space: ridge_cv. cv_options records explicit settings with null native defaults for cv, scoring and gcv_mode. Honest cross-fitting uses ten folds and seed 110011; T0, T1, max_n_pl and ret_CI: false describe the simulated native call. Per-draw placebo seeds remain described in the existing row seed fields. This does not capture every private solver detail or prove calibration.

Older schema 2.0 rows can lack this new provenance. Numeric decision eligibility is unchanged; regenerate rows when method provenance is required, especially if earlier non-default settings were supplied and silently ignored.

A successful draw must return a finite real numeric p-value in [0, 1]. Missing values, arrays, strings and booleans are invalid outputs. Valid zero and one are retained, and significance uses the original scalar’s strict p_value < alpha comparison. Counts and warnings distinguish invalid p-values from execution failures. If any draw fails, ordinary power and its primary interval bounds are NaN in the DataFrame and empty cells in the CSV. The row is valid: false and cannot produce ordinary plots or MDE recommendations. The conditional rejection rate remains diagnostic; all-failed rows have unavailable conditional rates and all-attempt completion bounds [0, 1].

Completion bounds describe the observed attempts with unknown failed-draw rejection outcomes. They do not include Monte Carlo uncertainty or bound true population power. With no failures they collapse to the observed rejection rate, although its Wilson interval still has sampling width.

The retired failure_rate_threshold output is absent. Remove the retired power_failure_rate_threshold input and regenerate old results. Decision consumers require schema 2.0, complete positive whole attempt counts, matching success counts, zero failures, true validity, and finite numeric primary fields. They normalise accepted CSV numbers before sorting, comparisons and plotting; unsupported or malformed rows remain excluded even with a stale true flag.