Output Schemas
Interpret fields, diagnostic statuses, and limits in generated artefacts.
Interpret fields, diagnostic statuses, and limits in generated artefacts.
donor_eval_results.csvEach row is a treated-candidate donor pair. Maintained columns include treatment
and donor IDs; composite, fit, correlation, and seasonality scores; design-stage
recommendation weight and role; rank; quality labels; warnings; metric failure
reasons; correlation and its p-value; RMSE, percentage RMSE, MAPE, DTW; and
metrics_complete.
Score normalisation and ranking occur within each treated geography. Cross- treatment score values are therefore not a common absolute scale.
donor_pool_quality.jsonThe shared JSON persistence contract
applies: nested non-finite metrics become null, supported values are complete,
and unsupported payloads fail explicitly. The JSON file is atomically replaced;
the CSV and JSON pair is not a transaction.
Schema 1.0 records run configuration, an overall summary, and
by_treatment summaries. Summary fields include recommendation counts, quality
distribution, warning and metric-failure counts, weight role, maximum weight,
dominant donor count, effective donor count, score and correlation summaries,
strength score, and quality_assessment.
run_config.adaptive_scoring records scope, treatment_date,
n_observations, outcome_mean, outcome_std (sample standard deviation),
coefficient_of_variation, metric_weights, rmse_thresholds, and
dtw_thresholds. The scope is all_geographies_pre_treatment; each observed row
contributes equally to these statistics. The existing weight branch uses
standard deviation divided by mean, with a value of one when the mean is zero.
These adaptive thresholds are recorded settings, not calibrated acceptance
tests; the separate explicit quality-screen thresholds retain their role.
quality_assessment is a deterministic screening label derived from configured
thresholds. It is not a test of exchangeability and does not guarantee good
SparseSC fit.
run_config.dtw_radius is the fixed approximation search radius (1),
dtw_backend identifies the effective FastDTW callable module, and
dtw_local_cost is absolute_difference. Pair-specific standardisation precedes
approximate alignment and distance is divided by finite overlap count. The
radius does not bound calendar lag. These fields are additive in schema 1.0.
Analysis, validation and donor JSON artefacts use strict JSON: non-finite
Python/NumPy numbers and pandas missing scalars become null, including inside
arrays and nested containers. null means unavailable; it does not establish
the reason or change the recorded result status. Finite estimates, interval
bounds and other supported values retain their values.
Finite extended-precision NumPy floats that overflow or underflow Python’s
float range raise an error rather than becoming unavailable or zero.
Lists and NumPy arrays are written in full, without truncation or flattening.
DataFrames retain split orientation (index, columns, data); Series retain
a mapping from labels to values. Dates and times use ISO strings. JSON object
keys are strings: supported scalar and date/time keys are converted explicitly.
Duplicate Series labels, keys that collide after conversion (such as 1 and
"1"), non-finite keys and unsupported values raise an error rather than
silently discarding or stringifying data. Tuple values become JSON arrays;
tuple keys and arbitrary Python objects are unsupported.
The diagnostics whitelist still omits raw fitted matrices and frames; their
existing availability metadata remains. Accepted containers have no artificial
size cap, so callers remain responsible for the size of explicitly supplied
payloads. Historical files with non-standard NaN/Infinity tokens or truncated
data must be regenerated from the original analysis inputs.
All analysis payloads are converted before any existing artefact is replaced.
Each JSON file is written to a unique temporary file and atomically replaced;
failed writes clean up their temporary file and preserve that destination.
The artefact family is not a transaction: an I/O failure after an earlier
successful replacement can leave files from different runs. JSON configuration
saves reject non-finite settings rather than turning them into null, and
serialisation errors leave the existing configuration file intact.
geolift_results.jsonSchema 1.0 contains the primary unscaled result:
| Field | Contract |
|---|---|
status |
ok, partial, or failed. |
estimand |
att_unscaled. |
att |
Average post-period treated-minus-counterfactual effect in outcome units. |
p_value |
Two-sided in-space placebo p-value for the average effect. |
p_value_method |
in_space_placebo. |
n_placebos, max_n_placebos, possible_placebos, p_value_placebo_mode, placebo_seed, p_value_granularity |
Finite reference-set metadata. |
ci_lower, ci_upper, ci, ci_source |
Unscaled SparseSC placebo interval or explicit unavailability. |
se / se_available |
No estimator-provided standard error is claimed. |
se_approx_from_ci |
Optional normal approximation from interval width; not a sampling SE. |
significance_available, significant |
Detection availability and p_value < alpha flag. |
analysis_window |
Launch, cooldown, and measurement metadata. |
assumption_status, assumption_required, assumption_checks_completed |
Diagnostic/gate summary; advisory never establishes identification. |
warnings, errors, caveats |
Interpretation controls. |
CLI, recipe and pipeline summaries lead with status, errors and warnings.
A failed or unknown status, non-empty errors, missing unscaled estimand or
non-finite ATT withholds primary inference. Otherwise, significance_available
and the stored significant flag determine the displayed decision. An
unavailable decision is not reported as non-significance; a retained finite
p-value is explicitly diagnostic when significance is unavailable.
The nested ci object supplies canonical lower, upper and level values.
Rendering requires ci.available: true, ci.source: sparsesc_unscaled_placebo,
ci_source: unscaled, finite ordered bounds and a numeric level strictly in
(0, 1). For ci.level: 0.8, the label is 80% CI for average ATT (unscaled).
Top-level bounds alone do not establish availability or level, and scaled
diagnostics never replace the primary interval. Unavailable intervals retain
their recorded unavailable_reason when present.
plot_results() rejects failed or incomplete primary results. Partial plots
show status, warnings and uncertainty/significance limitations. The horizontal
band represents the interval for the average ATT, not pointwise or simultaneous
coverage of the period-by-period effect path. Older incomplete dictionaries
must be regenerated; these presentation checks do not change schema 1.0 or
the estimator’s numerical output.
geolift_diagnostics.jsonContains pre-period RMS effect and p-value, scaled post-effect diagnostics, period effects, fitted donor-weight summaries, synthetic-control availability, assumption state, analysis window, and warnings/errors. Scaled effects are not interchangeable with the unscaled top-level ATT and interval.
data_validation.json records file and model panel construction.
When imputation is used, imputation.method is
linear_interpolate_within_analysis_windows and imputation.windows records
affected-cell counts, period counts, and start/end dates for pre_treatment,
cooldown, and measurement. Input missing counts remain available after
preparation. When CSV covariates are supplied, covariates records
method: pre_treatment_period_mean, the intervention date, retained pre-period
count, missing pre-value count, and missing-value handling rule. These are
additive fields in schema 1.0.
assumption_validation.json uses schema 1.2. Regression and visual pre-trend
checks and the interference screen have gate_eligible: false, with signal,
no_signal or indeterminate statuses. A regression signal concerns mean raw
unit slopes, not the fitted synthetic trajectory. An unavailable or failed
advisory diagnostic remains visible without changing estimator status or
significance availability.
Only the experimental pre-period placebo method can have gate_eligible: true
and pass, fail, error or skipped status. Its numerical procedure is
unchanged; passing means only that its configured discrepancy criterion was
met. It does not establish causal identification or calibrated error rates.
When checks ran without an eligible gate, overall_status is advisory and
overall_valid is null. Otherwise the overall status reflects eligible
placebo checks, or is skipped when none ran. overall_valid refers only to
that diagnostic gate. all_checks_completed concerns eligible gate checks
when present, otherwise advisory checks; an indeterminate check is incomplete.
Remove required-gate requests from regression/visual configurations, or
explicitly request parallel_trends_method: placebo. Regenerate schema 1.1
artefacts before current release validation.
The pipeline writes geolift_pipeline_report.md and
geolift_pipeline_report.html. Each identifies the three config paths and
summarises available stage artefacts.
The power section reports grid-based MDE by duration at configured
target_power and excludes rows where valid is false. The donor section lists
the five highest composite-score rows per treatment from the CSV. The inference
section reports available ATT, p-value, interval, and assumption status.
These reports are convenience summaries. They do not contain the full data validation, power failure diagnostics, donor pool quality JSON, assumption checks, or sensitivity evidence. Audit and interpret the source artefacts.
power_analysis_results.csv has one row per effect-size and duration pair.
| Field | Meaning |
|---|---|
schema_version |
Row contract version (2.0); older rows are withheld from decision consumers. |
estimator_settings |
Deterministic JSON recording effective native method/CV/defaults, match-space learner, cross-fitting folds/seed, T0/T1, placebo cap and effect-CI request. Additive schema 2.0 provenance. |
target_power |
CLI planning threshold copied from YAML. |
effect_size, duration, alpha |
Simulated grid and detection threshold. |
power |
n_significant / n_simulations, available only when every draw succeeds. |
conditional_power |
n_significant / n_successful; unavailable when no draws succeed. |
power_all_attempt_lower_bound, power_all_attempt_upper_bound |
n_significant / n_simulations and (n_significant + n_failed) / n_simulations; completion bounds, not confidence intervals. |
power_ci_lower, power_ci_upper |
Wilson interval for a complete run, using alpha as the tail setting; unavailable if any draw fails. |
n_simulations, n_successful, n_significant, n_failed |
Monte Carlo counts. |
n_invalid_p_values, n_estimation_failures |
Invalid estimator output versus generation/estimation failures; their sum equals n_failed. Failure categories retained from the previous schema. |
failure_rate, valid |
Failure diagnostics; valid requires zero failures. It does not validate DGP plausibility. |
random_seed, simulation_seed, placebo_seed_root, placebo_seed_strategy |
Reproducibility metadata. |
dgp_rank, dgp_explained_var, dgp_factor_dynamics |
Fitted DGP metadata. |
requested_max_n_pl, possible_placebos, effective_max_n_pl, p_value_placebo_mode |
Placebo support. |
dgp_backend, generation_backend, seeded_reproducibility_mode |
Execution metadata. |
effect_pattern, effect_baseline, effect_full_absolute, effect_baseline_definition |
Injection contract. |
detection_rule, power_denominator |
Strict detection rule and primary denominator all_simulations. |
warnings |
Semicolon-delimited row warnings. |
Direct calls to SparseSCPowerCalculator.calculate_power() do not add the CLI’s
target_power column; target power affects MDE selection, not simulation.
New rows use estimator_settings.model_type: retrospective, fast: true and
match_space: ridge_cv. cv_options records explicit settings with null native
defaults for cv, scoring and gcv_mode. Honest cross-fitting uses ten folds
and seed 110011; T0, T1, max_n_pl and ret_CI: false describe the simulated
native call. Per-draw placebo seeds remain described in the existing row seed
fields. This does not capture every private solver detail or prove calibration.
Older schema 2.0 rows can lack this new provenance. Numeric decision
eligibility is unchanged; regenerate rows when method provenance is required,
especially if earlier non-default settings were supplied and silently ignored.
A successful draw must return a finite real numeric p-value in [0, 1].
Missing values, arrays, strings and booleans are invalid outputs. Valid zero and
one are retained, and significance uses the original scalar’s strict
p_value < alpha comparison. Counts and warnings distinguish invalid p-values
from execution failures. If any draw fails, ordinary power and its primary
interval bounds are NaN in the DataFrame and empty cells in the CSV. The row is
valid: false and cannot produce ordinary plots or MDE recommendations. The
conditional rejection rate remains diagnostic; all-failed rows have unavailable
conditional rates and all-attempt completion bounds [0, 1].
Completion bounds describe the observed attempts with unknown failed-draw rejection outcomes. They do not include Monte Carlo uncertainty or bound true population power. With no failures they collapse to the observed rejection rate, although its Wilson interval still has sampling width.
The retired failure_rate_threshold output is absent. Remove the retired
power_failure_rate_threshold input and regenerate old results. Decision
consumers require schema 2.0, complete positive whole attempt counts, matching
success counts, zero failures, true validity, and finite numeric primary fields.
They normalise accepted CSV numbers before sorting, comparisons and plotting;
unsupported or malformed rows remain excluded even with a stale true flag.