Skip to main content

Subsections of How-to Guides

Define the Estimand and Windows

Write the estimand before configuring GeoSC:

Average difference, in outcome units per observed period, between treated outcomes and their SparseSC counterfactual over the declared measurement window for the declared treated geographies.

GeoSC labels this att_unscaled. The top-level att is the average across the post-period effect path. If several geographies are treated, the SparseSC procedure forms a joint treated effect; do not describe the result as a unit-specific effect unless a separate analysis supports that claim.

Set:

  • intervention_date_str: first observed treatment period;
  • cooldown_periods: observed periods after launch excluded from measurement;
  • optional measurement_start_date_str: explicit start that must agree with the cooldown calculation;
  • min_pre_treatment_periods and min_post_treatment_periods: admissibility thresholds where supplied.

Cooldown periods are neither pre-period fit data nor measured post-period data. They count observations, not calendar units. A seven-period cooldown is seven days only for a complete daily panel.

Pre-specify the end of measurement by trimming the input panel to the intended window. The shipped inference config has no active end_date key.

Define Treatment and Donor Eligibility

Create and freeze an eligibility table before running GeoSC. Treat it as the analysis-universe contract: it records which geographies are treated, which untreated geographies are eligible controls, which geographies are excluded, and why.

Use one row per geography:

Column Purpose
unit_id Geography identifier used in the input panel.
role treated, eligible_control, or excluded.
included_in_panel Whether the unit is present in the versioned panel used by a stage.
reason Design reason for inclusion or exclusion.
decision_date Date the eligibility decision was made.
owner Person or team accountable for the decision.
contamination_notes Known campaign, operational, or measurement contamination.
leakage_notes Geographic adjacency or audience leakage concerns.

Example:

unit_id role included_in_panel reason decision_date owner contamination_notes leakage_notes
region_ne excluded false Adjacent to treatment region and exposed to overlapping out-of-home media. 2026-08-05 Measurement lead Same creative ran during the test window. Commuter leakage into treatment area.

Eligible controls should be unaffected by treatment, measured consistently, and plausible counterfactual contributors. Pre-period correlation and donor-screen scores are diagnostics; they are not eligibility rules and do not prove exchangeability. Exclude geographies only for design reasons established before reviewing post-treatment outcomes.

Freeze the Universe

  1. Define treated units and the first observed treatment period.
  2. Remove contaminated, leaking, inconsistently measured, or otherwise ineligible controls.
  3. Materialise the approved treated units and eligible controls as a versioned panel.
  4. Run power, donor evaluation, and inference from that versioned panel.
  5. If donor screening changes the eligible universe, create a new panel version and rerun the affected stages.

Do not tune exclusions, treatment dates, or donor membership after seeing post-treatment lift. That creates a different design and invalidates ordinary interpretation unless the search and multiplicity are explicitly modelled.

What GeoSC Includes

Power treats every non-treatment unit in its input as a control. Donor evaluation scores every eligible non-treatment unit in its own input. Inference uses the controls in its prepared panel, where controls are units without a treatment period. GeoSC does not automatically apply donor recommendations to the other stages.

For several treated geographies, confirm that the eligible control pool can support joint in-space placebo assignments: with N0N_0 controls and N1N_1 treated units, the exhaustive assignment count is (N0N1)\binom{N_0}{N_1}, provided N0≥N1N_0 \geq N_1. If the eligible control pool is too small, placebo inference is infeasible or too coarse for the planned alpha.

See also Donor Quality and the end-to-end synthetic study.

Diagnose a Weak Design

Work from failure mechanism to remedy.

Symptom Likely issue Action
Poor pre-period fit Donors cannot reproduce treated dynamics Revisit eligibility, geography level, outcome, and comparable history.
Coarse p-value granularity Too few effective placebo assignments Add eligible controls or reduce treated count only if the design permits it.
Fewer than ten retained controls Infeasible ten-fold honest cross-fitting Add eligible controls and check data-policy drops; changing match-space CV folds cannot enable this pool.
Dominant fitted donor weight Counterfactual depends on one geography Audit that donor and run exclusion sensitivity.
High power failure rate SparseSC or DGP instability Inspect exceptions, rank, panel completeness, and placebo feasibility.
High grid-based MDE Noise or weak counterfactual relative to plausible lift Redesign duration, outcome, treated scale, or method before launch.
Interference signal Outcome changes among controls are compatible with contamination Investigate exposure using campaign and geographic evidence; the screen is not causal proof.
Result partial Missing uncertainty or warnings Resolve the stated fields before decision use.

Do not tune exclusions, dates, or donors by searching for significance. If the remedy changes the design, rerun power and record the new specification.

Evaluate Donors

Run donor evaluation on pre-treatment data after applying substantive eligibility exclusions.

geosc donors --config data-config/donor_eval_config.yaml --jobs -1

The evaluator aligns each treatment-donor pair by time, then calculates Pearson correlation and its p-value, root mean squared error (RMSE), percentage RMSE, mean absolute percentage error (MAPE), and normalised dynamic time-warping (DTW) distance. It normalises metrics within each treated geography, combines them using adaptive weights, and ranks complete rows.

Review:

  • overlap_count, treatment_coverage, and donor_coverage before scores;
  • metrics_complete and metric_failure_reasons;
  • correlation, rmse, mape, and dtw, not only composite_score;
  • warning_flags, quality bands, concentration, and effective donor count;
  • maps only as a geographic diagnostic, not as proof of comparability.

selected_weight is normalised from positive composite scores among the recommended donors. Its role is design_stage_recommendation_weight; it is not a fitted SparseSC counterfactual weight.

The evaluator’s output is a screening artefact. If it changes the donor pool, filter the canonical panel and rerun power and inference. To render maps, see Use Shapemaps.

DTW standardises each pair’s finite, timestamp-aligned overlap using its own mean and population standard deviation. FastDTW uses scalar absolute local cost, a fixed approximation search radius of one, and divides the resulting distance by the overlap count. Constant series become zero vectors. The radius controls the approximation neighbourhood around a recursively estimated path; it does not bound the displacement between aligned dates.

Version 0.4.0 replaces the history-dependent approximation radius. This can change DTW, composite rankings, recommendation weights and seasonality_score (which aliases normalised DTW). Regenerate recommendations and inspect the underlying metrics. Backend and radius are recorded in quality JSON; rankings are screening outputs and have no calibrated acceptance guarantee.

How to Use Shapemaps

GeoSC donor-stage maps use shapemap_file to read a GeoPandas-compatible vector file and shapemap_id_column to choose the geography ID used for joins. The values in that ID column must match the unit IDs used in your experiment data, such as DMA IDs, postcode districts, postcode areas, or another market key. The inference and power stages do not consume shapemap configuration.

Built wheel and sdist installs do not include the repository-local shapemap/ directory. Packaged users should provide map files from their own runtime environment.

Basic Config

shapemap_file: "/path/to/markets.geojson"
shapemap_id_column: "id"

GeoSC can read formats supported by GeoPandas, including GeoJSON, TopoJSON, Shapefile, and GeoPackage inputs when the local geospatial stack supports them.

Repository-local shapemap examples are available for common markets:

  • shapemap/uk/postcode_areas.geojson
  • shapemap/uk/postcode_districts.geojson
  • shapemap/us/dma_legacy_normalized.geojson
  • shapemap/fr/fr_nuts3.geojson
  • shapemap/de/de_nuts3.geojson
  • shapemap/uk/meta_commuting_zones.geojson
  • shapemap/us/meta_commuting_zones.geojson
  • shapemap/fr/fr_meta_commuting_zones.geojson
  • shapemap/de/de_meta_commuting_zones.geojson

These are source-checkout assets. They are not included in built wheel or sdist artifacts.

Meta Commuting Zones are also available as optional geographies for Meta-aligned study design. They are commuting/economic zones, not Meta ad-targeting boundaries.

UK Postcode District Example

For UK production workflows, postcode district is often the practical market geography:

location_col_name: "postcode_district"
treatment_locations: ["SW1A", "M1"]
shapemap_file: "/secure/maps/uk_postcode_districts.geojson"
shapemap_id_column: "id"

The input data must use the same IDs as the shapemap:

postcode_district,date,Y
SW1A,01/01/2025,125.3
SW1A,02/01/2025,128.7
M1,01/01/2025,98.2

Use scripts/prepare_uk_postcode_shapemap.py from a source checkout to normalize licensed or user-supplied postcode polygon data:

python scripts/prepare_uk_postcode_shapemap.py \
  --input /path/to/licensed-postcode-polygons.gpkg \
  --output shapemap/uk/postcode_districts.geojson \
  --level district \
  --postcode-column postcode \
  --metadata-output shapemap/uk/postcode_districts.metadata.json

The generated file exposes:

  • id: postcode district, such as SW1A, M1, or BT7
  • name: display name, defaulting to the same value as id
  • country: UK
  • geography_level: postcode_district
  • geometry: dissolved polygon geometry

The repository includes an approximate version built from ONSPD live postcode centroids. Its metadata file retains the derivation year in its name:

location_col_name: "postcode_district"
treatment_locations: ["SW1A", "M1"]
shapemap_file: "shapemap/uk/postcode_districts.geojson"
shapemap_id_column: "id"

Validate the generated file before using it in donor evaluation:

python scripts/validate_shapemap.py \
  --shapemap shapemap/uk/postcode_districts.geojson \
  --id-column id \
  --expected-ids SW1A,M1 \
  --json

UK Postcode Area Example

For broader UK markets, use postcode areas:

location_col_name: "postcode_area"
treatment_locations: ["SW", "M"]
shapemap_file: "/secure/maps/uk_postcode_areas.geojson"
shapemap_id_column: "id"

Prepare the map with:

python scripts/prepare_uk_postcode_shapemap.py \
  --input /path/to/licensed-postcode-polygons.gpkg \
  --output shapemap/uk/postcode_areas.geojson \
  --level area \
  --postcode-column postcode

Provenance and Licensing

Do not commit or distribute postcode geometry unless the source license permits that use. The repository records shapemap provenance policy in shapemap/README.md and shapemap/manifest.yaml.

For UK data:

  • OS Code-Point Open is an open postcode unit point dataset, not polygon geometry.
  • OS Code-Point with Polygons is the appropriate official polygon source when you have the necessary access and licensing.
  • ONS Postcode Directory is useful for postcode-to-geography lookup workflows.

Troubleshooting

If donor maps do not highlight treatment or donor units:

  • confirm the shapemap contains the configured shapemap_id_column
  • confirm the shapemap ID values exactly match the experiment unit IDs
  • normalize both sides to the same case and spacing convention
  • confirm the file can be opened by geopandas.read_file

scripts/validate_shapemap.py reports missing expected IDs, duplicate IDs, blank IDs, null or empty geometry, invalid geometry, row counts, unique ID counts, and the detected CRS.

Prepare an Input Panel

Use a long CSV with one row per geography and observed period. The inference loader requires configured date, geography, and outcome columns; treatment is defined in YAML, not read from a treatment flag in the CSV.

Required checks

  1. Use stable geography identifiers and one geographic level.
  2. Use a regular time interval appropriate to the campaign and outcome.
  3. Confirm that every intended unit-period has one finite outcome.
  4. Make the intervention date an observed period.
  5. Preserve enough observations before and after the resolved measurement start.
  6. Use a consistent outcome definition, currency, attribution window, and data production process across geographies and time.

Inference defaults to duplicate_policy: error and missing_outcome_policy: error. Alternative policies are explicit analytical decisions. mean or sum changes duplicate rows before pivoting; drop_unit, drop_period, or impute_with_report changes the estimable panel.

With impute_with_report, interpolation and endpoint filling use observations from the same pre-treatment, cooldown, or measurement window. A geography with no observed outcome in a window fails; supply data or choose an explicit drop policy. The method uses observed-period positions rather than elapsed time, so keep the observation cadence appropriate to the analysis.

Optional covariates are numeric pre-period baseline means. They use only retained outcome periods, average duplicate records within a period before averaging periods, and skip missing covariate observations. Every unit/feature needs an observed finite baseline. For direct-data runs, prepare this baseline table before passing covariates_df; GeoSC cannot infer its source dates.

data_path: data/client_panel.csv
date_col_name: date
location_col_name: region_id
outcome_col_name: revenue
date_format: "%Y-%m-%d"
treatment_unit_ids: ["FRJ"]
intervention_date_str: "2026-09-01"
duplicate_policy: error
missing_outcome_policy: error

Do not use locale-dependent date parsing in an audited run. After execution, review data_validation.json; a successful model does not supersede a failed data contract.

Produce a Stakeholder Summary

Lead with the decision and its conditions. A compact summary should include:

  • treated geography, outcome, launch, cooldown, and measurement dates;
  • eligible donor universe and material exclusions;
  • design-stage MDE at the stated target power and plausible lift range;
  • inference status, ATT in outcome units, interval, p-value, and placebo count;
  • pre-period fit and donor concentration;
  • concurrent media, spillover, data, and sensitivity caveats;
  • a clear recommendation: proceed, interpret cautiously, redesign, or reject.

Example:

GeoSC estimated average incremental revenue of £X per week against the pre-specified SparseSC counterfactual over dates A–B. The two-sided in-space placebo p-value was P using N placebo assignments; the interval was [L, U]. Interpretation is conditional on the recorded donor-eligibility and treatment- isolation assumptions. The design-stage grid-based MDE was M% at T% target power, so effects below M% were not reliably detectable under the fitted DGP.

Do not translate statistical significance into ROI without campaign cost and a commercial value definition. Do not describe status: ok as proof of causal validity.

Use the interval level recorded in ci.level, independently of the configured significance threshold. An 80% average-ATT interval must not be labelled 95%, and its bounds must remain in the ATT’s unscaled outcome units. Scaled diagnostic bounds cannot fill missing primary uncertainty. The uplift plot’s horizontal band concerns average ATT uncertainty, not coverage of the effect path.

For partial results, retain warnings and state which quantities remain available. For failed or unknown status, report errors and unavailable inference instead of interpreting retained diagnostic numbers. Read significance_available before the stored significant flag; an unavailable decision is not evidence of no effect. The maintained renderers apply these rules consistently and the HTML report escapes artifact-derived text.

Run Inference

Freeze the panel and inference YAML before viewing post-treatment results, then run:

geosc infer \
  --config data-config/geolift_analysis_config.yaml \
  --create-plots

Use --data to override only the CSV path and --no-create-plots to suppress plotting. The command returns non-zero when the stage raises an error or the result status is failed.

Review outputs in this order:

  1. data_validation.json for panel construction and window metadata;
  2. assumption_validation.json for advisory pre-trends/interference and any explicitly requested experimental pre-period placebo gate;
  3. geolift_results.json for status, unscaled ATT, p-value, interval, placebo count, granularity, warnings, and errors;
  4. geolift_diagnostics.json for pre-fit and fitted donor-weight diagnostics;
  5. uplift_timeseries.png as a visual diagnostic.

Summary intervals use the recorded ci.level and unscaled nested bounds. Disabled or unavailable primary intervals remain unavailable, even when scaled diagnostics exist. Partial plots display their limitations; failed or incomplete primary results cannot generate uplift plots. Regenerate older dictionaries missing status, estimand or nested interval metadata rather than assigning a default confidence level. See the result contract.

Describe the result as an estimated difference from the configured SparseSC counterfactual. Causal interpretation requires defensible donor exchangeability, treatment isolation, stable measurement, timing, and absence of material unmodelled shocks. The advisory interference screen cannot prove or disprove spillover.

Read the estimated effect

Illustrative observed and synthetic outcome paths and their period-by-period difference, with cooldown excluded and the measured average marked. Illustrative observed and synthetic outcome paths and their period-by-period difference, with cooldown excluded and the measured average marked.

This synthetic example illustrates the distinction between the outcome path, period effects and their measurement-period average. It is not an output from your campaign. Read the worked explanation before interpreting the equivalent diagnostics in your own run.

Run Power Analysis

Define a plausible effect grid and duration grid, then run:

geosc power --config data-config/power_analysis_config.yaml --jobs -1

Use a fixed random_seed for an audited run. Seeded runs use CPU simulation paths. Start with a small n_simulations only to debug the configuration, then increase it until Monte Carlo intervals are adequate for the decision.

For each row:

  1. require valid: true;
  2. inspect failure_rate and warnings;
  3. compare power with its Wilson interval;
  4. verify the DGP rank, explained variance, backends, placebo support, and injection metadata;
  5. interpret MDE as the smallest tested effect reaching target_power for a duration.

GeoSC does not interpolate between effect sizes. If no grid value reaches the target, report MDE as greater than the maximum tested effect. The CLI’s default planning threshold is 80%; it is configurable through target_power and is not a statistical property of SparseSC.

Power uses the SparseSC estimator and p-value detection rule inside each simulation, but it does not execute the full inference workflow. It does not run inference data policies, cooldown resolution, assumption checks, result processing, or plotting. Align the power input panel and intervention split with the intended inference design yourself.

Run Sensitivity Analysis

Pre-specify a small set of defensible perturbations:

  • alternative comparable pre-period windows;
  • exclusion of a substantively suspect or dominant donor;
  • plausible cooldown or measurement-start choices;
  • DGP rank and effect-pattern choices for power;
  • outcome definitions fixed before post-period inspection;
  • eligible geography aggregations justified by media delivery and spillover.

For each run, save its panel hash, config, artefacts, and rationale. Compare ATT, intervals, p-values, pre-fit, donor weights, placebo support, MDE, and failure rates. Distinguish robustness to a design choice from fishing over arbitrary specifications.

If the decision changes under small, equally defensible perturbations, report the result as specification-sensitive. Do not average incompatible estimands.

Run the Pipeline

Place the three canonical stage configs in one directory, then point the pipeline at any one of them:

geosc pipeline \
  --config config/geolift_analysis_config.yaml \
  --output-dir outputs/run_001

The required filenames are power_analysis_config.yaml, donor_eval_config.yaml, and geolift_analysis_config.yaml. The command runs power, donors, then inference and writes Markdown and HTML summaries by default.

Useful selectors:

geosc pipeline --config config/geolift_analysis_config.yaml --skip-power
geosc pipeline --config config/geolift_analysis_config.yaml --skip-donor
geosc pipeline --config config/geolift_analysis_config.yaml --only-inference
geosc pipeline --config config/geolift_analysis_config.yaml --no-report

--output-dir creates the legacy stage subdirectories below the supplied root. --jobs and --use-gpu are passed as runtime overrides, subject to each stage’s support.

The pipeline is an orchestrator, not a design-state engine. Donor recommendations do not filter power or inference inputs. The compact report is not a decision gate; inspect each underlying artefact and record any manual panel hand-off.

Use the Python API

For file-based inference, initialise the public analyser with a config and data path:

from geolift import GeoLiftAnalyzer

analyser = GeoLiftAnalyzer(
    config_path="config/inference.yaml",
    data_path="data/panel.csv",
)
result = analyser.run_analysis()

if result["status"] == "failed":
    raise RuntimeError(result["errors"])
if result["status"] == "partial":
    print("Review warnings:", result["warnings"])

Direct-data construction is also supported by GeoLiftAnalyzer; the panel must be wide with units as rows and datetime-like periods as columns, with a unit- indexed treatment-period series. Use the Python API reference for signatures.

Power is available through geolift.power.SparseSCPowerCalculator. Donor screening is available through geolift.donor_evaluator.DonorEvaluator, but neither is re-exported from the package root. Prefer the CLI for versioned file-based runs because it writes the maintained artefact family.

Validate a Power Simulation

Treat a power run as a model requiring diagnostics.

Reproducibility

Repeat the same CPU run with the same seed and compare the CSVs. Record the package version, input hash, YAML, seed, and execution backends.

Monte Carlo precision

Use power_ci_lower and power_ci_upper. If the interval straddles the target power, increase simulations before making a boundary decision. The interval quantifies simulation-count uncertainty only; it does not include uncertainty about the DGP specification.

DGP plausibility

Review dgp_rank, dgp_explained_var, and dgp_factor_dynamics. Compare observed and simulated distributions outside GeoSC where the decision is material: unit means and variances, cross-unit correlations, autocorrelation, seasonal patterns, extremes, and zeros. The current DGP does not model calendar effects or heteroskedastic idiosyncratic noise explicitly.

Grid stability

Power should usually rise with larger injected effects in a well-resolved simulation, but finite Monte Carlo estimates can move up or down. Investigate large reversals, high failure rates, coarse placebo p-values, or sensitivity to rank, history window, seed, donor eligibility, and effect pattern.

Do not call the run validated merely because all rows are valid. That flag requires zero failed simulations. Failed rows retain a conditional rejection rate and completion bounds, but ordinary power and its interval are unavailable. Do not select a design from those conditional rates or interpret an all-failed row as zero power. Remove the retired power_failure_rate_threshold key and regenerate older CSVs; decision consumers require current schema 2.0.