Frequently Asked Questions

Quick answers to the most common GeoSC questions.

Business Questions

Q: What problem does GeoSC solve?

A: GeoSC estimates the incremental impact of regional marketing campaigns under a synthetic-control design. It answers the practical question: “How much did observed outcomes differ from a SparseSC counterfactual for the treated geographies?” The estimate is only credible when the donor pool, treatment timing, measurement source, and spillover assumptions are defensible.

Q: Why is this important for my business?

A:

  • Estimate incremental lift: Quantify observed outcomes against a documented counterfactual.
  • Improve budget decisions: Compare campaigns using a consistent measurement design.
  • Avoid unsupported claims: Separate evidence of lift from descriptive trends where the design is weak.
  • Support governance: Preserve JSON/CSV artifacts for review, audit, and sensitivity analysis.

Q: Is this the only measurement tool I need?

A: No, GeoSC is one component of a comprehensive measurement strategy:

  • Best for: Regional campaigns, store rollouts, geo-targeted advertising
  • Complements: Media Mix Modelling (MMM), multi-touch attribution, A/B testing
  • Integrates with: Your existing analytics stack (Google Analytics, Adobe, etc.)
  • Use alongside: Brand studies, customer surveys, and other measurement approaches

Q: What’s the cost and expected ROI?

A: GeoSC does not estimate commercial ROI by itself. Cost and ROI depend on your license terms, campaign spend, margin assumptions, operational constraints, and whether the experiment produces a credible effect estimate. Use GeoSC’s statistical outputs as inputs to a separate business-case calculation.

Q: How does the process work?

A: GeoSC supports a three-stage workflow:

  1. Evaluate donors: Identify control geographies that are plausible comparisons for the treated geographies.
  2. Check test strength: Use simulation-based power analysis to assess detectable effects and failure rates under explicit DGP assumptions.
  3. Measure lift: Run SparseSC inference and review result, data-validation, diagnostic, and assumption-validation artifacts.

Elapsed time depends on data readiness, review requirements, donor-pool quality, and campaign complexity.

Getting Started

Q1: What data do I need to run a GeoSC analysis?

A: You need:

  • Time series data: At least the configured minimum number of pre-treatment observations (12 in the shipped inference example). More history is usually preferable when it is comparable to the campaign period.
  • Geographic units: Markets, DMAs, states, or regions
  • Outcome metric: Sales, conversions, or other KPIs
  • Treatment assignment: Which markets received the campaign

Data format: CSV with columns for date, geographic unit, outcome metric, and treatment information. The exact required columns and date format come from your YAML configuration.

Q2: How long should I run my campaign to get reliable results?

A: Use the power-analysis command to assess duration and detectable effects:

geosc power --config data-config/power_analysis_config.yaml --use-gpu --jobs -1

This source-checkout example uses the shipped demo configuration. With a packaged installation, supply your own configuration path instead. Review power_analysis_results.csv for simulated power by duration and effect size, including valid, failure_rate, and Monte Carlo uncertainty fields.

There is no universal duration that guarantees power. Longer post periods can help, but power also depends on outcome noise, effect size, donor comparability, placebo support, and campaign timing.

Q3: How many control markets do I need?

A: There is no fixed safe number. You need enough comparable, uncontaminated control geographies to support a useful synthetic counterfactual and placebo reference set. Use the donor evaluator to inspect candidate quality:

geosc donors --config data-config/donor_eval_config.yaml --jobs -1

Outputs are saved in outputs/multicell_donor_eval/ (CSV with scores and donor maps).

Analysis Issues

Q4: My pre-period fit looks poor. What should I do?

A: Investigate the design before changing the estimator:

  1. Check data quality: Confirm consistent measurement methodology, treatment timing, and geography identifiers.
  2. Inspect unusual periods: Document holidays, promotions, outages, or other shocks. Do not remove observations simply to improve fit; pre-specify exclusions where possible and report sensitivity analyses.
  3. Extend comparable history: Add more pre-treatment observations only if measurement definitions and market conditions remain comparable.
  4. Revisit the design: Consider an alternative, defensible geographic aggregation or donor pool if the current one cannot support a credible counterfactual.

Q5: My results show no significant effect. What went wrong?

A: Common causes:

  • Low statistical power: Campaign too short or effect too small
  • Poor control selection: Controls don’t match treatment markets well
  • External factors: Market disruptions during campaign period
  • Data issues: Measurement problems or missing data

Check power analysis first to ensure the study design was capable of detecting the expected effect size.

Q6: The effect size seems too large to be believable. Is this normal?

A: Large effects can be real, but verify:

  1. Check data definitions: Ensure consistent measurement
  2. Review external events: Major market changes during campaign
  3. Validate treatment assignment: Confirm which markets got treatment
  4. Run sensitivity analysis: Test with different time periods or alternative donor pools.

Interpretation

Q7: How do I interpret the confidence intervals?

A: Confidence intervals show the range of effect sizes implied by the SparseSC placebo interval when the estimator returns finite bounds:

  • Narrow intervals: More precise estimates
  • Wide intervals: More uncertainty in the estimate
  • Intervals excluding zero: Statistically significant effects

Example interpretation:

Relative Lift: +15.2% (95% CI: +7.8% to +22.6%)

Read this as: “SparseSC returned a finite 95% placebo interval from +7.8% to +22.6% for this configured analysis.” It is not proof that the campaign-level causal assumptions hold.

Q8: What’s the difference between absolute and relative lift?

A:

  • Absolute lift: Raw units of incremental impact (e.g., +1,000 sales)
  • Relative lift: Percentage increase over baseline (e.g., +15%)

Both are important:

  • Use absolute lift for revenue calculations
  • Use relative lift for comparing across campaigns

Q9: How do I calculate ROI from the results?

A: GeoSC reports statistical effect estimates (e.g. ATT, confidence intervals). ROI calculations depend on your business context (price, margin, cost). Combine effect estimates with your cost and unit economics to derive ROI externally.

Q10: When should I trust the results vs. be sceptical?

A: Treat results as more credible when:

  • status is ok, data validation is clean, and required assumption checks have passed.
  • Pre-period fit and donor comparability are good, with an adequate placebo reference set.
  • The effect remains interpretable across pre-specified sensitivity analyses.
  • Treatment isolation, outcome definitions, and the measurement window are defensible for the estimand.

Be sceptical when:

  • Poor pre-period fit or large residuals
  • status: partial or failed, missing uncertainty metadata, or failed required assumption checks
  • Very large or very small effect sizes
  • A coarse or unstable placebo reference set, including very few eligible donors
  • Results change dramatically with small specification changes

A small placebo p-value is evidence within the configured reference set, not a standalone proof of a campaign-level causal effect.

Need More Help?