Skip to main content

Subsections of Explanation

Causal Estimand and Identification

GeoSC reports an unscaled average post-period treated-minus-counterfactual difference. Calling it causal requires more than estimator completion.

For treated set T\mathcal{T} and measured post periods P\mathcal{P}, the target can be written conceptually as

τ=1∣P∣∑t∈P(YˉTt(1)−YˉTt(0)).\tau = \frac{1}{|\mathcal{P}|} \sum_{t \in \mathcal{P}} \left(\bar{Y}_{\mathcal{T}t}(1)-\bar{Y}_{\mathcal{T}t}(0)\right).

The observed treated outcome supplies Yˉ(1)\bar{Y}(1). SparseSC estimates the unobserved Yˉ(0)\bar{Y}(0) from eligible donor outcomes.

A causal interpretation needs a stable outcome definition; treatment timing and assignment measured correctly; donors unaffected by treatment; no uncontrolled geography-specific shock aligned with launch; adequate pre-period support for the counterfactual; and an estimand whose geography and period match the business question. These conditions are design arguments, not outputs of the optimiser.

GeoSC’s raw pre-trend comparison is advisory because it compares unweighted group-mean slopes. Synthetic control instead fits a donor combination: treated slopes near two can be matched by mixing donors with slopes zero and four, even when most donors have slopes near zero and the raw mean comparison rejects. This counterexample establishes a mismatch between the raw test and synthetic matchability; it does not establish that the constructed design is causally valid.

Prespecify held-out synthetic-fit checks for model adequacy and keep the causal identification argument separate. The distinction between a weighted comparison, pre-intervention validation and contextual requirements is discussed in Abadie’s methodological review. Neither raw-trend non-rejection nor good synthetic fit proves exchangeability or rules out launch-aligned confounding.

Only GeoSC’s experimental pre-period placebo diagnostic can be explicitly required as an operational gate. Its procedure is unchanged and is not calibrated here; passing it does not establish identification. The interference screen is advisory and has no exposure model. Power addresses detection under a simulated DGP, not identification. State each evidence source separately.

Cooldown, Seasonality, and Anticipation

Cooldown excludes observed periods between launch and measurement. It is useful when response builds gradually, attribution matures, or short-term launch noise is outside the estimand. It also removes data and can reduce power.

GeoSC counts observed periods. With daily data, cooldown_periods: 7 excludes seven observed days; missing dates make that different from seven calendar days. The resolved analysis window is persisted in validation and result artefacts.

Cooldown does not fix anticipation. If customers or media respond before the declared launch, the pre-period is contaminated. Move the intervention boundary only when the behavioural timing supports it, not to improve fit.

Power’s native CLI does not apply the inference cooldown resolver. Set the power split and simulated duration to represent the actual measurement design, and document the translation.

Historical data should cover comparable seasonal conditions. The factor VAR(1) DGP has no explicit day-of-week, holiday, or campaign-season terms. A long but non-comparable history can be worse than a shorter seasonally relevant window; use pre-specified sensitivity checks.

Design Rejection

GeoSC should reject or defer a design when the requested causal statement is not supported. At minimum, stop when:

  • treatment, outcome, geography, or measurement window is unresolved;
  • treatment exposure cannot be separated from donors;
  • the panel cannot satisfy the explicit data contract;
  • no substantively eligible donor pool remains;
  • pre-period fit is materially inadequate or dependent on an implausible donor;
  • placebo support cannot resolve the pre-specified alpha;
  • power is inadequate for the commercially plausible effect range;
  • simulation failures or DGP sensitivity make MDE unstable;
  • required assumption checks fail or inference status is failed;
  • conclusions reverse under small, equally defensible specifications.

Some failures can be repaired by changing geography, outcome, history, duration, or eligibility. That creates a new design and requires a new power assessment. If no repair is defensible before treatment, document the experiment as descriptive or infeasible. A forced estimate is not a more helpful answer.

Donor Quality

Donor quality has two distinct meanings in GeoSC.

The donor stage measures pairwise pre-period similarity. Correlation captures co-movement; RMSE and MAPE capture level error; normalised DTW captures temporal shape. Composite scores are relative within each treated geography and support screening. They do not estimate final counterfactual weights.

Inference measures joint synthetic-control fit. Several individually modest donors can combine well, while a highly correlated donor can receive little weight. Review fitted pre-period error and the SparseSC weight distribution.

Strong screening scores do not establish donor exchangeability. Eligibility also requires no treatment exposure, comparable outcome measurement, and no post-launch shock that differentially moves the donor. Conversely, poor overlap, incomplete metrics, concentrated recommendation weights, or weak joint pre-fit are direct reasons to reconsider the design.

Donor selection must be outcome-blind with respect to the measured post period. Changing the pool after seeing lift invalidates ordinary interpretation unless the search and multiplicity are explicitly modelled.

Frequently Asked Questions

Is a donor the same as a control?

A donor is an eligible control geography that can contribute to a synthetic counterfactual. In code, power calls all non-treatment units controls. Preserve the eligibility distinction in design documentation.

Do donor recommendations flow into inference?

No. The pipeline runs stages independently. Filter and version the panel if the donor screen changes the eligible universe, then rerun affected stages.

Is GeoSC power simulation-based?

Yes. It estimates a factor-VAR DGP from the pre-period, simulates post-periods, injects effects, calls SparseSC, and counts p-values below alpha. It is more computationally intensive than a closed-form MDE but not automatically more accurate.

Does power use the complete inference workflow?

No. It uses the SparseSC estimator and detection rule inside simulation but has a narrower configuration and does not run inference data policies, cooldown, assumption gates, or result processing.

Why can estimated power decrease at a larger effect or duration?

Finite Monte Carlo estimates can fluctuate. Large reversals can also indicate failures, coarse placebo support, a weak DGP, or specification sensitivity. Increase simulations and investigate; do not label an empirical rolling-window diagnostic a theoretical monotonic power curve.

What does MDE mean here?

For a duration, it is the smallest valid effect on the tested grid reaching configured target_power. It is not expected campaign lift and is not interpolated.

How is a relative effect injected?

GeoSC multiplies one pooled mean across treated units and all pre-periods by the relative effect, then adds the same absolute path to each treated unit.

What does valid mean in power output?

At least one simulation succeeded and failure rate did not exceed the threshold. It says nothing about DGP plausibility or causal identification.

What is the top-level inference p-value?

It is a two-sided, add-one-corrected in-space placebo p-value for the absolute average post-period effect. It is not a post/pre MSPE-ratio p-value.

Can a small donor pool produce significance?

Only at the resolution allowed by its effective placebo count. The smallest step is 1 / (n_placebos + 1) under the implemented calculation.

Does status: ok prove the campaign caused lift?

No. It means the software result contract completed. Identification depends on eligibility, isolation, measurement, timing, pre-fit, and sensitivity evidence.

Does GeoSC calculate ROI or iROAS?

No. Combine a defensible incremental outcome with campaign cost and the correct commercial value or margin definition downstream.

Does GeoSC support GPU inference?

No. Optional CuPy work is limited to supported unseeded power DGP paths. Seeded power and main SparseSC inference are CPU-based.

Method Comparisons

Compare estimands and decision rules before comparing numbers.

Method family Counterfactual Typical uncertainty Main diagnostic focus
GeoSC SparseSC-weighted individual donors In-space placebo effects and intervals Joint pre-fit, donor weights, placebo support, simulation diagnostics
Google TBR/GBR and matched-market workflows Time-based regression on aggregated treatment and control signals Model-based intervals; design power depends on the implementation Stable treatment-control relationship, match quality, model form, assignment constraints
Meridian GeoX Randomised or stratified assignment with TBR inference at the documented launch boundary Product-specific Randomisation, balance, TBR stability, and delivery integrity
Meta GeoLift Augmented synthetic control with generalised synthetic-control components Conformal and package-specific procedures L2 imbalance, augmentation model, market selection, test fit
Bayesian causal-impact tools Structural time-series counterfactual Posterior distribution Prior/model fit and posterior predictive behaviour

GeoSC’s native power module is simulation-based, but that does not imply a TBR or GeoX MDE is necessarily analytic; implementations vary. GeoSC repeatedly simulates a factor-VAR post period, injects effects, and calls SparseSC. A TBR workflow may use a fitted treatment-control regression, residual variance, analytic formula, resampling, or simulation.

Before comparing MDEs align outcome and transformation, treatment geography, control eligibility, pre-period, measurement and cooldown windows, lift denominator, effect pattern, alpha, target power, one- or two-sided rule, failure handling, and whether placebo assignments are sampled. Before comparing effects align the estimand, scale, aggregation, and uncertainty interpretation.

A material discrepancy is a diagnostic. Decompose it rather than averaging the answers or assuming the larger MDE is conservative. GeoSC is useful when individual donor diagnostics, Python operation, and auditable artefacts matter; it is not automatically superior to a well-specified regression design.

Primary method references

These references describe their respective implementations. They do not imply that an internal or modified workflow uses the same defaults.

Multi-Treatment Designs

GeoSC supports several treated geographies, but interpretation and computation change.

The top-level effect is joint across the treated set; it is not a separate ATT for each geography. Placebo assignments must contain the same number of control units as the treated set, giving (N0N1)\binom{N_0}{N_1} possible assignments. Each physical control contributes one residual path, so two treated units and ten controls give 45 assignments. Controls with identical residual values remain separate geographies. An assignment cannot contain the same geography twice, although separate sampled assignments can repeat. SparseSC work can grow rapidly, so sparse_sc_max_n_pl may trigger sampling.

There must be at least as many controls as treated units. This subset feasibility rule does not establish sufficient p-value resolution or satisfy the separate cross-fitting requirements; a larger control pool may be needed.

Power injects the same absolute effect path into every treated unit, based on the pooled treated pre-period mean. This is not unit-specific proportional lift. When treated units have very different outcome levels, supplement the standard run with a carefully labelled sensitivity analysis or avoid describing the injected effect_size as each unit’s own percentage lift.

All treated units must share one launch date equal to intervention_date. GeoSC checks supplied dates before excluding cooldown periods and retiming the estimator to the common measurement start. Staggered dates and a shared date that differs from intervention_date are rejected. The private SparseSC estimator also rejects staggered and dose-normalised requests; they require different assignment rules.

Regenerate multi-treatment p-values, intervals, and power outputs produced before the distinct-geography correction, including runs with a small placebo cap. Earlier construction replicated the control block once per treated unit; a low cap could hide the count mismatch without repairing the reference set. The correction preserves fitted weights and point effects. It does not qualify the placebo exchangeability assumptions or establish simulation calibration.

Placebo Inference

GeoSC uses SparseSC in-space placebo inference because geo experiments often have too few treated units for conventional large-sample approximations.

For each treated-sized subset of eligible controls, SparseSC constructs an honest placebo effect. The top-level p-value compares the absolute average treated effect with the absolute average placebo effects and applies an add-one correction. It is two-sided.

Read the empirical reference set

Histogram of honest placebo mean effects, with the observed mean marked and both absolute-effect tails shaded; annotations give the add-one p-value and finite resolution. Histogram of honest placebo mean effects, with the observed mean marked and both absolute-effect tails shaded; annotations give the add-one p-value and finite resolution.

This uses the same synthetic example as the counterfactual chart. With one treated geography and 49 controls, all 49 single-control placebo assignments are enumerated. The bars show honest placebo mean effects from the native result; marks below the axis locate each value. Orange shading identifies both tails where the absolute placebo mean is at least as large as the absolute observed mean. The printed numerator includes one for the add-one correction. Histogram bins are only a display choice; counts and the p-value are computed from individual placebo values, including ties.

The finest p-value step is 1/50, or 0.02, in this example. No Gaussian distribution is fitted. The figure explains the implemented comparison; it does not establish exchangeability or statistical calibration. The numerical source records the observed mean, every placebo mean and the counted exceedances.

Each eligible physical control appears once in the reference matrix. A subset contains distinct geographies, even when some residual paths have equal values. All treated units share one launch and measurement window.

The unchanged backend uses ten honest cross-fitting folds, so GeoSC requires at least ten effective controls before SparseSC fitting. At the ten-control boundary, each fold holds out one control and retains nine for fitting in the retrospective fast-path fixture. This is an operational feasibility check, not a qualification of error rates or donor exchangeability. Adding periods or treated geographies does not increase the control count. Match-space CV settings have separate training-size constraints; lowering sparse_sc_cv_folds does not change honest cross-fitting.

When feasible assignments exceed sparse_sc_max_n_pl, SparseSC samples assignments using sparse_sc_placebo_seed; otherwise it enumerates them. The result reports the possible and effective counts and labels the mode exact, sampled, or unknown. Here exact means exhaustive enumeration of the supported physical-geography assignment set; it does not assert exact causal calibration. Sampled draws can repeat an assignment across draws, so the effective count is the number of draws, not necessarily the number of unique assignments.

Finite support matters. With M∗M^* effective placebos the nominal resolution is 1/(M∗+1)1/(M^*+1). If this exceeds alpha, significance at that alpha cannot be attained. Sampling adds reference-set variability even when the optimiser is stable; preserve the seed.

The p-value is calibrated only relative to the placebo assignment logic and its exchangeability assumptions. Non-comparable donors, spillover, differential measurement changes, or poor pre-fit can make a numerically small p-value misleading. SparseSC placebo intervals are likewise finite-reference-set quantities, not model-free guarantees.

Power and MDE Methodology

GeoSC power is a simulation-based design diagnostic. For each effect and duration, let N be attempted simulations, S successful estimates, F=N-S failed estimates, and R successful estimates with p < alpha. Ordinary power is R/N and is available only when F=0. If any draw fails, primary power and its Wilson interval are unavailable, and the row cannot support ordinary MDE recommendations.

The successful-draw rate remains explicitly named:

P^conditional=R/S(S>0).\widehat{P}_{\mathrm{conditional}} = R/S \quad (S>0).

Failed draws have unknown rejection outcomes. The observed all-attempt rate therefore has completion bounds:

[R/N, (R+F)/N].[R/N,\ (R+F)/N].

For eight rejecting successes and two failures, conditional rejection is 100% and the completion bounds are 80% to 100%; ordinary power is unavailable. These are missing-outcome completion bounds for the simulated sample, not confidence intervals or bounds on true population power. When every draw fails, conditional rejection is unavailable and the completion bounds are [0,1]. When none fails, the bounds collapse to the observed rate, while Monte Carlo uncertainty remains.

A row is valid only when its complete positive integer attempt count contains zero failures. This does not establish DGP credibility, absence of future failures, identification, or statistical calibration. There is no permissive failure-rate override; power_failure_rate_threshold is retired.

Implemented workflow

For every effect-duration grid point, GeoSC:

  1. pivots the long CSV to a unit-by-period matrix and sorts parsed dates with their outcomes at the shared calculator boundary;
  2. uses periods before treatment_date as the fixed observed pre-period;
  3. demeans each unit, fits an SVD factor model, and estimates factor dynamics;
  4. simulates a post-period panel;
  5. computes one baseline as the mean across all treated units and all pre-periods;
  6. multiplies that pooled baseline by effect_size and applies the resulting same absolute effect path to every treated unit;
  7. appends simulated post data to the observed pre-period;
  8. calls SparseSC with treated periods at the split and counts results.p_value < alpha.

The direct calculator excludes rows outside its declared treatment and control sets before DGP fitting, retaining eligible rows in source order. File-driven power defines those sets as the configured treated geographies and all other CSV geographies. Unit identifiers must be unique and non-missing, and the two sets must be disjoint and present in the panel.

This denominator means a configured 10% lift is relative to the pooled treated pre-period mean. It is not a separate 10% of each treated unit’s own baseline. The output records the baseline and full absolute injection.

The power loop uses the SparseSC estimator and its top-level placebo p-value, but it is not identical to geosc infer. It does not run inference’s panel policies, cooldown resolver, assumption checks, CI configuration, result processor, or full SparseSC config surface. “Same inference path” is therefore too strong; align panels and settings explicitly.

Power supports retrospective fast RidgeCV detection and shares inference’s method/CV option builder for that path. It rejects other method requests and simulation-owned window, CI or placebo-seed keys rather than discarding them. Accepted CV controls are consumed by the native learner and recorded as JSON estimator_settings in each row. Both stages retain their independent YAML and placebo caps, so align the intended procedure explicitly.

Ignored lasso controls are retired. Match-space scoring uses a documented continuous multi-output regression subset; recognising a sklearn scorer name alone is insufficient because an incompatible scorer can yield all-NaN CV scores without raising an exception. Native checks verify the supported criteria on a manufactured panel; they do not establish campaign calibration.

Each successful simulation must return a finite real numeric p-value in [0, 1]; booleans, strings, arrays, missing values and out-of-range values count as failures. The original scalar is used for range checks and the strict p_value < alpha comparison, avoiding rounding across a boundary. Invalid outputs appear in n_invalid_p_values; other failures appear in n_estimation_failures, and both contribute to n_failed.

The diagnostic conditional_power divides detections by successful estimates. It must not be substituted for ordinary power or used to select an MDE. A failed row remains unavailable even if the lower completion bound exceeds target power; no failure-handling estimand is silently introduced. The corrections do not establish empirical size, coverage or power calibration.

Effect patterns

  • constant and immediate: full absolute effect in every simulated period;
  • ramp: linear sequence from zero to full effect, inclusive;
  • decay: full effect followed by exp⁡(−0.1t)\exp(-0.1t) decay;
  • any other value: configuration error.

The 0.1 decay rate is not configurable in version 0.4.0.

Grid-based MDE

For duration dd and target power qq, GeoSC reports the smallest valid tested effect that reaches the target:

MDEd=min⁡{e∈E:valid(e,d), Power^(e,d)≥q}.MDE_d = \min\{e \in \mathcal E: valid(e,d),\ \widehat{Power}(e,d) \geq q\}.

target_power defaults to 0.80 in the CLI and pipeline but is configurable. No interpolation is performed. If no effect reaches the target, MDE is above the tested range.

Monte Carlo uncertainty

The CSV includes a Wilson interval around the estimated detection proportion. Its confidence setting uses the configured alpha. This interval describes finite simulation uncertainty, not DGP misspecification, donor-selection uncertainty, or future-campaign uncertainty. If it straddles target power, increase n_simulations before making a threshold decision.

Worked output

duration effect_size power interval failure_rate valid
28 0.05 0.42 0.36–0.48 0.00 true
28 0.10 0.76 0.71–0.81 0.00 true
28 0.15 0.91 0.87–0.94 0.00 true
35 0.10 0.84 0.79–0.88 0.00 true
42 0.10 unavailable unavailable 0.25 false

At 80% target power, the 28-period grid-based MDE is 15% and the 35-period MDE is 10%. The 42-period row is withheld because some draws failed.

Power is computationally intensive because total work scales approximately with effect sizes × durations × simulations × SparseSC fit and placebo cost. This differs from a closed-form regression MDE, but computational intensity is not by itself evidence of better calibration.

Power DGP Assumptions

GeoSC estimates its power data-generating process (DGP) from the pre-period outcome matrix.

Each unit is demeaned. Singular value decomposition produces factor loadings and factor scores. When rank is automatic, GeoSC chooses the first rank reaching about 80% cumulative singular-value variance, bounded by power_dgp_max_rank and a short-panel VAR feasibility cap. It then estimates a VAR(1) intercept, transition matrix, and Gaussian innovation covariance on the factor series.

Simulation draws the initial factor state from the estimated marginal normal distribution; it is not conditioned on the final observed factor state. Later states follow the fitted VAR(1). Outcomes combine fixed unit loadings, simulated factors, stored unit means, and independent Gaussian idiosyncratic noise with one pooled residual standard deviation.

Consequences:

  • cross-unit dependence is represented through common factors;
  • lag-one factor dependence is represented;
  • calendar seasonality is not modelled explicitly;
  • the fitted VAR transition is not constrained to be dynamically stable;
  • idiosyncratic variance is homoskedastic across units and periods;
  • non-Gaussian tails, structural breaks, intermittent zeros, and conditional start-state dynamics may be poorly represented;
  • rank selection is a heuristic, not cross-validated DGP truth.

Validate simulated moments and time-series behaviour against the use case. Run sensitivity to history, rank, effect pattern, and eligible controls. If design decisions change materially, report DGP dependence rather than one definitive MDE.

SparseSC Estimator

GeoSC’s only implemented estimator selector is sparsesc. SparseSC learns a regularised match space and donor weights from pre-treatment outcomes and optional covariates.

Conceptually, it chooses unit weights WW and match-space weights VV to reduce treated-versus-synthetic pre-period discrepancy subject to regularisation:

(W^,V^)=arg⁡min⁡W,V∥YTpre−YCpreW∥V2+PW(W)+PV(V).(\widehat W, \widehat V) = \arg\min_{W,V} \left\|Y^{pre}_{\mathcal T}-Y^{pre}_{\mathcal C}W\right\|_V^2 + \mathcal P_W(W) + \mathcal P_V(V).

This is an explanatory simplification, not a line-by-line restatement of every vendored optimisation path. With sparse_sc_fast_estimation: true, GeoSC calls the vendored fast path, which uses RidgeCV-backed match-space machinery. The full path uses SparseSC’s penalty search and coordinate-descent machinery.

Regularisation controls overfit; it does not make an ineligible donor valid. Inspect pre-fit, fitted unit weights, weight concentration, and sensitivity to substantive donor exclusions. The selected_weight from GeoSC’s donor stage is not W^\widehat W; fitted weights are exposed in inference diagnostics.

The package vendors a pinned, locally governed SparseSC fork. See Vendored Dependencies for provenance.

Spillover, Contamination, and Concurrent Media

Synthetic control requires the donors to represent untreated outcomes. This can fail when audience delivery crosses boundaries, national media reaches both groups, travellers move between geographies, retail or supply changes differ by region, or another campaign starts near treatment.

Concurrent activity is not automatically fatal. A national or business-as-usual channel can be part of the baseline if its effect and delivery are sufficiently common across treated and donor geographies. Differential delivery correlated with treatment is the problem. Record channel, dates, spend or delivery where available, targeting logic, and expected geographic reach.

GeoSC’s interference screen compares outcome behaviour and can return signal, no_signal, or indeterminate. It has no exposure model and cannot establish whether media caused the pattern. Use campaign logs, geographic delivery data, and domain evidence. Keep the screen advisory.

If contamination is material and cannot be represented or excluded, reject the design or change the estimand. Do not reinterpret contaminated donors as valid because pre-period fit was good.

Synthetic-Control Formalism

Let YitY_{it} be the outcome for unit ii and period tt. Let T\mathcal{T} denote treated units, C\mathcal{C} eligible donors, and T0T_0 the last pre-treatment period after any window resolution.

For one treated unit, a synthetic counterfactual has the form

Y^1t(0)=∑j∈CwjYjt.\widehat{Y}_{1t}(0) = \sum_{j \in \mathcal{C}} w_j Y_{jt}.

The maintained retrospective fast estimator uses pre-treatment features and post-period control outcomes to learn the match space. It does not fit to the treated post-period outcomes. The period effect is

τ^t=Y1t−Y^1t(0),\widehat{\tau}_t = Y_{1t} - \widehat{Y}_{1t}(0),

and GeoSC’s top-level unscaled ATT is the average of that effect over measured post periods:

τ^=1T1∑t=T0+1T0+T1τ^t.\widehat{\tau} = \frac{1}{T_1}\sum_{t=T_0+1}^{T_0+T_1}\widehat{\tau}_t.

Read the fitted counterfactual

Observed and fitted synthetic outcomes above their difference over time, with launch, excluded cooldown and measured average effect marked. Observed and fitted synthetic outcomes above their difference over time, with launch, excluded cooldown and measured average effect marked.

This illustrative synthetic example has one treated geography, 49 eligible controls, 40 pre-period days, three cooldown days and 17 measured days. The upper panel compares observed outcomes with the fitted synthetic trajectory. The lower panel shows their difference; its dashed line averages only measured days. The three cooldown days are excluded from fitting and the reported effect, so the synthetic line has a gap there. The filled area between the upper lines shows an outcome difference, not a confidence interval.

The generator adds four outcome units per day after launch, but the chart shows the estimated effect rather than substituting that known injection. Its counterfactual and placebo figures come from the same native GeoSC analysis. See the numerical source and reproduction instructions. This is an explanation of the calculation, not a calibration study or campaign result.

The implementation also supports multiple treated units. SparseSC constructs a joint treated effect and an in-space placebo reference set formed from treated- sized subsets of controls. With N0N_0 controls and N1N_1 treated units, the exhaustive assignment count is

M=(N0N1).M = \binom{N_0}{N_1}.

GeoSC’s top-level p-value is not a post/pre mean-squared prediction-error ratio. If τ^(m)\widehat{\tau}^{(m)} is the average placebo effect for assignment mm, SparseSC uses the two-sided absolute-effect comparison with add-one correction:

p=1+∑m=1M∗1{∣τ^(m)∣≥∣τ^∣}M∗+1,p = \frac{1 + \sum_{m=1}^{M^*} \mathbf{1}\left\{|\widehat{\tau}^{(m)}| \geq |\widehat{\tau}|\right\}} {M^*+1},

where M∗M^* is the effective exhaustive or sampled placebo count. The finest attainable step under this calculation is 1/(M∗+1)1/(M^*+1). The result is an empirical comparison against the eligible placebo assignments; its causal meaning depends on the design.