Frontiers#

Bias/Variance, Imbalance/Variance and External-Validity/Variance Tradeoffs.

class pyrake.calibration.frontier.EfficientFrontierResults(points, corners, primary_objective, n_attempted, n_skipped, tangent_slopes=None)[source]#

Bases: FrontierResults

Pareto frontier results with PyRake-specific convenience properties.

Subclasses FrontierResults; inherits knee().

The .weights, .distances, and .variances properties all return lists sorted by variance ascending (low variance first), so that weights[i], distances[i], and variances[i] correspond to the same frontier point.

property weights#

Return the weights at each frontier point, sorted by variance ascending.

Returns:

weights – One weight vector per point.

Return type:

list[npt.NDArray[np.float64]]

property distances#

Return the distance D(w, v) at each point, sorted by variance ascending.

Returns:

distances – One distance per point.

Return type:

list[float]

property variances#

Return the variance (1/M) ‖w‖₂² at each point, sorted ascending.

Returns:

variances – One variance per point.

Return type:

list[float]

plot(annotate_knee=True, ax=None, x_label='Distance from Baseline Weights', y_label='Variance Inflation Factor', title='Bias-Variance Tradeoff', annotations=None, marker='*', marker_size=10, marker_color='orange', annotation_size=None)[source]#

Plot the bias/variance tradeoff frontier.

Parameters:
  • annotate_knee (bool, default=True) – If True, mark the knee point and draw a tangent line.

  • ax (matplotlib Axes, optional) – If provided, plot onto this Axes; otherwise create a new figure.

  • x_label (str, default="Distance from Baseline Weights")

  • y_label (str, default="Variance Inflation Factor")

  • title (str, default="Bias-Variance Tradeoff")

  • annotations (dict, optional) – Labelled reference points to mark, as {label: point}. A point is a FrontierPoint (e.g. results.corners[0], the baseline weights, or results.corners[1], the minimum-variance weights) or an objective vector [distance, variance], such as frontier.evaluate_objectives(w) for weights solved some other way.

  • marker (str, default="*") – Marker style for the annotations.

  • marker_size (float, default=10) – Marker size for the annotations.

  • marker_color (str, default="orange") – Marker color for the annotations.

  • annotation_size (float | str | None, optional) – Label font size for the annotations, or None for large.

Returns:

ax – The axes with the frontier drawn.

Return type:

matplotlib Axes

Raises:

ValueError – If an annotation is not a FrontierPoint or a finite 2-entry objective vector.

class pyrake.calibration.frontier.EfficientFrontier(rake, engine='pd')[source]#

Bases: MultiObjectiveOptimizer

Trace the bias/variance Pareto frontier.

Solves:

minimize    D(w, v)
subject to  (1/M) Xᵀw = μ,
            (1/M) ‖w‖₂² ≤ φ,
            w ≥ 0,

for a range of φ values spanning from the minimum achievable variance to the variance at the minimum-distance solution. Increasing φ allows more variance but potentially less bias.

Parameters:
  • rake (Rake) – Configured calibration solver.

  • engine ({"ipm", "pd"} or None, optional) – Inner solver engine. Defaults to "pd" because PD warm-starts naturally across phi sweep iterations (~2.3x speedup over IPM cold-restart on the bench shape; see 2026-05-21 re-benchmark). IPM cannot safely warm-start barrier methods across active-set changes. Pass "ipm" to opt out; pass None to inherit whatever engine rake was constructed with. Setting this argument mutates rake.engine.

solve_with_bounds(bounds, x0=None)[source]#

Minimize D(w, v) subject to variance <= bounds[0].

Parameters:
  • bounds (npt.NDArray[np.float64]) – bounds[0] is the upper bound on variance (1/M) ‖w‖₂².

  • x0 (npt.NDArray[np.float64] | None, optional) – Warm-start point. Forwarded only when self.rake.engine == 'pd' (PD’s infeasible-start primal-dual method tolerates x0 sitting on the active boundary of the previous phi). Dropped when engine == 'ipm' because the barrier method requires strictly-interior x0 and a warm point from the previous sweep iteration typically lands on the boundary, triggering ProblemInfeasibleError from Phase I.

Returns:

res – The distance-minimizing weights within the variance bound.

Return type:

SolveResult

minimize_objective(objective_index, x0=None)[source]#

Minimize a single objective with no bounds on the other.

Parameters:
  • objective_index (int) – 0 to minimize D(w, v); 1 to minimize (1/M) ‖w‖₂².

  • x0 (npt.NDArray[np.float64] | None, optional) – Warm-start point forwarded to the inner solver.

Returns:

res – The single-objective minimizer.

Return type:

SolveResult

Raises:

ValueError – If objective_index is not 0 or 1.

evaluate_objectives(x)[source]#

Return [D(w, v), (1/M) ‖w‖₂²] at x.

Parameters:

x (npt.NDArray[np.float64]) – Weight vector.

Returns:

objectives – Array of shape (2,).

Return type:

npt.NDArray[np.float64]

trace(num_points=50, callback=None, n_jobs=1)[source]#

Trace the bias/variance tradeoff.

Solves the optimization problem for num_points values of φ between the two Pareto corners. Corner 0 minimizes D(w, v) (no variance constraint); corner 1 minimizes (1/M) ‖w‖₂² (no distance constraint).

Parameters:
  • num_points (int, default=50) – Number of grid points along the φ dimension.

  • callback (callable or None, optional) – Forwarded to the parent trace.

  • n_jobs (int, default=1) – Forwarded to the parent trace; n_jobs != 1 sweeps the grid across worker processes.

Returns:

corners[0] minimizes D(w, v); corners[1] minimizes variance. The .weights, .distances, and .variances convenience properties are sorted by variance ascending.

Return type:

EfficientFrontierResults

class pyrake.calibration.frontier.ExternalValidityVarianceFrontierResults(points, corners, primary_objective, n_attempted, n_skipped, tangent_slopes=None, knee_space='se', seam=None)[source]#

Bases: FrontierResults

Pareto frontier results for the external-validity/variance tradeoff.

Subclasses FrontierResults; inherits knee().

Reports effective sample size rather than the variance inflation factor the optimizer works in. The two are the same quantity: both experiment groups are pinned to mean weight one, so sum(w) = M and Kish’s ESS = (sum w)^2 / sum w^2 = M / mean(w^2). ESS is often more interpratable than the variance inflation factor. plot will also draw it in the optimizer’s own coordinates (variance_space="vif") or in their square root ("se"), the two spaces where the frontier is provably convex; see plot.

The .weights, .effective_sample_sizes, and .imbalances properties all return lists sorted by imbalance ascending (tight balance first), so that entry i of each refers to the same frontier point.

Parameters:
  • knee_space ({"vif", "se"}, default="se") – Which coordinate a band chosen off this frontier was chosen in, so plot can annotate the bend the calibration actually used instead of asking the caller to remember. A constructor argument rather than an attribute set afterwards, for the reason tangent_slopes is one: it is state not implied by the points, and a rebuild that drops it fails silently – here by marking a band nobody used, which looks exactly like a correct plot.

  • seam (float or None) – The smallest band every external row can meet, when the frontier was traced past it (see ExternalValidityVarianceFrontier). Below it a point’s imbalance coordinate is the band it was solved at, not its max imbalance, and plot marks the seam. None when the frontier stops at the seam or the bands can be met all the way down.

property weights#

Return the weights at each frontier point, sorted by imbalance ascending.

Returns:

weights – One weight vector per point.

Return type:

list[npt.NDArray[np.float64]]

property effective_sample_sizes#

Return Kish’s ESS at each point, sorted by imbalance ascending.

Returns:

ess – One effective sample size per point.

Return type:

list[float]

property imbalances#

Return the max external covariate imbalance at each point, sorted ascending.

Past the seam (see seam), the band the point was solved at.

Returns:

imb – One imbalance per point.

Return type:

list[float]

plot(annotate_knee=True, ax=None, x_label=None, y_label=None, title=None, variance_space='ess', knee_space=None, annotations=None, marker='*', marker_size=10, marker_color='orange', annotation_size=None)[source]#

Plot the external-validity/variance frontier.

Does not delegate to FrontierResults.plot, which always plots the primary objective itself. Here the x-axis is selectable: "vif" is that objective, "se" is its square root, and "ess" is a reciprocal transform of it. The interpolant is carried across the transform rather than redrawn – see _draw_frontier.

VIF and SE run the same way across the page, since the square root is increasing; ESS is their mirror image, since ESS = M / VIF is decreasing, so the curve that falls from top-left to bottom-right in VIF units rises from bottom-left to top-right in ESS units.

The three are not interchangeable, and the difference is not only cosmetic. The frontier is convex in VIF and in SE – the square root of a squared norm is a norm, and a value function under a convex objective is convex – but not in ESS, where a reciprocal destroys it. So a bend read off an ESS panel is not the same quantity as the bend the knee machinery solves for; see knee_space, and pass the two consistently if the marker is meant to be believed.

Parameters:
  • annotate_knee (bool, default=True) – If True, mark the knee with crosshairs and its dual-slope tangent.

  • ax (matplotlib Axes, optional) – If provided, plot onto this Axes; otherwise create a new figure.

  • variance_space ({"vif", "se", "ess"}, default="ess") – Which variance coordinate to put on the x-axis. "vif" plots the variance inflation factor (the design effect) the optimizer minimizes; "se" plots its square root, the factor by which a standard error is inflated (Kish’s DEFT), which is the version that enters a confidence interval; "ess" plots Kish’s effective sample size, which is the version a reader can price.

  • x_label (str, optional) – Defaults to the name of the chosen variance space.

  • y_label (str, optional) – Defaults to “Covariate Imbalance with Target Population”, or, on a frontier traced past the seam, to a label saying that below the seam the coordinate is the band rather than the imbalance.

  • title (str, optional) – Defaults to “External Validity vs <the chosen variance space>”.

  • knee_space ({"vif", "se"}, optional) –

    Which coordinate the annotated bend is measured in, independent of which one is drawn. Defaults to the space this frontier was traced for, so a band chosen in SE coordinates is annotated in SE coordinates without the caller having to say so twice. Override only to compare the two. See knee_external_scale.

    Read only on a frontier whose knee has not been refined. Once refine_knee has run, the knee it solved for is the one marked, and it already carries the space it was solved in – see _annotate_knee.

  • annotations (dict, optional) – Labelled reference points to mark, as {label: point}. A point is a FrontierPoint or an objective vector [variance inflation factor, max external imbalance], carried into variance_space like the curve. The corners are results.corners[1], the tightest external balance attainable (exact balance when it is feasible), and results.corners[0], no external balance at all. For weights solved some other way, pass frontier.evaluate_objectives(w); for Cochran’s band that is frontier.evaluate_objectives(calib.solve_with_fallback().solution).

  • marker (str, default="*") – Marker style for the annotations.

  • marker_size (float, default=10) – Marker size for the annotations.

  • marker_color (str, default="orange") – Marker color for the annotations.

  • annotation_size (float | str | None, optional) – Label font size for the annotations, or None for large.

Returns:

ax – The axes with the frontier drawn.

Return type:

matplotlib Axes

Raises:

ValueError – If the frontier has no points, variance_space is not one of the three recognized values, or an annotation is not a FrontierPoint or a finite 2-entry objective vector.

class pyrake.calibration.frontier.ExternalValidityVarianceFrontier(Z1, Z2, Z3, Z4, nu1, nu2, min_weight1, min_weight2, psi3=None, engine='pd', continue_past_seam=True)[source]#

Bases: MultiObjectiveOptimizer

Trace the external-validity/variance Pareto frontier for a joint calibration.

The two-block analogue of the single-population frontier. Where that class trades a single population’s covariate imbalance against variance, this one calibrates two experiment groups jointly and trades their external validity – each group against the target population – against variance, holding internal validity – the two groups against each other – fixed at psi3.

The two objectives are:

minimize    ((1/M) ‖w‖₂², ‖Zᵀw - ν‖∞)
subject to  mean(w1) = 1,
            mean(w2) = 1,
            w1 ≥ lb1,
            w2 ≥ lb2,
            ‖(1/M2) Z4ᵀw2 - (1/M1) Z3ᵀw1‖∞ ≤ ψ3,

traced by moving the second objective into a constraint with threshold φ:

minimize    (1/M) ‖w‖₂²
subject to  mean(w1) = 1,
            mean(w2) = 1,
            w1 ≥ lb1,
            w2 ≥ lb2,
            ‖(1/M2) Z4ᵀw2 - (1/M1) Z3ᵀw1‖∞ ≤ ψ3,
            ‖Zᵀw - ν‖∞ ≤ φ.

In this notation Z = [Z1 / √M1, Z2 / √M2] and ν = [√M1 ν1; √M2 ν2], so ‖Zᵀw - ν‖∞ ≤ φ is equivalent to the pair of per-group bands

‖(1/M1) Z1ᵀw1 - ν1‖∞ ≤ φ / √M1 =: ψ1, and
‖(1/M2) Z2ᵀw2 - ν2‖∞ ≤ φ / √M2 =: ψ2,

which is why a single scalar φ suffices to sweep both groups at once.

Parameters:
  • Z1 (npt.NDArray[np.float64]) – External-validity covariates for group 1 (control/Holdout) and group 2 (treated/Marketing), of shape (M1, q) and (M2, q). Whiten these by the population standard deviation, as elsewhere in PyRake, so that φ / √M reads directly as an allowed standardized mean difference.

  • Z2 (npt.NDArray[np.float64]) – External-validity covariates for group 1 (control/Holdout) and group 2 (treated/Marketing), of shape (M1, q) and (M2, q). Whiten these by the population standard deviation, as elsewhere in PyRake, so that φ / √M reads directly as an allowed standardized mean difference.

  • Z3 (npt.NDArray[np.float64]) – Internal-validity covariates for the two groups, same shapes. Usually the same matrices as Z1, Z2.

  • Z4 (npt.NDArray[np.float64]) – Internal-validity covariates for the two groups, same shapes. Usually the same matrices as Z1, Z2.

  • nu1 (npt.NDArray[np.float64]) – Target population means for Z1 and Z2, of shape (q,), whitened the same way.

  • nu2 (npt.NDArray[np.float64]) – Target population means for Z1 and Z2, of shape (q,), whitened the same way.

  • min_weight1 (float or npt.NDArray[np.float64]) – Lower bound on the weights of each group. Weights below M_b / N imply a propensity score above one, so M1 / N and M2 / N are the principled floors.

  • min_weight2 (float or npt.NDArray[np.float64]) – Lower bound on the weights of each group. Weights below M_b / N imply a propensity score above one, so M1 / N and M2 / N are the principled floors.

  • psi3 (float, optional) – Internal-validity tolerance, held fixed for the whole sweep. Defaults to 1.5 * sqrt(1 / M1 + 1 / M2), the two-sample analogue of Rake’s default and the same value JointCalibrator would choose.

Raises:

ValueError – If Z1/Z2 disagree with Z3/Z4 on the number of rows, or if the four do not share a column count. The external and internal blocks are reported against one shared covariate list, so they must describe the same covariates in the same order.

Notes

The motivation is JointCalibrator.solve_with_fallback, which aims for phi = 1.5 and, when that is infeasible, falls back to minimizing phi. That fallback silently selects the far end of this frontier, where the last few percent of covariate balance can cost most of the effective sample size. Tracing the frontier makes the choice explicit: results.knee() is typically a far better trade than the minimize-phi corner.

The sweep uses SquaredL2(v=0), so it reports the minimum-variance weighting at each phi rather than the minimum-divergence one. Because both groups are pinned to mean weight one, minimizing mean(w^2) is exactly maximizing Kish’s effective sample size: ESS = M / mean(w^2).

Past the seam. The band cannot always be tightened to zero. Below some phi – the seam, seam() – no weighting meets it on every row: a level with no respondents in one group, say, or a group whose shares must sum to one while its targets do not. With continue_past_seam (the default) the frontier does not stop there. Below the seam a point is what JointCalibrator.solve_with_fallback returns at that band: the rows that cannot meet it are relaxed, holding their summed excess to the least achievable, and the rest are held to it. The frontier’s imbalance coordinate is then the band, not the max imbalance, which stays pinned at the seam (see point_objectives).

It runs on down to the floor, floor(): the tightest band at which the rows that can meet it can all do so at once. Below the floor, solve_with_fallback would have to relax a row that could meet the band on its own and pin the rest at the band’s edge, so it widens the band back to the floor instead. A group whose shares must sum to one has a floor whenever one of its levels is empty: the observed levels must take up the empty one’s share, within their bands. The tight corner sits at the floor, and the knee may land on either side of the seam; a band between the two handed to solve_with_fallback reproduces the point the frontier traced. Where no row is unattainable – the band simply cannot be met on every row at once – the floor is the seam and there is nothing past it. Below the seam the curve is a two-stage solve, so it has no dual slope (those points report nan) and need not be convex.

classmethod from_calibrator(rake, psi3=None, engine='pd', continue_past_seam=True)[source]#

Build the frontier for a configured joint calibration.

Lives here rather than on JointCalibrator so the dependency stays one-way: this module imports rake, and nothing in rake needs to know that frontiers exist.

Parameters:
  • rake (JointCalibrator) – Source of the covariate blocks and weight floors.

  • psi3 (float or ndarray, optional) – Internal-validity band to hold fixed. Defaults to the calibrator’s own, per row, so a heterogeneous internal band is carried through rather than collapsed to its first entry.

  • engine ({"ipm", "pd"} or None, optional) – Inner solver for the sweep, matching EfficientFrontier: None inherits whatever rake was built with. Defaults to "pd" – on this shape (pure joint calibration, active l-infinity bands, no exact-balance block) PD measured ~4x faster than the barrier cold, 21ms against 88ms, and it warm-starts across the sweep where the barrier cannot.

  • continue_past_seam (bool, default=True) – See the class docstring.

Returns:

frontier – The frontier for the calibrator’s blocks and floors.

Return type:

ExternalValidityVarianceFrontier

Raises:

ValueError – If rake has no external-validity blocks, so there is no tradeoff to trace.

solve_with_bounds(bounds, x0=None)[source]#

Minimize (1/M) ‖w‖₂² subject to max external imbalance <= bounds[0].

Parameters:
  • bounds (npt.NDArray[np.float64]) – Array of shape (1,); bounds[0] is φ, which splits into the per-group bands ψ1 = φ / √M1 and ψ2 = φ / √M2.

  • x0 (npt.NDArray[np.float64] or None, optional) – Warm-start weights of length M1 + M2. MultiObjectiveOptimizer threads the previous grid point through trace, which is where the primal-dual engine earns its keep.

Returns:

res – The minimum-variance weights within the band.

Return type:

SolveResult

Raises:

ProblemInfeasibleError – If no weighting meets the band and the frontier does not continue past the seam, or internal validity cannot be met. trace catches this and drops the grid point, so an infeasible stretch shows up as a gap in the frontier rather than an exception.

Notes

Below the seam, and with continue_past_seam, this is solve_with_fallback at the band: see the class docstring. x0 is not used there.

seam()[source]#

Return the smallest band every external row can meet.

Found by the linear program behind the tight corner of a frontier that stops there. Memoized: it depends only on the covariates, the targets, the weight floors and the internal band.

Returns:

s – The seam band.

Return type:

float

floor()[source]#

Return the tightest band past the seam at which nothing has to be forced.

solve_with_fallback at a band near zero widens it to exactly this (see the class docstring); the band it reports is the floor, and the calibration there is the frontier’s tight corner. Memoized, like the seam.

Returns:

f – The floor band.

Return type:

float

Raises:

ProblemInfeasibleError – If internal validity, which the frontier holds fixed, had to be relaxed there as well.

point_objectives(result, bounds)[source]#

Record the objectives for a point: the band, past the seam.

Above the seam the band binds, so it equals the max imbalance, or the weights balance better than the band and their own max imbalance is the coordinate – evaluate_objectives either way. Below it the max imbalance is pinned at the seam by the rows that cannot move, so the coordinate is the band the point was solved at. The tight corner is solved at a band too, and reports it.

Parameters:
  • result (SolveResult) – The point’s solve.

  • bounds (npt.NDArray[np.float64]) – The band the point was solved at, or non-finite for a corner.

Returns:

objectives – [variance, band-or-imbalance].

Return type:

npt.NDArray[np.float64]

tangent_slopes(results)[source]#

Compute the frontier slope d(variance) / d(φ) from the imbalance duals.

By the envelope theorem the derivative of the optimum with respect to a constraint’s right-hand side is minus that constraint’s multiplier. Here the swept scalar phi is not one right-hand side: it is spread over the external rows as ψᵢ = φ / √M_b, so the slopes have to be aggregated with those weights,

d(variance)/d(phi)
    = -[(1/√M1) Σ_{i in block 1} λᵢ
       + (1/√M2) Σ_{i in block 2} λᵢ],

where λᵢ is the total multiplier on covariate i’s band. ImbalanceConstraintLayer encodes each band as two one-sided rows, the positive side then the negative, both carrying -ψᵢ, so λᵢ is their sum – at most one is active at a time. The internal rows do not move with phi and drop out.

Corners are reported as nan: corner 0 has no external-validity block and corner 1 is a linear program over an augmented variable, so neither shares this layer layout. A vertical tangent at corner 1 would be nan in any case.

Parameters:

results (FrontierResults) – The traced frontier, carrying the duals per point.

Returns:

slopes – None when duals are unavailable throughout, which leaves FrontierResults.envelope with the chord bound alone.

Return type:

npt.NDArray[np.float64] | None

minimize_objective(objective_index, x0=None)[source]#

Minimize a single objective with no bound on the other.

These are the two corners of the frontier, and trace calls this for each of them first: they fix the φ range the grid then sweeps between.

Parameters:
  • objective_index (int) – 0 to minimize (1/M) ‖w‖₂² with external validity unconstrained; 1 for the tight end: the least-variance weighting at the smallest band every row can meet (the seam) or, past the seam, at the floor.

  • x0 (npt.NDArray[np.float64] or None, optional) – Warm-start weights of length M1 + M2.

Returns:

res – Whose solution always has length M1 + M2, including for corner 1, where the augmenting φ is stripped before returning.

Return type:

SolveResult

Raises:

ValueError – If objective_index is not 0 or 1.

evaluate_objectives(x)[source]#

Return [(1/M) ‖w‖₂², ‖Zᵀw - ν‖∞] at x.

This is the frontier’s ground truth: MultiObjectiveOptimizer builds every FrontierPoint from this rather than from the inner solver’s own objective value, so the two need not agree on scale (and do not – SquaredL2(v=0) minimizes ‖w‖₂², a factor of M larger).

Parameters:

x (npt.NDArray[np.float64]) – Weights of length M1 + M2.

Returns:

objectives – Array of shape (2,). The first entry is M / ESS, since both groups are pinned to mean weight one.

Return type:

npt.NDArray[np.float64]

diagnose(x, covariate_names=None, phi=None)[source]#

Per-covariate imbalance against its band, tightest slack first.

Answers “what is stopping this from balancing better?”. A single row at slack_frac near zero means one covariate is the bottleneck; a cluster of them means the constraint set is pinned on many faces at once, which is the common case once a categorical is entered as a full set of dummies – those obey Σₖ σₖ dₖ = 0, so their imbalances cannot all be driven to zero independently and the solver ends up electing one level to absorb the residual.

Parameters:
  • x (npt.NDArray[np.float64]) – Weights of length M1 + M2.

  • covariate_names (sequence of str, optional) – Labels for the columns of Z, in order. Defaults to positional indices.

  • phi (float, optional) – The external-validity level the weights were solved at. Defaults to the level they actually achieve, which is what you want at a corner or anywhere the band is tight.

Returns:

One row per (kind, block, covariate), with imbalance, band, slack and slack_frac, sorted tightest first. slack_frac is the comparable column: the three groups carry different bands (phi/sqrt(M1), phi/sqrt(M2), psi3), so absolute slack is not comparable across them.

Return type:

pandas DataFrame

trace(num_points=50, callback=None, n_jobs=1, knee_space='se', refine_knee=True)[source]#

Trace the external-validity/variance tradeoff.

Solves for num_points values of φ between the two Pareto corners. Corner 0 minimizes (1/M) ‖w‖₂² with external validity unconstrained; corner 1 minimizes the max external imbalance.

Parameters:
  • num_points (int, default=50) – Number of grid points along the φ dimension.

  • callback (callable or None, optional) – Forwarded to the parent trace.

  • n_jobs (int, default=1) – Forwarded to the parent trace. Leave at 1: the sequential path warm-starts each solve from the previous one, which is where the primal-dual engine wins, and these solves are individually cheap.

  • knee_space ({"vif", "se"}, default="se") – Recorded on the results, and the space the knee is refined in when refine_knee is True. The traced curve is the same either way; only the question asked of it differs. Set it to whatever the band will be chosen in so the results carry their own answer to “which bend was this?”.

  • refine_knee (bool, default=True) – Solve for the knee with refine_knee once the sweep is done, so plot marks the knee a calibration would use rather than the nearest of the sampled points. Costs a handful of extra solves, which are folded into points. A refinement that finds no knee (a straight frontier) is recorded as such, and plot then marks none. Pass False to get the bare sweep, e.g. to refine it yourself and act on the returned knee, as knee_external_scale does.

Returns:

results – Whose .effective_sample_sizes and .imbalances are sorted by imbalance ascending, and whose plot() is labelled in ESS units by default, or in the optimizer’s own variance inflation factor with plot(variance_space="vif").

Return type:

ExternalValidityVarianceFrontierResults

Notes

The grid is uniform in phi, so a frontier that turns sharply near its minimum phi is sampled least densely exactly where it bends most. Infeasible grid points are skipped silently, so a gap in the curve means no weighting met that band, not a solver failure; results.n_skipped against results.n_attempted says how much of the grid landed. (Knee refinement adds its probes to both counts and to points, so len(results.points) can exceed num_points + 2.)

pyrake.calibration.frontier.solve_with_knee_fallback(rake, num_points=15, n_jobs=1, engine='pd', knee_space='se')[source]#

Calibrate, choosing the external band from the frontier’s knee.

JointCalibrator.solve_with_fallback minimizes the imbalance when the requested external band is infeasible, which silently selects the far end of the external-validity/variance frontier – where the curve is near-flat and the last few percent of balance cost most of the sample. On one real survey (M = 1007, smallest achievable band 4.19) that landed at an effective sample size of 164 where the knee offered 448: 2.7x the variance, 284 respondents’ worth of effective sample, for 7% better balance.

This traces the frontier and calibrates at max(1.5, phi_knee) instead – aim for the conventional Cochran scale, but never pay past the knee to reach it – then hands back the usual cascade result with the frontier attached.

Only the band comes from the frontier. The cascade still runs its own solve, so rake’s configured distance is what gets minimized; the frontier minimizes variance, and for anything but SquaredL2(v=0) those are different objectives.

Parameters:
  • rake (JointCalibrator) – A configured joint calibration. Its psi is restored before this returns, so it is not left holding a band nobody asked for.

  • num_points (int, optional) – Grid resolution for the sweep.

  • n_jobs (int, optional) – Worker processes for the sweep, forwarded to trace. The default runs sequentially and warm-starts each solve from the previous one, which is where the primal-dual engine earns its keep; n_jobs != 1 fans the grid out and solves each point cold, so it pays off when a single solve is expensive rather than when there are many cheap ones. The calibration itself is unaffected either way.

  • engine ({"ipm", "pd"} or None, optional) – Inner solver for the sweep only – the final calibration always runs through rake’s own engine. None inherits rake’s.

  • knee_space ({"vif", "se"}, optional) – Which coordinate the bend is located in; see knee_external_scale.

Returns:

result – With frontier set when a sweep informed the band, and None when the sweep could not run and the requested band was used instead.

Return type:

FallbackOptimizationResult

Notes

A sweep that cannot run – a corner solve that raises, say – degrades to the requested band and logs. That includes NewtonStepError and BacktrackingLineSearchError, which derive from Exception rather than OptimizationError and so are not caught by the grid sweep’s own swallow. A diagnostic sweep around a calibration should not turn a calibration that would otherwise succeed into an exception.

A sweep that runs but finds no knee is a different case and no longer degrades to the requested band: refine_knee returns None for a missing corner or a frontier too straight to have a bend, and knee_external_scale then uses the conventional band floored at what the frontier reached. The traced frontier is still attached.

pyrake.calibration.frontier.relabel_for_the_chosen_band(result, scale, results)[source]#

Say which external band was used, and why it is not the conventional one.

The cascade reports against the band it was handed, so calibrating at the knee and succeeding reads as “we achieved the desired covariate balance (both internal and external validity)”. True of the band the cascade was given, false of the band the caller wanted – and the second is the more misleading reading. A table of results would claim full success on every row while quietly sitting at several times the conventional imbalance.

Three outcomes, distinguished because they call for different responses:

  • The knee is tighter than the conventional scale, so the floor decided and the conventional band was met. The cascade’s own label is accurate.

  • The conventional scale is unattainable. Nothing could have done better: status 2, external validity not achieved.

  • The conventional scale was attainable and we declined it, buying variance with imbalance. A deliberate trade rather than a shortfall, and the one a reader must be told about explicitly, so it gets its own status.

scale is the band actually used, which need not be this calibration’s own knee: a caller holding several cells to one standard passes the loosest of them, so the wording speaks of the chosen band rather than the knee. results stays this calibration’s own frontier, which is what phi_min has to come from – attainability is a property of the cell, not of the agreement.

Parameters:
Returns:

relabelled – result with the message and status rewritten for the chosen band.

Return type:

FallbackOptimizationResult

pyrake.calibration.frontier.knee_external_scale(rake, num_points=15, n_jobs=1, engine='pd', knee_space='se')[source]#

Compute the external-validity scale implied by this calibration’s knee.

Returns max(1.5, phi_knee) – the conventional Cochran scale unless the knee says balance gets expensive before then – as a scale, not a band. Callers converting it to a band divide by sqrt(M_b) per group, which is what external_band does.

Split out from solve_with_knee_fallback so several calibrations can agree on one scale before any of them is solved. Cells of a lift study are compared against each other, so they should be held to a common standard; the caller takes the max across them and passes it back in.

Parameters:
  • rake (JointCalibrator) – A configured joint calibration.

  • num_points (int, optional) – Grid resolution for the sweep.

  • n_jobs (int, optional) – Worker processes for the sweep, forwarded to trace.

  • engine ({"ipm", "pd"} or None, optional) – Inner solver for the sweep. None inherits rake’s.

  • knee_space ({"vif", "se"}, default="se") – Which coordinate to locate the bend in. "vif" uses the variance inflation factor the sweep minimizes. "se" uses its square root, the standard-error multiplier, which is the quantity that actually enters a confidence interval – so trading imbalance against it is closer to a bias/precision rule than trading against a variance. Both are legitimate: the frontier is convex in either (the square root of a squared norm is a norm, and a value function under a convex objective is convex), so the root-find is well posed in both. They need not agree, and where they differ "se" is the looser of the two, because the square root stretches the low-variance end of the axis and the maximum-distance point migrates into the stretch. They coincide where curvature is concentrated at a single point, since every monotone transform of the axis then finds that same point – so the size of the disagreement measures how sharply the bend is defined, and the choice matters least where the two spaces differ least.

Returns:

  • scale (float or None) – None only when the sweep could not run at all – no imbalance rows, or a solve that raised – in which case the caller should keep the requested band. A sweep that ran but yielded no determinable knee (no slopes, a missing corner, or a frontier straight to within tolerance) returns a scale rather than None: the conventional _COCHRAN_SCALE, floored at the tightest band the frontier actually reached. Those are different failures. One leaves us knowing nothing about this calibration; the other leaves us with a traced frontier and only the location of the bend undetermined, which is enough to say the conventional band is attainable.

  • frontier (ExternalValidityVarianceFrontierResults or None) – The traced frontier, present whenever the sweep ran – including on the no-knee path, so callers can plot what was traced and see why no bend was found.

pyrake.calibration.frontier.external_band(rake, scale)[source]#

Compute the full psi vector for one external-validity scale.

The vector holds this calibration’s external validity at scale.

Each group’s band is scale / √M_b, so a shared scale gives groups of different size the bands that make them equally stringent – the same convention as PyRake’s 1.5 / √M default. The internal rows are left as they are.

A hair of slack is added because a scale read off a knee sits exactly on the band that knee was solved at, and adopting it verbatim leaves the subsequent solve with no strictly interior point to start from.

Parameters:
  • rake (JointCalibrator) – The joint calibration.

  • scale (float) – The shared external-validity scale.

Returns:

psi – The full band vector, or None when there are no external rows.

Return type:

npt.NDArray[np.float64] | None