bootstrap_custom
bootstrap_custom(
data,
resample,
fit=None,
statistic=None,
jackknife=None,
replicates=1000,
alpha=0.1,
ci_method='Percentile',
seed=12345,
parameters=None,
inner_replicates=None,
max_retries=None,
run_type='regular',
fit_with_covariance=None,
original_covariance=None,
pivotal_links=None,
pivotal_invalid_draw_policy='drop',
regularize_pivotal_covariances=True,
pivotal_z_limit=None,
add_pivotal_jitter=False,
pivotal_jitter_scale=0.01,
)Bootstrap your own statistic.
Runs the ported Numerics bootstrap against resampling, fitting and statistic functions you write. This is the class upstream exposes as four delegates – ResampleFunction, FitFunction, StatisticFunction and JackknifeFunction – so any quantity you can compute from a fitted parameter set can be given a confidence interval, not just the built-in distribution quantiles :func:corehydropy.bootstrap_analysis covers.
Getting an argument order wrong is the likeliest mistake here, and the C++ side cannot tell a swapped pair from a deliberate one, so each signature is given exactly:
resample(data, parameters, rng) Returns one bootstrap sample. data is the original sample, parameters is the current parameter list, and rng is a :class:corehydropy.Rng handle on THIS replicate’s generator – draw from it with rng.uniform() and rng.integers(), not with :mod:random or :mod:numpy.random, or the seeded run stops being reproducible and stops agreeing with R. The returned sample need not be the same length as data. fit(data) Returns the parameters fitted to data: one number per parameter, the same count every time. A one-parameter model may return the bare number rather than [x], which is the spelling R uses for it. statistic(parameters) Returns the numbers to put intervals on, computed from a fitted parameter vector: one or more, the same count every time. A single statistic may likewise be returned bare. jackknife(data, index) Returns data with observation index left out. index counts from 0, matching the ported delegate, so the Python spelling is data[:index] + data[index + 1:]. Only the "BCa" method uses it; every other method ignores it. fit_with_covariance(data) Returns {"parameters": ..., "covariance": ...} – or the tuple (parameters, covariance) – with the parameters fitted to data and their covariance matrix, one row and one column per parameter. Only run_type="pivotal" uses it, and that run type uses it INSTEAD of fit.
The pivotal run type. run_type="pivotal" is upstream’s other bootstrap mode. Rather than treating each resampled fit as a draw from the sampling distribution, it standardizes the fit against the original one through the resample’s OWN covariance and reinflates it through the original’s, so a replicate fitted on an unusually flat likelihood contributes an appropriately smaller step. It therefore needs a covariance with every fit, which is what fit_with_covariance supplies in place of fit.
The original fit is the parent every draw is compared against. Left alone it is fit_with_covariance(data); parameters replaces its parameter vector and original_covariance its covariance, either independently of the other.
The result gains pivotal_diagnostics – the six replicate counts the run kept, from requested_replicates down to retained_pivotal_replicates – and a second interval block, raw_lower/raw_upper and companions, which is the plain percentile interval of the RAW covariance-aware fits before the transform. Comparing the two is the point of reporting both. Only ci_method="Percentile" exists after a pivotal run, and asking for another is refused before the first replicate rather than after all of them.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| data | array_like | The original sample: non-empty and finite. | required |
| resample | callable | The two always-required functions, with the signatures above. | required |
| statistic | callable | The two always-required functions, with the signatures above. | required |
| fit | callable | The fitting function, required by run_type="regular" and unused – and so refused – by run_type="pivotal", which fits through fit_with_covariance instead. |
None |
| jackknife | callable | The leave-one-out function, required by ci_method="BCa" and unused by every other method. |
None |
| replicates | int | Number of bootstrap replicates. | 1000 |
| alpha | float | The interval’s total tail probability: 0.1 gives a 90% interval, alpha / 2 in each tail. |
0.1 |
| ci_method | ('Percentile', 'BiasCorrected', 'Normal', 'BootstrapT', 'BCa') | "Normal" and "BootstrapT" work on the ported cube-root transform of the statistic; "BootstrapT" runs the studentized workflow, which nests inner_replicates further resample-and-fit pairs inside every replicate. |
"Percentile" |
| seed | int | PRNG seed; 12345 is the C# default. | 12345 |
| parameters | array_like | The original parameter vector the replicates are compared against. Left unset, fit(data) is used, which is what the bootstrap ordinarily means by it. |
None |
| inner_replicates | int | Inner replicates for ci_method="BootstrapT", ignored by every other method. Left unset, the ported default (300) applies. |
None |
| max_retries | int | The maximum number of times a single failed replicate is retried before it is counted in failed_replicates. Left unset, the ported default (MaxRetries, 20) applies. Each retry is another crossing into Python, so lowering it caps the worst case rather than changing the typical one. |
None |
| run_type | ('regular', 'pivotal') | See the section above. | "regular" |
| fit_with_covariance | callable | The covariance-aware fitting function run_type="pivotal" fits through, with the signature above. Required by that run type and refused by the other. |
None |
| original_covariance | array_like | The parent fit’s covariance matrix as a sequence of rows, one row and one column per parameter. Left unset, it is taken from fit_with_covariance(data). Pivotal only. |
None |
| pivotal_links | sequence | One entry per parameter, each a link function name ("Identity", "Log", "Logit", "Probit", "ComplementaryLogLog", "YeoJohnson" or "FisherZ") or None for the identity. The standardization happens in link space, so "Log" on a scale parameter keeps every reinflated draw positive. Left unset, the identity throughout. Pivotal only. |
None |
| pivotal_invalid_draw_policy | ('drop', 'use_raw', 'use_parent') | What to do with a draw the transform could not produce (a non-finite standardized vector, or one outside pivotal_z_limit): leave it out of the ensemble, keep the untransformed fit, or substitute the original fit. Pivotal only. |
"drop" |
| regularize_pivotal_covariances | bool | Whether each link-space covariance is made symmetric positive definite before it is factored. True by default, as upstream. Pivotal only. | True |
| pivotal_z_limit | float | An absolute limit on every component of the standardized vector, beyond which the draw is invalid and the policy above applies. Left unset, no limit. Pivotal only. | None |
| add_pivotal_jitter | (bool, float) | Whether to add Gaussian jitter to the standardized vector before the limit is checked, and its base standard deviation (the applied scale is this divided by the square root of the parameter count). False and 0.01 by default, as upstream. Pivotal only. | False |
| pivotal_jitter_scale | (bool, float) | Whether to add Gaussian jitter to the standardized vector before the limit is checked, and its base standard deviation (the applied scale is this divided by the square root of the parameter count). False and 0.01 by default, as upstream. Pivotal only. | False |
Returns
| Name | Type | Description |
|---|---|---|
| dict | Per statistic: estimate (the statistic of parameters, not a bootstrap average), lower, upper, standard_error, mean (the mean over valid replicates, so mean - estimate is the bias estimate) and valid_count; the same first three for the fitted parameters as parameter_estimate, parameter_lower and parameter_upper; and replicates, failed_replicates, alpha, ci_method and run_type. A pivotal run adds pivotal_diagnostics (a dict of the six replicate counts) and the raw block: raw_estimate, raw_lower, raw_upper, raw_standard_error, raw_mean, raw_valid_count, raw_parameter_estimate, raw_parameter_lower and raw_parameter_upper. |
Notes
How many times your functions are called. replicates calls of each of resample, fit and statistic, plus one extra statistic call to learn how many values it returns and one fit call when parameters is not supplied. A failed replicate is retried up to max_retries times (20 by default), and "BCa" adds one jackknife + fit + statistic per observation. "BootstrapT" is the expensive one: it multiplies the resample and fit counts by inner_replicates, so the ported defaults (10,000 x 300) would be three million crossings back into Python. Start small. A pivotal run calls fit_with_covariance in place of fit, once per replicate plus once up front for the parent fit, and statistic once per retained draw plus once per raw fit – so about twice as often as a regular run of the same length.
Reproducing a run in R. The draws come from the core’s seeded Mersenne Twister, so an identical bootstrap_custom() call in R resamples the identical observations. The numbers your functions compute from them are your own Python code, though, and Python and R do not guarantee identical rounding for the same formula: arithmetic (+ - * /) is IEEE-deterministic and does reproduce, while log, exp and friends come from each platform’s math library. Note also that R’s sum() and mean() accumulate in extended precision where Python’s do not, so a plain loop is the portable spelling on both sides.
See Also
corehydropy.bootstrap_analysis : the built-in parametric bootstrap of a fitted distribution’s quantiles. corehydropy.Rng : the generator handle resample is given.
Examples
>>> import corehydropy as ch
>>> x = [4.1, 5.2, 4.8, 5.5, 4.9, 5.1, 5.3, 4.7]
>>> def resample(data, parameters, rng):
... return [data[k] for k in rng.integers(len(data), 0, len(data))]
>>> def fit(data):
... acc = 0.0
... for xi in data:
... acc += xi
... return acc / len(data)
>>> res = ch.bootstrap_custom(x, resample, fit, lambda p: p, replicates=500, seed=12345)
>>> bool(res["lower"][0] < res["estimate"][0] < res["upper"][0])
TrueThe pivotal run type. It fits through fit_with_covariance instead of fit: a Normal location-scale MLE here, whose covariance is diag(s2 / n, s2 / 2n) in closed form. The "Log" link standardizes the scale parameter in log space, keeping every draw positive.
>>> def fit_with_cov(data):
... n = len(data)
... mu = sum(data) / n
... s2 = sum((xi - mu) ** 2 for xi in data) / n
... return {"parameters": [mu, s2 ** 0.5],
... "covariance": [[s2 / n, 0.0], [0.0, s2 / (2 * n)]]}
>>> piv = ch.bootstrap_custom(
... x, resample, statistic=lambda p: p, fit_with_covariance=fit_with_cov,
... run_type="pivotal", pivotal_links=[None, "Log"], replicates=200, seed=12345)
>>> piv["pivotal_diagnostics"]["retained_pivotal_replicates"]
200Two interval blocks: the pivotal ensemble, and the raw fits it was built from.
>>> bool(piv["lower"][0] < piv["raw_lower"][0])
True