Algorithm Recommendations

How humpday.minimize() picks an algorithm when you don't specify one — and which algorithms are eligible for which kinds of problem.

How auto-selection works

When you call minimize(f, bounds=...) without a method= argument, three filters decide which optimizer runs:

1. Dimensional cap. Some algorithms are structurally unfit above a certain n_dim: GridSearch is exponential; BayesianOpt's Gaussian process scales cubically in observations and its RBF kernel degenerates; PRIMA_UOBYQA needs (n+1)(n+2)/2 interpolation points before it can build its full quadratic model.
2. Minimum trials. BayesianOpt's GP doesn't pay off until you've drawn enough samples to fit it; CMAEvolutionStrategy wants several generations of 4·n evaluations to adapt its covariance; Powell's PRIMA family must initialize an interpolation set before the first iteration.
3. Overhead tier vs. measured eval-time. At minimize() entry we time your objective four times and take the median. The resulting per-call cost decides which algorithms are worth running: when each call is a microsecond, BayesianOpt's GP fit and CMA-ES's eigendecomposition would dominate wall-clock and are filtered out; when each call is a second, every algorithm is eligible and we lean toward the sample-efficient ones.

The recommender consults a benchmarks-driven grid (benchmarks/recommendation_grid.json) when one is available, picking the eligible algorithm with the smallest median best-value on the nearest (n_dim, n_trials) cell. When no grid is present it falls through to the rule-based ranking that mirrors suggest_pure.

Dimensional caps

Algorithms not listed here are treated as having no cap (linear-in- dim or better).

Overhead tiers

Each algorithm is classified by its per-iteration bookkeeping cost. An algorithm is eligible only when your objective's measured eval-time is at least the tier's threshold — below that, the algorithm's own work would dominate wall-clock.

Calling convention

The auto-selection is the default; you can always override.

from humpday import minimize

# Auto-selection (the default).
result = minimize(my_objective, bounds=[(-5, 5)] * 8)
print(result.method)              # "DifferentialEvolution" (say)
print(result.eval_time_measured)  # 1.4e-04 (s/call)
print(result.tier)                # 1
print(result.fun, result.x)

# Disable the timing call — useful for stochastic objectives.
result = minimize(my_objective, bounds=..., options={"auto_timing": False})

# Force a specific algorithm.
result = minimize(my_objective, bounds=..., method="CMAEvolutionStrategy")

Reading and building the grid

benchmarks/recommendation_grid.json is the empirical input to auto-selection. Each cell is keyed "n_dim/n_trials" and holds, for every algorithm that cleared the structural eligibility filter at that cell, the raw per-(objective, seed) best-value plus an aggregated median_best / mean_wall summary. The objective suite is sphere, Rosenbrock, Rastrigin, Griewank, and Salomon — chosen to cover separable convex, banana, rugged multimodal, and nearly-separable landscape regimes.

Parallel. Every (cell, algorithm, objective, seed) tuple is an independent unit of work; pass --workers N to use a process pool. The script pins BLAS to one thread per worker so the numpy-heavy algorithms (CMA-ES, BayesianOpt) don't contend.

Incremental. Re-running the script reads any existing grid file, skips cells / algorithms / objectives / seeds that already have a recorded result, and only runs the missing tuples. Re-running with new --dims or --trials extends the grid rather than overwriting it, and the meta.first_built timestamp is preserved. Writes are atomic (sibling-temp + rename) so a kill mid-write can't corrupt the file.

# Fast development sweep (~30s, parallel, default cpu_count - 1 workers).
python benchmarks/build_recommendation_grid.py --quick

# Full default sweep (dims 2,5,10,20,50,100 × trials 50,200,1000).
python benchmarks/build_recommendation_grid.py --workers 16 --seeds 5

# Extend an existing grid with a new dimension only.
python benchmarks/build_recommendation_grid.py --dims 30

# Retry algorithms previously marked "too slow" (e.g. on a beefier machine).
python benchmarks/build_recommendation_grid.py --include-skipped

Algorithms whose probe run exceeds the per-call wall-clock budget (default 30s) are marked skipped_too_slow for that cell — the next run won't waste cycles on them unless you pass --include-skipped.

When to override the recommender

• Stochastic objectives. The timing call samples f(x) once at the cube centre — if your objective is non-deterministic, that one sample sets random state. Pass options={"auto_timing": False} to skip it; the recommender then falls back to dim & trials only.
• Known landscape structure. If you know your surface is convex-ish, pick LBFGSB or Powell directly; if it's known multimodal with expensive evaluations, pick BayesianOpt.
• Reproducibility. Algorithm choice that depends on timing isn't bitwise reproducible across machines. Pass method= explicitly when you need byte-for-byte determinism.

See SOTA status for how each algorithm compares against its canonical reference implementation, and Algorithms for the per-algorithm implementation notes.