API Reference

minimizeCompass

noisyopt.minimizeCompass(func, x0, args=(), bounds=None, scaling=None, redfactor=2.0, deltainit=1.0, deltatol=0.001, feps=1e-15, errorcontrol=True, funcNinit=30, funcmultfactor=2.0, paired=True, alpha=0.05, disp=False, callback=None, **kwargs)

Minimization of an objective function by a pattern search.

The algorithm does a compass search along coordinate directions. If errorcontrol=True then the function is called repeatedly to average over the stochasticity in the function evaluation. The number of evaluations over which to average is adapted dynamically to ensure convergence.

The algorithm terminates when the current iterate is locally optimally at the target pattern size deltatol or when the function value differs by less than the tolerance feps along all directions.

Parameters:
  • func (callable) – objective function to be minimized: called as func(x, *args), if paired=True, then called with keyword argument seed additionally
  • x0 (array-like) – starting point
  • args (tuple) – extra arguments to be supplied to func
  • bounds (array-like) – bounds on the variables
  • scaling (array-like) – scaling by which to multiply step size and tolerances along different dimensions
  • redfactor (float) – reduction factor by which to reduce delta if no reduction direction found
  • deltainit (float) – initial pattern size
  • deltatol (float) – target pattern size, function differences at this scale need to be larger than stochasticitiy in evaluations to ensure convergence if errorcontrol=False
  • feps (float) – smallest difference in function value to resolve
  • errorcontrol (boolean) – whether to control error of simulation by repeated sampling
  • funcNinit (int, only for errorcontrol=True) – initial number of iterations to use for the function, do not set much lower than 30 as otherwise there is no sufficient statistics for function comparisons
  • funcmultfactor (float, only for errorcontrol=True) – multiplication factor by which to increase number of iterations of function
  • paired (boolean, only for errorcontrol=True) – compare for same random seeds
  • alpha (float, only for errorcontrol=True) – significance level of tests, the higher this value the more statistics is acquired, which decreases the risk of taking a step in a non-descent direction at the expense of higher computational cost per iteration
  • disp (boolean) – whether to output status updates during the optimization
  • callback (callable) – called after each iteration, as callback(xk), where xk is the current parameter vector.
Returns:

special entry: free Boolean array indicating parameters that are unconstrained at the optimum (within feps)
Return type:

scipy.optimize.OptimizeResult object

minimizeSPSA

noisyopt.minimizeSPSA(func, x0, args=(), bounds=None, niter=100, paired=True, a=1.0, alpha=0.602, c=1.0, gamma=0.101, disp=False, callback=None)

Minimization of an objective function by a simultaneous perturbation stochastic approximation algorithm.

This algorithm approximates the gradient of the function by finite differences along stochastic directions Deltak. The elements of Deltak are drawn from +- 1 with probability one half. The gradient is approximated from the symmetric difference f(xk + ck*Deltak) - f(xk - ck*Deltak), where the evaluation step size ck is scaled according ck = c/(k+1)**gamma. The algorithm takes a step of size ak = a/(0.01*niter+k+1)**alpha along the negative gradient.

See Spall, IEEE, 1998, 34, 817-823 for guidelines about how to choose the algorithm’s parameters (a, alpha, c, gamma).

Parameters:
  • func (callable) – objective function to be minimized: called as func(x, *args), if paired=True, then called with keyword argument seed additionally
  • x0 (array-like) – initial guess for parameters
  • args (tuple) – extra arguments to be supplied to func
  • bounds (array-like) – bounds on the variables
  • niter (int) – number of iterations after which to terminate the algorithm
  • paired (boolean) – calculate gradient for same random seeds
  • a (float) – scaling parameter for step size
  • alpha (float) – scaling exponent for step size
  • c (float) – scaling parameter for evaluation step size
  • gamma (float) – scaling exponent for evaluation step size
  • disp (boolean) – whether to output status updates during the optimization
  • callback (callable) – called after each iteration, as callback(xk), where xk are the current parameters
Returns:
Return type:

scipy.optimize.OptimizeResult object

minimize

noisyopt.minimize(*args, **kwargs)

deprecated: use minimizeCompass instead

bisect

noisyopt.bisect(func, a, b, xtol=1e-06, errorcontrol=True, testkwargs={}, outside='extrapolate', ascending=None, disp=False)

Find root by bysection search.

If the function evaluation is noisy then use errorcontrol=True for adaptive sampling of the function during the bisection search.

Parameters:
  • func (callable) – Function of which the root should be found. If errorcontrol=True then the function should be derived from AverageBase.
  • b (a,) – initial interval
  • xtol (float) – target tolerance for interval size
  • errorcontrol (boolean) – if true, assume that function is derived from AverageBase.
  • testkwargs (only for errorcontrol=True) – see AverageBase.test0
  • outside (['extrapolate', 'raise']) – How to handle the case where f(a) and f(b) have same sign, i.e. where the root lies outside of the interval. If ‘raise’ throws a BisectException.
  • ascending (allow passing in directly whether function is ascending or not) – if ascending=True then it is assumed without check that f(a) < 0 and f(b) > 0 if ascending=False then it is assumed without check that f(a) > 0 and f(b) < 0
Returns:
Return type:

float, root of function

Average classes

These helper classes perform averages over function values. They provide extra logic such as tests whether function values differ signficantly.

class noisyopt.AverageBase(N=30, paired=False)

Base class for averaged evaluation of noisy functions.

N

number of evaluations

test0(x, type_='smaller', alpha=0.05, force=False, eps=1e-05, maxN=10000)

Compares the mean at x to zero.

Parameters:
  • type (in ['smaller', 'equality']) – type of comparison to perform
  • alpha (float) – significance level
  • force (boolean) – if true increase number of samples until equality rejected or meanse=eps or N > maxN
  • eps (float) –
  • maxN (int) –
class noisyopt.AveragedFunction(func, fargs=None, **kwargs)

Average of a function’s return value over a number of runs.

Caches previous results.

N

number of evaluations

diffse(x1, x2)

Standard error of the difference between the function values at x1 and x2

test(xtest, x, type_='smaller', alpha=0.05)
Parameters:
  • type (in ['smaller', 'equality']) – type of comparison to perform
  • alpha (float) – significance level
test0(x, type_='smaller', alpha=0.05, force=False, eps=1e-05, maxN=10000)

Compares the mean at x to zero.

Parameters:
  • type (in ['smaller', 'equality']) – type of comparison to perform
  • alpha (float) – significance level
  • force (boolean) – if true increase number of samples until equality rejected or meanse=eps or N > maxN
  • eps (float) –
  • maxN (int) –
class noisyopt.DifferenceFunction(func1, func2, fargs1=None, fargs2=None, **kwargs)

Averages the difference of two function’s return values over a number of runs

N

number of evaluations

test0(x, type_='smaller', alpha=0.05, force=False, eps=1e-05, maxN=10000)

Compares the mean at x to zero.

Parameters:
  • type (in ['smaller', 'equality']) – type of comparison to perform
  • alpha (float) – significance level
  • force (boolean) – if true increase number of samples until equality rejected or meanse=eps or N > maxN
  • eps (float) –
  • maxN (int) –