If a nonlinear model is fitted to the data one often needs to estimate coefficients through optimization. A number of optimisation algorithms have the following general structure. Suppose that the function to be optimized is Q(β). Then the algorithms are iterative, defining a sequence of approximations, β_{k} given by

,

where is the parameter estimate at step k, and is a parameter (called step size) which partly determines the particular algorithm. For the BHHH algorithm λ_{k} is determined by calculations within a given iterative step, involving a line-search until a point β_{k}_{+1} is found satisfying certain criteria. In addition, for the BHHH algorithm, Q has the form

and A is calculated using

In other cases, e.g. Newton–Raphson, can have other forms. The BHHH algorithm has the advantage that, if certain conditions apply, convergence of the iterative procedure is guaranteed.^{[citation needed]}

Harvey, A. C. (1990). The Econometric Analysis of Time Series (Second ed.). Cambridge: MIT Press. pp. 137–138. ISBN0-262-08189-X.

Luenberger, D. (1972). Introduction to Linear and Nonlinear Programming. Reading, Massachusetts: Addison Wesley.

Sokolov, S. N.; Silin, I. N. (1962). "Determination of the coordinates of the minima of functionals by the linearization method". Joint Institute for Nuclear Research preprint D-810, Dubna.