****optimize augmented_lagrangian

Description

This optimizer is a basic implementation of the classic augmented-lagrangian method for constrained optimization. For problems of the form:

\(\left\{ \begin{array}{l} Min ~~f(x)\\ g_i(x) \leq 0 ~~~~(i=1,...,m)\\ x \in \Re^n \end{array} \right.\)

a dual method is used to find a saddle point of the augmented lagrangian function \(\hat{L}(x,\lambda,r)\):

\(\left\{ \begin{array}{l} Max_{\lambda} ~\hat{w}(\lambda,r) ~~=~~ Max_{\lambda} ~\left\{ ~Min_x ~\hat{L}(x,\lambda,r) ~\right\}\\ \lambda \in \Re^{m+} \end{array} \right.\)

where \(\hat{L}(x,\lambda,r)\) is defined by:

\(\hat{L}(x,\lambda,r)~=~f(x)~+~\sum_{i=1}^m \lambda_i~g_i(x) ~+~r~\sum_{i=1}^m g_i(x)^2\)

with \(\lambda_i\) the lagrange multipliers, and \(r\) a penalty parameter that insure the differentiability of the dual function \(\hat{w}(\lambda,r)\) near the optimum \(\lambda^{*}\).

The penalty parameter \(r\) must then be chosen large enough to improve convergence of the dual problem, but not too large because the lagrange function \(\hat{L}(x,\lambda,r)\) can become ill-conditioned in this case.

For each value of the lagrange multiplier \(\lambda_i\) the unconstrained problem \(Min_x ~\hat{L}(x,\lambda,r)\) is solved by a BFGS method. At each iteration of the BFGS method an economic line-search procedure based upon the Goldstein rule has been implemented.

Syntax

The following adjustable parameters are allowed in the ***convergence section.

iter

Maximum number of function \(f\) evaluations before terminating. Default is 100. Note that this doesn’t include function evaluations associated with the line-search procedure.

perturb

Magnitude of the perturbation (in percent of the normalized design variables) used to calculate gradient through finite differences. Default value is 0.5 percent.

r0

Value of the penalty parameter \(r\). Default is 1.0.

aug_conv

Convergence parameter for dual problem based upon the relative variation of design variables. Default value is 1.e-7.

bfgs_conv

Convergence parameter for the inner unconstrained optimization problem based upon the relative variation of design variables. Default value is 1.e-7.

oned_conv

Convergence parameter for the line search procedure. Default value is 1.e-7.

gradient_tol

Additional convergence parameter for the inner unconstrained problem, based upon the gradient of the lagrangian function. Default value is 1.e-7.

constraint_tol

Constraint violation tolerance value, ie. constraints are verified when \(g_i(x)<constraint\_tol\), and the corresponding design point \(x\) retained as feasible. Default value is 0.001.