Textbook: Textbook and other reading materials will be specified in class.
Objective
An introduction to the modern theory and methods of numerical optimization for computer/software engineering and computer science students.
Program
Introduction to the subject of numerical optimization
General optimization problem, extremality conditions, extrema of a quadratic form
Newton-Raphson optimization method
Gradient descend optimization
Trust regions, quasi Newton methods, DFP and BFGS; line search methods, backtracking, Armijo rule, Wolfe conditions
Levenberg-Marquardt method
Constrained optimization, general equality constraints, the method of Lagrange multipliers
MIDTERM
General inequality constraints, KKT theory, KKT equations, dual problems
The methods of nonlinear constrained optimization, barrier method, interior point method
Karmarkar's algorithm and its significance
The theory of convex optimization, convex functions and their properties
Constrained convex optimization, the art of constructing convex optimization programs
Overview of main types of convex optimization programs; linear programming, quadratic programming, cone programming, SOCP, geometric programming, semidefinite programming
Evaluation
Evaluation is performed according to the corresponding Faculty guidelines.