Читать книгу Mathematical Programming for Power Systems Operation - Alejandro Garcés Ruiz - Страница 38

The gradient method is, perhaps, the most simple and well-known algorithm for solving optimization problems. Cauchy invented the basic method in the 19th century, but the computed advent leads to different applications that encompass power systems operation and machine learning. Let us consider the following unconstrained optimization problem:

(2.29)

where the objective function f : ⁿ → is differentiable. The gradient ∇f(x) represents the direction of greatest increase of f. Thus, minimizing f implies to move in the direction opposite to the gradient. Therefore, we use the following iteration:

(2.30)

The gradient method consists in applying this iteration until the gradient is small enough, i.e., until ‖∇f(x)‖ ≥ ϵ. It is easier to understand the algorithm by considering concrete problems and their implementation in Python, as given in the next examples.