an introduction to
Optimization on smooth manifolds

I taught Riemannian optimization at EPFL in spring 2023 for the course MATH-512. The recorded lectures are available below. A semester runs for 14 weeks. Most weeks have a lecture and an exercise session, which each last 90-95 minutes (week 5 had double lectures). The students further learned the material by completing two extensive projects.

Video links bring you to an external hosting service. There, you can change the playback speed of videos (play faster, slower): see controls at the bottom-right of the video panel. You can also download the videos.

I gratefully acknowledge EPFL's CEDE (Matthew Goodman in particular) for video post-production!

Exercises and solutions prepared with Christopher Criscitiello and Timon Miehling in 2023.

Click a question to display a sketch of the answer if available. Click a sketch to display a detailed answer if available. If no sketch is available but a detailed answer is, then clicking the question displays the detailed answer.

To load exercises below, go to the Lectures tab, and click "Exercises" next to a week's header.

You may find the Manopt toolboxes helpful to use Riemannian optimization in your projects:

• Manopt in Matlab,
• PyManopt in Python,
• Manopt.jl in Julia.

For example, in Matlab, to minimize the function $f(x) = x^\top A x$ on the sphere in $\Rn$, you can write:

n = 42;
A = randsym(n);
problem.M = spherefactory(n);
problem.cost = @(x) x'*A*x;
problem.egrad = @(x) 2*A*x;  % or use autodiff: problem = manoptAD(problem);
x = trustregions(problem);   % call an overly fancy algorithm

In Python, you could use JAX (among other backends) and write:

import pymanopt
import jax

n = 42
key = jax.random.key(0)
A = jax.random.normal(key, [n, n])
A = A + A.T

manifold = pymanopt.manifolds.Sphere(n)
@pymanopt.function.jax(manifold)
def cost(x):
    return x.T @ A @ x

problem = pymanopt.Problem(manifold, cost)
optimizer = pymanopt.optimizers.TrustRegions()
result = optimizer.run(problem)

In Julia, you might write:

using Manopt, Manifolds, LinearAlgebra
n = 42
A = Symmetric(randn(n, n))
M = Sphere(n-1)
f(E, x) = x'*A*x
∇f(E, x) = 2*A*x  # define the Euclidean gradient
x = trust_regions(M, f, ∇f; objective_type=:Euclidean)

Contact: nicolas.boumal@epfl.ch.

For general questions about geometry and optimization, please use the Manopt forum, MathOverflow or Math StackExchange.

I gave one of the SIAM Optimization 2023 minitutorials. It introduces the basics of differential geometry and Riemannian geometry for optimization, paralleling the early material in this book.

The two lectures of 90 minutes each are available as video 1 and video 2.

Here are the slides and also the Max-Cut example code.

Earlier (and more condensed) versions of that tutorial are available as a one-hour video (Simons Institute) and a two-hour video (ETHZ/EPFL summer school).

If you enjoy such topics, then you will definitely enjoy the classic book Optimization Algorithms on Matrix Manifolds by Absil, Mahony and Sepulchre (Princeton University Press, 2008), also freely available online.