Over the past 20 years, a last collection of work has been dedicated to the definition of shape, and shape spaces, as mathematical objects, and to their applications to various domains in computer graphics and design, computer vision and medical imaging. In this last context, an important scientific field has emerged, initiated by U. Grenander and M. Miller, called Computational Anatomy. One of the primary goals of computational anatomy is to analyze diseases via their anatomical effects, i.e., via the way they affect the shape of organs. Shape analysis has demonstrated itself as a very powerful approach to characterize brain degeneration resulting from neuro-cognitive impairment like Alzheimer's or Huntington's, and has contributed to deeper understanding of disease mechanisms at early stages.

Whether represented as a curve, or a surface, or as an image, a shape requires an infinite number of parameters to be mathematically defined. It is an infinite-dimensional object, and studying shape spaces requires mathematical tools involving infinite-dimensional spaces (functional analysis) or manifolds (global analysis). Some example are reviewed in the excellent survey paper from Bauer et al.. 

Spaces emerging from Grenander's Pattern Theory have a special interest, because of their generality and flexibility. These spaces derive from the structure induced by groups of diffeomorphisms through the deformation induced by their actions on shapes. In this context, a shape is not represented as such but as a deformation of another (fixed) shape, called template. The deformable template paradigm is rooted in the work of D'Arcy-Thompson in his celebrated treatise (On Growth and Form), and developed in Grenander's theory. Even if Pattern Theory can be more general, recent models of deformable templates in shape analysis focus on deformations represented by diffeomorphisms acting on landmarks, curves, surfaces or other structures that can represent shapes. More precisely, if \(T_0\) is the template, one represents shapes via the map \(\pi(\varphi) = \varphi \cdot T_0\), which denotes the action of a diffeomorphism, \(\varphi\) on \(T_0\) (\(\varphi\) being a diffeomorphism in the ambient space, in contrast to changes of parametrization, which are diffeomorphisms of the parametrization space). With this model, the diffeomorphism can be interpreted as an extrinsic parameter for the representation. 

Letting \(\mathrm{Diff}\) denote the space of diffeomorphisms, and \(\mathcal Q\) be the shape space, one can use the transformation \(\pi : \mathrm{Diff} \to \mathcal Q\) to  "project" a mathematical structure defined on diffeomorphisms to the shape space. Using this paradigm, one can, from a single modeling effort (on \(\mathrm{Diff}\)) design many shape spaces, like spaces of landmarks, curves surfaces, images, density functions or measures, etc.

The space of diffeomorphisms, which forms an algebraic group, is a well studied mathematical object. The relationship between right-invariant Riemannian metric on this space and classical equations in fluid mechanics has been described in V.I. Arnold's seminal work, followed by a large literature, by J.E. Marsden, T. Ratiu, D.D. Holm and others. It is remarkable that the same construction induces interesting shape spaces leading to concrete applications in domains like medical image analysis.

Because the transformation \(\varphi \rightarrow \pi(\varphi)\) is many-to-one in general, the projection mechanism from \(\mathrm{Diff}\) to \(\mathcal Q\) involves an optimization step over the diffeomorphism group: given a target shape \(T\), one looks for an optimal diffeomorphism \(\varphi\) such that \(\pi(\varphi) = \varphi\cdot T_0 = T\). Shapes are then compared by comparing these optimal diffeomorphisms, or some parametrization that characterizes them. Optimality is based on the Riemannian metric on \(\mathrm{Diff}\), and more precisely on the distance between \(\varphi\) and the identity mapping \(\mathrm{id}\) for this metric. The resulting \(\pi\) then has the properties of what is called a Riemannian submersion. Because the constraint \(\pi(\varphi)=T\) is hard to achieve numerically in general, one preferably replaces this constraint by a penalty term in the minimization, so that the  diffeomorphism representing a shape is sought via the minimization of
\[
\varphi \mapsto \mathrm{dist}(\mathrm{id}, \varphi) + \lambda E(\varphi\cdot T_0, T)
\]
where \(E\) is an error function. This formulation leads to the LDDMM (large deformation diffeomorphic metric mapping) algorithm, first introduced for landmarks and images, then for curves and surfaces.  In this approach, the optimal \(\varphi\) is computed as the flow of an ODE (ordinary differential equation), so that \(\varphi(x) = \psi(1,x)\) with
\[
\frac{d\psi}{dt}(t,x) = v(t, \psi(t,x))
\]
where \(v\) is a time-dependent vector field in the ambient space. The problem can then be reformulated as an optimal control problem where \(v\) is the control, minimizing
\[
(v , \psi) \mapsto \int_0^1 \|v(t, \cdot)\|^2_V dt + E(\psi(1, \cdot)\cdot T_0, T)
\]
subject to \(\frac{d\psi}{dt}(t,x) = v(t, \psi(t,x))\), where \(\|\cdot\|_V\) is a norm over a Hilbert space \(V\) of smooth vector fields (e.g., reproducing kernel Hilbert space). Introducing the time variables results in a continuous deformation from the template to the target.
 

More details can be found in the following papers, and in other works of  M. Miller, S. Joshi, A. Trouvé, J. Glaunès, D.D. Holm, F-X Vialard, S. Durleman etc.

Matching deformable objects, A Trouvé, L Younes, Traitement du Signal 20 (3), 295-302, 2003 The metric spaces, Euler equations, and normal geodesic image motions of computational anatomy, M.I. Miller, A. Trouvé, L. Younes, Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on, 2003
Computing large deformation metric mappings via geodesic flows of diffeomorphisms, M.F. Beg, M.I. Miller, A. Trouvé, L. Younes, International journal of computer vision 61 (2), 139-157, 2005
The Euler-Lagrange equation for interpolating sequence of landmark datasets, MF Beg, M Miller, A Trouvé, L Younes, Medical Image Computing and Computer-Assisted Intervention-MICCAI 2003, 918-925, 2003
On the metrics and Euler-Lagrange equations of computational anatomy, MI Miller, A Trouvé, L Younes, Annual review of biomedical engineering 4 (1), 375-405, 2002
Diffeomorphic matching of distributions: A new approach for unlabelled point-sets and sub-manifolds matching, J. Glaunes, A. Trouvé, L. Younes, Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, 2004
Large deformation diffeomorphic metric mapping of vector fields, Y. Cao, M.I. Miller, R.L. Winslow, L. Younes, Medical Imaging, IEEE Transactions on 24 (9), 1216-1230, 2005
Modeling planar shape variation via Hamiltonian flows of curves, J. Glaunès, A. Trouvé, L. Younes, Statistics and analysis of shapes, 335-361, 2006
Diffeomorphic matching of diffusion tensor images, Y. Cao, M.I. Miller, S. Mori, R.L. Winslow, L. Younes, Computer Vision and Pattern Recognition Workshop, 2006. CVPRW'06. Conference on, 2006
Large deformation diffeomorphic metric curve mapping, J. Glaunès, A. Qiu, M.I. Miller, L. Younes, International journal of computer vision 80 (3), 317-336, 2008
A kernel class allowing for fast computations in shape spaces induced by diffeomorphisms, A. Jain, L. Younes, Journal of Computational and Applied Mathematics, 2012

The following are review papers focusing on the optimal control approach to LDDMM registration.
Diffeomorphometry and geodesic positioning systems for human anatomy, MI Miller, L Younes, A Trouvé, Technology 2 (01), 36-43
Hamiltonian systems and optimal control in computational anatomy: 100 years since D'Arcy Thompson MI Miller, A Trouvé, L Younes Annual review of biomedical engineering 17, 447-509, 2015
Computational anatomy and diffeomorphometry: A dynamical systems model of neuroanatomy in the soft condensed matter continuum MI Miller, S Arguillère, DJ Tward, L Younes Wiley Interdisciplinary Reviews: Systems Biology and Medicine 10 (6), 2018

Beyond the registration problem, which is addressed in the previous papers, additional issues can be raised, and refinements can be brought via the rich structure brought by the Riemannian submersion \(\pi\). The optimality equation, which has the same structure as the one discovered by Arnold, was presented in relation with shape analysis and computational anatomy in
Geodesic shooting for computational anatomy, M.I. Miller, A. Trouvé, L. Younes, Journal of mathematical imaging and vision 24 (2), 209-228, 2006
Geodesic shooting and diffeomorphic matching via textured meshes, S. Allassonnière, A. Trouvé, L. Younes, Energy Minimization Methods in Computer Vision and Pattern Recognition, 365-381, 2005
Soliton dynamics in computational anatomy, D.D. Holm, J. Tilak Ratnanather, A. Trouvé, L. Younes, NeuroImage 23, S170-S178, 2004

Further developments around this equation, and the important problem of transport of vectors or covectors along geodesics is addressed in the following papers, which, among other things, provide equations for parallel and coadjoint transport.
Jacobi fields in groups of diffeomorphisms and applications, L. Younes, Quarterly of applied mathematics 65 (1), 113-134, 2007
Transport of relational structures in groups of diffeomorphisms, L. Younes, A. Qiu, R.L. Winslow, M.I. Miller, Journal of mathematical imaging and vision 32 (1), 41-56
Evolution equations in computational anatomy, L. Younes, F. Arrate, M.I. Miller, NeuroImage 45 (1), S40-S50, 2009

The diffeomorphic mapping approach has also been applied to surface evolution (introducing area-minimizing diffeomorphic flows), segmentation (diffeomorphic active contours) and tracking.
Diffeomorphic surface flows: A novel method of surface evolution, S. Zhang, L. Younes, J. Zweck, J.T. Ratnanather, SIAM journal on applied mathematics 68 (3), 806-824, 2008
Diffeomorphic active contours, F. Arrate, J.T. Ratnanather, L. Younes, SIAM journal on imaging sciences 3 (2), 176-198, 2012
Modeling and Estimation of Shape Deformation for Topology-Preserving Object Tracking, V Staneva, L Younes, SIAM Journal on Imaging Sciences 7 (1), 427-455, 2014