Consider the well-known morphing videos in which a person's
face smoothly transforms into someone else's. In such videos, facial
features from the original image progressively deform to reach their
target, but at the same time, these feature may change color, as
illustrated by the strip below, which was created by Laurent Garcin
during his Ph. D. Thesis. Here, you see eyes, mouth or hair moving,
but also the color progressively changing when transitioning from
the first image to the last. The first image's background fades out
and is replaced by the last image's one.
This sequence provides an interpolation, in some appropriate space
of images, between the first image and the last one. These were the
input of the algorithm, and intermediate images are computer
generated.
The same principles can be applied to a whole variety of deformable
objects -- not only images. A general framework involves a Lie group
of "deformations" acting on a Riemannian manifold of shapes or
images, and evolution processes on the manifold that continuously
deform a time-dependent object, called template. Optimal
trajectories are then obtained by penalizing large deformations and
template variation.
Some examples.
The following videos that provide more examples of image
evolution for optimal metamorphosis. In all these examples, the
algorithm started with the first and last images of the sequence,
and estimated a best interpolation using metamorphosis. In some
cases, the interpolation results in an apparent motion of the image.
In other cases, more deformation and image intensity changes are
involved. The last examples provides metamorphosis in the
space of densities, for which the transformation multiplies the
deformed image by a Jacobian determinant, like in the change of
variable formula.
Various examples of image metamorphosis, followed by one example of density metamorphosis.
References:
[1] Group actions, homeomorphisms, and
matching: A general framework, M.I.
Miller, L. Younes, International
Journal of Computer Vision 41 (1-2), 61-84, 2001
[2] Geodesic
interpolating splines, V. Camion, L. Younes, Energy
Minimization Methods in Computer Vision and Pattern Recognition,
513-527, 2005
[3] Local geometry of deformable templates, A. Trouvé, L. Younes, SIAM J. Math. Anal.,
37(1), 17-59, 2005
[4] Metamorphoses through lie group action, A. Trouvé, L. Younes, Foundations of Computational Mathematics 5 (2),
173-198, 2005
[5] Geodesic
image matching: a wavelet based energy minimization scheme, L.
Garcin, L. Younes, Energy Minimization Methods in Computer Vision
and Pattern Recognition, 349-364, 2005
[6] Geodesic
matching with free extremities, L. Garcin, L. Younes,
Journal of Mathematical Imaging and Vision 25 (3), 329-340, 2006
[7] The
Euler-Poincaré theory of metamorphosis, D.D. Holm, A. Trouvé,
L. Younes, Quarterly of Applied Mathematics 97, 661-685, 2009
[8] Computing
Metamorphoses between Discrete Measures, C.L. Richardson, L.
Younes, J. Geom. Mech. 5 (1), 131-150, 2013
[9] Metamorphosis of images in reproducing kernel Hilbert spaces
, C.L. Richardson, L.
Younes, Adv. Comp. Math. 42, 573-603, 2016