Deep Learning Approaches in Image Morphing
speaker: Marko Rajković (Rheinische Friedrich-Wilhelms-Universität Bonn)abstract: Image morphing in computer vision amounts to computing a visually appealing transition of two images. A prominent model for these purposes, known as image metamorphosis, was originally proposed by Trouve and Younes. Here, the space of images is endowed with a Riemannian metric that separately quantifies the contributions dues to transport and image intensity variations along a transport path. The classical metamorphosis model consider images as an square-integrable function on some image domain. Such an approach is non-sensitive to image features such as sharp interfaces or fine texture patterns, and additionally has problems in processing binary images. To resolve these drawbacks, we give two possible approaches.
The first improvement is to treat images not as intensity maps but as maps into some feature space that encode local structure information. To appropriately treat local structures and semantic information, the images are processed by deep convolutional neural networks such as VGG. The structure of this network is additionally suitable for necessary multi-layer approach. Additional improvement of the model is achieved by using anisotropic regularization which reflects the edge structure of underlying images. The resulting model is formulated directly in terms of a variational time discretization develop for the classical metamorphosis model by Berkels, Effland and Rumpf. The key ingredient is a mismatch energy that locally approximates the squared Riemannian distance and consists of a regularization energy of the time discrete flow and a dissimilarity energy that measures the feature vector modulation along discrete transport paths. The spatial discretization is based on a finite difference and a stable spline interpolation and for minimization a variant of the iPALM algorithm is used.
The another approach that we propose is learning a sparse and rotationally invariant image representation using a convolutional model that results in a decomposition into a cartoon image and textures encoded in discretized roto-translation spaces. Then we apply a similar time and space discrete model as for the previously proposed model.
timetable:
Wed 9 Oct, 19:00 - 19:20, Aula Dini
<< Go back