Video Inpainting Under Camera Motion

A framework for filling-in missing parts of a video sequence taken from a moving or stationary camera is presented in this work. An important scenario in video inpainting, namely, completing repeating moving foreground when it is either occluded by an obstacle or simply damaged, is considered. Video inpainting is achieved in two main steps. First, the occluded moving object in each frame is completed by copying information from undamaged portions of the video. The remaining parts of the frames are then filled-in with background, extending to the spatio-temporal domain spatial texture synthesis techniques. The moving object is progressively filled-in following a priority-based inpainting of the occluding area. A priority scheme with an automatic stopping criterion is proposed, which detects whether the moving object has been successfully completed. Moving object completion is accelerated by using a foreground mosaic to search for candidate frames containing similar motion. The proposed framework is a background-independent way of completing the moving object, which permits to efficiently inpaint regardless of the rich and changing background, which is typical of moving camera scenarios. The presentation of the underlying algorithm is completed with a number of real examples.

Preprint [PDF] ( Submitted For Review to IEEE - TIP )

Results:

(Videos are viewable only using I.E. 6.0 or later. If you are using a different browser, kindly download the videos using the link given below each place-holder.)

The following is a synthetic example where a part of the video is manually cut-out, to simulate a damaged camera sensor or speckle on the lens or camera film damage as in the case of old movies. Observe the background independent nature of our "moving person inpainting".

The inpainting scheme proposed here also works fairly well for camera motions that do not adhere to our constraints (section 2A of paper). The following is a simulation of a 'home-video' situation where a person of interest is being occluded as he moves along his trajectory.

The following result illustrates that our algorithm can deal with very large and moving occlusions in presence of camera motion. The original videos are of 640x480 resolution and can be obtained here.

NEW !! More Results :

1. Inpainting a moving object that slightly changes size: original and inpainted video

2. Comparison with previous work: videos with ladies-crossing and jumping-girl

Last updated on : Friday, May 12, 2006

The views and opinions expressed in this page are strictly those of the page author.
The contents of this page have not been reviewed or approved by the University of Minnesota.