Removing camera shake via weighted Fourier burst accumulation

Value Proposition

One of the most challenging experiences in photography is taking images in low-light environments. The basic principle of photography is the accumulation of photons in the sensor during a given exposure time. In general, the more photons reach the surface of the sensor the better quality of the final image, as the photonic noise is reduced. However, this basic principle requires the photographed scene to be static and that there is no relative motion between the camera and the scene. Otherwise, the photons will be accumulated in neighboring pixels, generating a loss of sharpness (blur). This problem is significant when shooting with hand-held cameras, the most popular photography device today, in dim light conditions. Current cameras including those found in camera phones are designed to take a burst of images. This has been exploited in several approaches for accumulating photons in the different images and then forming an image with less noise. However, this principle is disturbed if the images in the burst are blurred. The classical mathematical formulation of this problem as a multi-image deconvolution, seeks to solve an inverse problem where the unknowns are the multiple blurring operators and the underlying sharp image. This procedure is computationally very expensive (prohibiting its on-camera implementation), and very sensitive to a good estimation of the blurring kernels. Furthermore, since the inverse problem is ill-posed it relies on priors either or both for the calculus of the blurs and the latent sharp image.

Technology

Duke inventors report a method for automatically removing the blur and noise of images intended to be implemented in cameras, camera phones, or photo-editing software. If the photographer takes a burst of images, a modality available in virtually all modern digital cameras, this invention combines them to generate a clean, sharp version. This is done without explicitly solving any blur estimation and subsequent inverse problem. The proposed algorithm is strikingly simple: it performs a weighted average in the Fourier domain, with weights depending on the Fourier spectrum magnitude. The method can be seen as a generalization of the align and average procedure, with a weighted average, motivated by hand-shake physiology and theoretically supported, taking place in the Fourier domain. The method's rationale is that camera shake has a random nature, and therefore, each image in the burst is generally blurred differently. Experiments with real camera data, and extensive comparisons, show that the proposed Fourier burst accumulation algorithm achieves state-of-the-art results an order of magnitude faster, with simplicity for on-board implementation on camera phones. The inventors have demonstrated experiments in real high dynamic range (HDR) scenes, showing how the method can be straightforwardly extended to HDR photography.

Advantages

  • A simple method for generating crisp images using digital cameras, including phone cameras
  • Can be easily implemented with any camera capable of taking burst photos
  • Technology has been reduced to practice