Parallel Curvature Filter for High Performance Image Processing

The problem

The curvature filter smooths and denoises an image by implicitly reducing a curvature energy, such as Gaussian curvature, mean curvature or total variation. It treats the image as a surface and repeatedly applies a cheap local operator to every pixel, each pass lowering the regularisation energy. The filter is fast, but many images that need enhancing are high-resolution or must be processed in real or near-real time, and on a CPU a large image such as 2048 × 2048 still takes minutes.

The most expensive part of the filter is a set of eight convolutions over the image. Inspired by the way GPUs run the convolutions of convolutional neural networks, the authors parallelise these convolutions on the GPU.

How it works

Each iteration of the curvature filter enumerates eight candidate moves for every pixel, picks the smallest one and applies it. PCF computes the eight candidates for all pixels at once with convolution kernels on the GPU.

1. The image as a surface. The image intensity U is interpreted as a surface over the pixel grid. The curvature regularisers all work on the local 3 × 3 neighbourhood around each pixel.
2. Eight tangent-plane projections. A surface can be locally approximated by its tangent plane. In the 3 × 3 neighbourhood, eight tangent planes through neighbouring pixels are enumerated, giving eight projection distances d₁ … d₈ for the centre pixel.
3. Eight convolutions on the GPU. Each distance is computed for every pixel at once as a convolution of the image. These eight convolutions are the most computationally expensive part of the algorithm; PCF runs them in CUDA and uses the GPU’s shared memory to accelerate them.
4. Pick the smallest distance and update. For every pixel, the distance d_m with the smallest magnitude is selected (ABSMIN) and added to the pixel: U^t+1 = U^t + d_m. The smallest distance indicates the closest fit to the desired geometry and to the original image.
5. Iterate. Repeating the update regularises the image surface step by step, which the paper shows by plotting the intensity along one line of the image after 10 and 30 iterations.

Results

The CUDA implementation was run with GPU block sizes from 16 to 32 on an NVIDIA GeForce GTX 1080 Ti (3,584 CUDA cores, 11,264 MB of global memory, 49,152 bytes of shared memory per block) and compared with the original curvature filter on an Intel Xeon E5-1620 v4 CPU at 3.50 GHz. In Fig. 3 the computation time grows with the number of iterations on both, but far more slowly on the GPU. The authors conclude that this speed makes the filter suitable for real-time applications such as video processing and biomedical image processing.

The paper also applies Gaussian-curvature and mean-curvature versions of PCF to a photograph and plots the intensity along one line after 10 and 30 iterations. In both, the profile becomes smoother as iterations accumulate; judging from these profiles, the authors find the mean-curvature PCF more efficient at smoothing than the Gaussian-curvature PCF (a comparison of smoothing, not of running time).