There is a new and apparently better way to fix grainy photos, and it uses AI. Artificial Intelligence, one must say, coupled with lots of computing power, but that’s growing day after day.
Artificial Intelligence and machine learning will never cease to surprise us, apparently. Each new day, there’s some new announcement, covering different fields: AI can render 3D hair in real time, smell illnesses in human breath, assess infrastructure quality in Africa or help transform audio into music playing avatars. Now it can also help photographers get rid of noise in their photos. Filmmakers will be next, I assume…
https://youtu.be/pp7HdI0-MIo
This “now” has to be taken with a grain of salt, because we’re in the R&D phase of the whole process. A team of researchers from Nvidia, Aalto University, and MIT developed a way to restore images using Artificial Intelligence. According to Nvidia, “Recent deep learning work in the field has focused on training a neural network to restore images by showing example pairs of noisy and clean images. The AI then learns how to make up the difference. This method differs because it only requires two input images with the noise or grain.”
According to the information available, this Artificial Intelligence can remove artifacts, noise, grain, and automatically enhance your photos, without ever being shown what a noise-free image looks like. The paper published by the researchers indicates that “It is possible to learn to restore signals without ever observing clean ones, at performance sometimes exceeding training using clean exemplars,” adding that “[The neural network] is on par with state-of-the-art methods that make use of clean examples — using precisely the same training methodology, and often without appreciable drawbacks in training time or performance.”
The available results demonstrate the, we could call it, “magic” involved, but it is important to understand that the researchers had access to special equipment not yet available for your desktop computer. The team used Nvidia Tesla P100 GPUs with the cuDNN-accelerated TensorFlow deep learning framework, to train their system on 50,000 images in the ImageNet validation set.
The Nvidia Tesla P100 used is considered by Nvidia to be the world’s first AI supercomputing data center GPU. It taps into Nvidia Pascal GPU architecture to deliver a unified platform for accelerating both HPC and AI, dramatically increasing throughput while also reducing costs.
The Nvidia CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. cuDNN is part of the Nvidia Deep Learning SDK.
Deep learning researchers and framework developers worldwide rely on cuDNN for high-performance GPU acceleration. It allows them to focus on training neural networks and developing software applications rather than spending time on low-level GPU performance tuning. cuDNN accelerates widely used deep learning frameworks, including Caffe2, MATLAB, Microsoft Cognitive Toolkit, TensorFlow, Theano, and PyTorch.
TensorFlow is a software library for designing and deploying numerical computations, with a key focus on applications in machine learning. The library allows algorithms to be described as a graph of connected operations that can be executed on various GPU-enabled platforms ranging from portable devices to desktops to high-end servers. TensorFlow runs up to 50% faster on the latest Pascal GPUs and scales well across GPUs. Using ~ the time needed to train models has been reduced to hours instead of days.
The method now presented by the team of researchers, will appeal to photographers, if it becomes available in a commercial format, but the authors of the paper indicate that it can also be applied in other imaging areas and can even be used to enhance MRI images, perhaps paving the way to drastically improve medical imaging. The 12-page paper published by the team of researchers, entitled Noise2Noise: Learning Image Restoration without Clean Data is available online if you want to understand how the whole process works.
The team will present their work in an oral presentation and a poster at the 35th International Conference on Machine Learning, in Stockholm, Sweden, on Thursday, July 12.