This was the second time we partnered with a major mobile device manufacturer* that sells its devices in almost all countries around the world. In fact, they are one of the world's top 3 smartphone manufacturers.
During this project, we continued to work on optimizing neural network models that were used as a part of an image enhancement process for all photos that are located in the library of a phone. This specific neural network was responsible for removing noise in images that had been caused by poor lighting conditions or other factors.
As always, consumers demand latency-less image editing, thus enabling real-time edge processing is vital for enhancing a mobile phone users’ experience.
*Unfortunately, we cannot disclose the name of the company due to an NDA.
Task: Important to note that one of the fundamental challenges in the field of image processing and computer vision is image denoising, where the underlying goal is to estimate the original image by suppressing noise from a noise-contaminated version of the image.
The underlying neural network was used as a part of a broader image processing pipeline, and it acts as a denoiser, which is then applied to each photo. There were two important points that had to be considered:
- Computational limitations - the neural network model was being run onboard a smartphone with limited RAM memory and processing power;
- Large input stream - the underlying neural network had to process every photo on the device.
As a consequence, the model must be light to minimize energy consumption and fast to ensure a great user experience.
Main technical parameters
- Accuracy - MOS (mean opinion score, subjective image perception);
- Speed - inference time, ms.
- Target device - ARM Cortex A52;
- Fixed input resolution - 256×256;
- Fixed bit depth of inference - INT8.
Target: decrease the time required for processing one image as much as possible, while ensuring an insignificant decrease in subjective image perception.
Results
Important to note that the underlying neural network had already been optimized by the customer themselves by applying several manual methods.
Yet,
after applying enot.ai’s technology, it was possible to identify a specific architecture which accelerated the neural network by 5.1 times,
while the index of subjective image perception decreased insignificantly (1.75 to 1.79).
Summary
The company was able to successfully deploy a neural network for denoising with a resulting acceleration of 5.1 times.