UL Benchmarks, previously known as Futuremark, has delisted over 50 smartphone models powered by MediaTek processors, following recent reports that Mediatek's chips behaved differently under benchmarks in order to benefit from higher scores.

"Unfortunately, benchmark boosting continues to be a problem in the smartphone industry, despite the past scandals when manufacturers have been caught," UL said.

Last week, AnandTech reported that several smartphone models powered by MediaTek processors were producing artificially high and misleading benchmark scores. MediaTek is one of the largest fabless semiconductor companies in the world.

"After reviewing AnandTech's benchmark data, we have delisted over 50 smartphone models powered by MediaTek processors. Delisted devices appear unranked, and without PCMark for Android scores, at the bottom of our popular list of the best smartphones," UL added.

The company has temporarily delisted the PCMark for Android scores for all devices that use the following MediaTek chipsets:

MediaTek Helio G90
MediaTek Helio G70
MediaTek Helio P95
MediaTek Helio P90
MediaTek Helio P65
MediaTek Helio P60
MediaTek Helio P20
MediaTek Helio A22

In total, over 50 devices from more than 25 different vendors have been delisted including several recent, high-performing models such as the Xiaomi Redmi Note 8 Pro, Oppo Reno3 Pro, Vivo Y19, and Realme 6.

AnandTech tested the Oppo Reno3 Pro with MediaTek Helio P95 chipset with the public version of PCMark for Android, available from Google Play, and a private version of the app that is not available to the public or manufacturers.

They found a big difference in the scores from the public and private apps, even though the tests in each are identical. AnandTech found a benchmark whitelist in the device's firmware files. Further investigation found similar whitelist files on a host of devices powered by MediaTek chipsets, "Inspect the file, there we find amongst what seems to be a list of popular applications with various power management tweaks applied to them, with lo and behold, also a list of various benchmarks. We find the APK ID for PCMark, and we see that there’s some power management hints being configured for it, one common one being called a 'Sports Mode.'"

The device uses a hidden "Sports Mode" and other settings to artificially boost its performance while running the public PCMark for Android app. With the private app, the phone performs as expected for normal, everyday use.

"The difference in scores tells us that the device is simply recognizing the PCMark app by name rather than adapting to the type of work in the test. This kind of detection and optimization is forbidden by our rules for manufacturers," UL said.

In 2018, UL Benchmarks delisted Huawei phones (P20, P20 Pro, Nova 3 and Honor Play). Following that, UL Benchmarks also delisted OPPO’s Find X and F7 from its benchmark results list due to the two versions of the 3DMark app having hugely different performance results. As far back as 2013, UL Benchmarks delisted a whole host of Android phones from HTC and Samsung for cheating on its benchmarks. However, Samsung updated its phones to remove this cheating feature and admitted guilt.