MediaTek turns benchmark cheating into a service, availed by Xiaomi, Vivo, Oppo, others

We’ve heard of individual smartphones and OEMs cheating on benchmarks, using manual overrides to remove the thermal cap and let benchmark apps run processor cores at their peak speeds, disregarding thermal constraints. But never before did a chipset vendor get caught fudging the chipset firmware to show higher benchmarks. If a report by AnandTech is to be believed, MediaTek has been up to some real shady practices with some of its chipsets that power a bunch of smartphones from Oppo, Vivo, Xiaomi, Realme and more.

The article is a long read and if you are interested in the technicalities of how MediaTek has been cheating on benchmarks, we recommend you read the full article by Andrei Frumusanu who has also worked on unmasking Huawei and Samsung when they cheated on benchmarks.  The short version of the story is that the publication noticed that the European variant of the Oppo Reno3 Pro powered by the MediaTek Helio P95 outperformed the MediaTek Dimensity 1000L chip in the Chinese variant of the Reno3 on PCMark. The Dimensity 1000L is the first high-end chipset from MediaTek after a long time and is considered a chipset that can give the Snapdragon 865 a run for its money, at least on paper.

AnandTech used a tweaked version of the PCMark app and found there was a 30 percent difference in general scores with some workloads even showing a 75% variance. Upon investigating, it was found that the device firmware files house a few lines of code that applied power management tweaks to popular benchmark apps for Android. A common name for the enhanced mode is called ‘Sports Mode’ in the code.

The investigation also led to finding the default thermal and scheduler settings (the things that control thermal throttling and assign workloads to processor cores respectively) are being altered when these benchmark apps are run. The Sports Mode basically lets the memory controller run at max frequency while the scheduler is set to be more aggressive in having CPU cores ramp up frequency for a sustained period.

This wouldn’t have been that big of an issue if MediaTek was transparent about including this unshackled state of existence, but the company chose to hide it in plain sight for years. AnandTech went back to a Sony Xperia device from 2016 and found the same code lying in the firmware. The same was found across devices from Oppo, Vivo, Xiaomi, Realme, iVoomi and Sony.

In the process of reporting this menace, AnandTech also found Oppo changing the firmware around through an update, and making the suspicious code disappear. The boosted scores are still there, however, even after the update which means Oppo has just moved the code somewhere else.

Upon reaching out to MediaTek, AnandTech was told that this follows ‘accepted industry standards’. MediaTek’s response was along the lines of vendors having the flexibility to configure their own devices, and that they routinely design devices to “run on the highest possible performance levels when benchmark tests are running.” Furthermore, it maintained that MediaTek chipsets are capable of dynamically scaling up the performance basis of the app that’s running or the tasks that’s been assigned. You can read MediaTek's justification in a blog post posted on its website.

That’s how today’s smartphones offer Gaming Modes which prioritises performance for one game at a time. Most smartphones also ship with a dedicated High-Performance mode that tweaks the scheduler to make processor cores run at higher frequencies for better performance. But this is something that the user may benefit directly from while playing a game or running an app. 

What is the benefit in showing benchmark scores that may not be the best representation of real-life scenarios? If someone is buying a smartphone based on the higher benchmark scores a MediaTek device has received and ends up being disappointed with it later, it’s doing more harm than good for MediaTek, especially at a time when the Taiwan-based chipmaker finally has a product that can compete with high-end Qualcomm mobile chipsets. We hope MediaTek learns from this and disables the artificial boosting of benchmark scores.

from Latest Technology News https://ift.tt/2RkXTrV