Uhh ohh, Huawei has been caught manipulating benchmark scores attained during 3DMark benchmarks.
This is according to director of engineering at UL Benchmarks, Jani Loki.
This week the firm was alerted to suspiciously high benchmark scores for Huawei and Honor handsets thanks to some sleuthing by Anandtech.
“After testing the devices in our own lab and confirming that they breach our rules, we have decided to delist the affected models and remove them from our performance rankings,” says Loki.
The director explains that the firm ran benchmarks on the devices using the publically available 3DMark app and a private app. The tests are reportedly identical.
“We found that the scores from the public 3DMark app were up to 47% higher than the scores from the private app,” explains the director.
“With the public 3DMark app, these devices appear to use a hidden ‘Performance Mode’ that overrides the devices’ usual power profile,” Loki adds.
Because of this secret mode that contravenes UL Benchmarks rules the following Huawei and Honor devices have been delisted from 3DMark benchmarks.
- Huawei P20 Pro
- Huawei Nova 3
- Honor Play
- Huawei P20 (based on reporting from Anandtech)
How bad is this?
The question now is can you trust benchmarks of Huawei devices?
Well not entirely but benchmarks really only give you a baseline against which to compare other devices. These benchmarks are synthetic and don’t really show off real world performance but rather what a piece of tech could do when you require more computing power.
True, this makes it hard to compare Huawei devices with other brands and for ordinary people this makes it harder to justify purchasing a Huawei handset based solely on benchmarks.
Thankfully this is why we have reviews and we always suggest reading as many of them as possible before parting ways with your money.
Huawei has already responded to the controversy saying that it will release an update that will give users access to this ‘Performance Mode’ which is okay according to UL Benchmarks but as it stands, Huawei handsets aren’t being compared in 3DMark for now.