The EU has announced that it will investigate multiple reports that Samsung's LCD panels cheat on energy efficiency by detecting when standardized tests are run and adjusting dynamic brightness levels in order to obfuscate their actual panel use. Samsung has denied any wrongdoing -- while the company acknowledges that power use can vary significantly by content, it claims that the observed differences are caused by general energy efficiency measures rather than any attempt to obfuscate a test. The EU is unlikely to accept any company's word on such topics in the near-future, however -- not after the recent VW scandal that's rocked the auto industry.
Samsung's hardware was tested by the EU's ComplianTV group, which found that Samsung's "motion lighting" feature reduced power consumption under internationally accepted test conditions, but not when television sets were deployed in normal use(Opens in a new window). A second Swedish group found evidence of additional cheating. “The Swedish Energy Agency’s Testlab has come across televisions that clearly recognize the standard film (IEC) used for testing... These displays immediately lower their energy use by adjusting the brightness of the display when the standard film is being run. This is a way of avoiding the market surveillance authorities and should be addressed by the commission."
Unfortunately, there's a confounding variable in all of this. Samsung may or may not be cheating on these standardized tests, but power consumption in an LCD panel can absolutely vary depending on what you're doing with it. In his recent LCD vs. OLED(Opens in a new window) shootout, Dr. Soneira posted power efficiency figures from multiple types of content between Samsung and LG displays:
Click to enlarge
What's tricky about this situation is that it's possible for all sets of claims to be true. Samsung claims that its motion lighting feature is meant to reduce power consumption in certain types of content, the EU is investigating claims that Samsung cheats, and it's obvious that different types of content produce very different levels of energy consumption. What this will come down to is whether Samsung and other TV manufacturers are claiming to hit efficiency targets in specific types of content that they don't actually deliver, and whether the TVs operate in the same mode when displaying standard content as with test content.
To some extent, this illustrates the problem with trying to create formal metrics for measuring performance. On the one hand, you want to have a unified set of tests that any organization can deploy to test common hardware. Designing custom benchmarks or sequences is extremely time-consuming, and there's always the chance that you may miss some type of content that ought to have been measured, but wasn't. As TVs become more sophisticated, however, they have gained the ability to detect and obfuscate certain kinds of tests.
These results also highlight the dangers of allowing companies to self-test and report the efficiency of their own hardware. Allowing the fox to guard the henhouse may save dollars in the short run, but it's an inevitable recipe for long-term trouble.
