So it all began last month, when Consumer Reports came up with an interesting battery result for the new MacBook Pro. What’s interesting about the result is that the upshots of the battery test ranged 3.75 hours to 19.5 hours. Naturally, due to such an inconsistent performance by the device, it was the first time ever that an Apple MacBook Pro did not receive a ‘buy’ recommendation from Consumer Reports. And now, after days of testing it turns out it was both a software bug and a benchmarking issue that caused such an erratic result.
Consumer Reports tested three MacBook Pro variants including the Touch Bar model and found that the battery results were “highly inconsistent from one trial to the next.” Now that the reason behind the erratic score is known, Apple has issued a software fix through its Beta Software Program to address the software end of the problem. Apple has posted its fix for the Safari bug on this website. The fix is available to anyone who signs up for Apple’s Beta Software program; it is then pushed to users through their computer’s Software Update function. Apple says the beta fix will be a part of a broader Software Update available to all MacBook Pro users in a few weeks, regardless of whether they sign up for the beta or not.
“We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results,” Apple said in a statement. “We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache…. We have also fixed the bug uncovered in this test.”
However, this still doesn’t explain the benchmarking error. Apple’s senior vice president of worldwide marketing, Phil Schiller, tweeted on December 23 that Apple had not seen the sort of wild swings in battery life in its own testing that Consumer Reposted experienced in theirs. ALSO READ: Consumer Reports stands by its findings on Apple MacBook Pro’s poor battery
Turns out that Consumer Reports while testing turned off Safari’s local cache for their browsing test. Basically doing that would makes the browser fetch a website’s data from the internet every time. When the cache is on, Safari stores images on the computer’s hard drive so that you don’t have to load them every time. “Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage,” Apple said in a statement to TechCrunch.
In Consumer Reports’ defence, the publication explained that it turns off the cache settings on the default browser for all laptops when testing, not just Apple’s laptops. This way, when they run a test it makes the system browse thousands of different websites, and not just the same ten websites over and over again.
Now, Consumer Reports method may not really reflect a real-world usage of a system, where people mostly use their computers to search the web, download music and watch movies and TV shows. However, deactivating a default setting on a browser, which only affects the working of the computer positively, seems unnecessary. ALSO READ: Disappointed MacBook Pro users switching to Surface, Microsoft claims
“This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life. We have also fixed the bug uncovered in this test. This is the best pro notebook we’ve ever made, we respect Consumer Reports and we’re glad they decided to revisit their findings on the MacBook Pro,” Apple said.
Consumer Reports has agreed for a retest and will soon be releasing its results. “Once our retesting of the MacBook Pro’s batteries is complete, we will report back with our update and findings,” said the publication.