“These benchmarks necessarily characterize only a narrow set of the browser functions in a very constrained way,” Microsoft's report said. “End users, however, do not operate in a controlled environment.”
Microsoft's tests pitted IE8 Release Candidate 1 (RC1), which launched in late January, against Google Inc.'s Chrome 1.0 and Mozilla Corp.'s Firefox 3.0.5, a version from mid-December. The company timed how long it took each browser to completely render the 25 most-popular destinations on the Web, as ranked by the Web metrics firm comScore Inc., which included google.com, facebook.com, amazon.com, and others.
IE8 was fastest in rendering 12 of the 25 sites, said Microsoft, while Chrome took second by beating the others on nine sites. Firefox, meanwhile, was a distant third, coming in first on just four of the 25 domains.
Microsoft did not test other browsers, such as Apple Inc.'s Safari or Opera Software ASA's Opera, said James Pratt, a senior product manager on the IE development team, because it wanted to focus on rivals that “had a good share on the Windows platform.”
Both Opera and Safari for Windows have shares of less than 1%, according to the most recent data from Net Applications Inc., with the former, on all platforms, accounting for 0.7% and the latter just 0.3%.
Dean Hachamovitch, IE's general manager, has called claims of competitors a “drag race” that Microsoft isn't interested in joining, while Pratt has downplayed comparisons of any kind. “We're at the point, with what people do in the browser, that users can't really tell the difference between browser [performance],” he said in a January interview.
Pratt said that the just-released report backed that up. “As you can see from the scores, the differences between the browsers are actually very small,” he said.
Although Google did not respond to a request for comment on Microsoft's benchmarks, Mozilla's Mike Shaver, who heads all development at the company, applauded any attempt to boost IE's performance. “I don't think anyone here has had a chance to really look at their methodology yet or tried to reproduce their results, but to whatever extent Microsoft is working to improve the performance of IE it's a good thing for the Web,” said Shaver in an e-mail late Thursday.
Microsoft's browser performance report can be downloaded from the company's Web site.