It’s fast becoming the measure smartphone manufacturers and OS developers use to convey how fast their products are, and now one Canadian firm has taken Apple’s iOS and Google’s Android through 45,000 tests to see just who has the fastest web browsing experience.
Blaze Software Inc is a Canadian outfit dedicated to improving browsing speeds and they were kind enough to run us through some of their testing to see who rendered websites the fastest – Android or iOS. Unfortunately though, the study was found to be flawed.
What Blaze did was that they created their own testing software based on Apple’s UIWebView and Google’s WebView. The tests were performed on an iPhone 4 (running both iOS 4.3 and 4.2) and Samsung Nexus S (running Android 2.3) while a Samsung Galaxy S was used for testing Android 2.2.
All the websites tested were those of Fortune 1000 companies with each site being tested 3 times, with a median result used.
Over 45,000 tests were run across the various platforms and operating system versions, with the results indicating that recent JavaScript handling improvements on both platforms don’t translate to real-world speed increases.
After taking over 45,000 measurements on the latest iPhone and Android devices, the study found that Android was 52% faster than iPhone on average. Android finished loading a Web page faster on 84% of the 1000 Websites tested. The study also found that the despite significant JavaScriptperformance gains in the latest Apple iOS 4.3 release and Google Android 2.3 releases, these improvement made no measurable improvement on the actual page load times of the sites tested.
Study proved Google’s OS to be faster than iOS in 84% of tests. Average load-times were also measured, with Android’s speed showing through with an average time of 2.1 seconds compared to the average 3.2 seconds required by iOS’ Safari.
Blaze also made the tools used available to the public via its website. Here users can test any site they like, using various Android and iOS versions to see how their site runs.
The full Blaze press release is attached below:
Ottawa, ON, March 17, 2011 – Blaze Software Inc released today the largest ever research study of smart phone browser performance. The purpose of the study was to determine once and for all which of the two leading smart phone vendors has the fastest browser.
Mobile Web browser usage is exploding. Emarketer estimates that 44.1% of US citizens will leverage mobile Internet by 2014. To capture market interest in mobile browsing, smart phone vendors have been aggressively touting the speed improvements in their products. However, due to the lack of mobile measurement tools, it has been difficult to measure which smart phone actually has the faster browser.
After taking over 45,000 measurements on the latest iPhone and Android devices, the study found that Android was 52% faster than iPhone on average. Android finished loading a Web page faster on 84% of the 1000 Websites tested. The study also found that the despite significant JavaScript performance gains in the latest Apple iOS 4.3 release and Google Android 2.3 releases, these improvement made no measurable improvement on the actual page load times of the sites tested.
“We were very surprised by the results”, said Guy Podjarny, Blaze CTO and Co-Founder. “We assumed that it would be closer race and that the latest JavaScript speed improvements would have a more material impact on performance. The fact that Android beat iPhone by such a large margin was not expected”.
What makes this study unique is the size of the study and the fact that it used real phones on real world websites to make the measurements. Past studies have often used fabricated benchmark sites or manual measurements on a small number of sites. This study was made possible through custom apps developed to measure page load time on mobile devices. These apps run on the actual devices, load a page on demand, and measure how long it took. These agents are available as a free service to measure any site with the Blaze Mobitest Tool.
Detailed blog post on the Blaze Mobile Measurement Study For more information on the details results and methodology of the study, please see: www.blaze.io/blog
For more information on the Mobile measurement service Blaze’s mobile measurement service can be found at: www.blaze.io/mobile
About Blaze Blaze was founded in 2010 with a mission to help clients deliver better performing Web businesses by optimizing websites to increase website speed. Blaze provides a hosted Web Performance Optimization service that improves frontend performance and reduces operational costs. For more information, see: www.blaze.io
*This report is based on our own analysis leveraging the technology and methodology outlined in this report. Blaze Software Inc. is in no way affiliated with Google or Apple.
How is it flawed you may ask? The Loop explains:
One of the biggest surprises the Blaze team found was that “despite significant JavaScript performance gains in the latest Apple iOS 4.3 release and Google Android 2.3 releases, these improvement made no measurable improvement on the actual page load times of the sites tested.”
There is a good reason for this. According to Blaze’s own documentation the “measurement itself was done using the custom apps which use the platform’s embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone.”
The problem with using UIWebView is that, even though it’s based on Safari, it didn’t receive any of the updates that Safari did in iOS 4.3. Using an embedded browser is not the same as using the official browser.
Apple’s Safari Web browser included the Nitro JavaScript engine that Apple said runs JavaScript up to twice as fast as its predecessor. Since UIWebView didn’t include any of those enhancements, it’s kind of disingenuous to say that Android beat Safari.
Apple also officially responded to the tests, saying:
Their testing is flawed because they didn’t actually test the Safari web browser on the iPhone. Instead they only tested their own proprietary app which uses an embedded web viewer that doesn’t take advantage of Safari’s web performance optimizations. Despite this fundamental testing flaw, they still only found an average of a second difference in loading web pages.
