Comparing clouds

Findings include:

• Some cloud providers rely heavily on the public Internet to transport traffic instead of their backbones, which can impact performance predictability. While Google Cloud and Azure rely heavily on their private backbone networks to transport their customer traffic, protecting it from performance variations associated with delivering over the public Internet, AWS and Alibaba Cloud rely heavily on the public Internet for the majority of transport, resulting in greater operational risk that can impact performance predictability. IBM takes a hybrid approach that varies regionally.
• Latin America and Asia have the highest performance variations across all clouds, whereas in North America, cloud performance is generally comparable. Decision-makers should consult the detailed findings to choose the best cloud provider on a per-region basis to ensure optimal performance globally, as regional performance differences can make a significant impact in terms of performance gains or losses. 
• AWS Global Accelerator doesn’t always out-perform the Internet. AWS Global Accelerator was introduced in November 2018 to let customers use the AWS private backbone network for a fee – rather than use the public Internet, which is AWS’ default behavior. While there are many examples of performance gains in various regions around the world, the Global Accelerator is not a one-size-fits-all solution, as there are several examples where the Internet actually performs faster and more reliably than Global Accelerator, or, the results are negligible. 
• Broadband ISP choice makes a difference in cloud performance. Businesses looking to get every performance edge possible should consider which broadband ISP provider they select, depending on which cloud they most heavily rely upon. There are performance gains and losses depending on which broadband provider businesses use to connect to each cloud.
• All cloud providers pay a performance toll when crossing the Great Firewall of China. Despite Alibaba’s origins in China, it experiences packet loss when crossing through China’s Great Firewall just as all the other cloud providers do, showing it does not get any preferential treatment. Enterprises serving customers in China, but hesitant to pick a hosting region in China due to stringent regulations on traffic and data privacy, might consider Hong Kong as a viable option - Alibaba Cloud traffic experienced the least packet loss from Hong Kong to China, followed by Azure and IBM.

Benchmark Findings By Cloud Provider: No Cloud is Created Equal

The inaugural Cloud Performance Benchmark published in November of 2018 benchmarked the three biggest cloud providers (by market share) AWS, Azure and GCP. Alibaba Cloud and IBM Cloud are new this year, meaning they have no prior comparison points. There are notable differences between each cloud in terms of overall performance and connectivity architectures. Key findings, by cloud provider include:

• AWS generally demonstrates low latency and made some minor improvements over the last year. It’s performance predictability metrics, however, noticeably improved, with its most dramatic change in Asia, which showed a 42 percent reduction in variability over last year. However, when compared to Azure and GCP, it still has lower performance predictability due to its extensive reliance on the Internet rather than leveraging its own backbone for delivery.
• Azure continues its strong network performance based on aggressive use of its own backbone to carry user traffic to cloud hosting regions. Standout changes from 2018 to 2019 include on average a 50 percent improvement in performance predictability in Sydney, whereas in India, there was a 31 percent decrease in performance predictability. Despite a slight decrease year over year, Azure continues to lead in performance predictability in Asia when compared to the other cloud providers.
• Google Cloud continues to favor use of its own backbone for user to cloud hosting region traffic delivery and has strong performance in most regions, but it still has some significant global gaps that haven’t been addressed since last year’s report. Traffic from Europe and Africa takes 2.5-3 times longer to get to India, going around the rest of the world instead of taking a direct route. Google Cloud also decreased visibility into their internal network, making it harder for its users to understand its network paths and performance.    
• Alibaba Cloud offers comparable performance to other cloud providers. It closely resembles AWS in terms of connectivity patterns, region locations, and even region naming constructs. Like AWS, Alibaba Cloud leans heavily on the Internet rather than their own private network backbone for the majority of user traffic transport. Uniquely, inter-region traffic between Alibaba Cloud regions is not contained within their own cloud infrastructure as it is for the other four, instead it exits Alibaba Cloud, traverses the Internet and then makes its way back into Alibaba Cloud.
• IBM Cloud performance is comparable to the major players. IBM takes a hybrid approach to traffic delivery, fluctuating between using its own private backbone and the public Internet, depending on which regions user traffic is accessing.

Archana Kesavan, research author and director of product marketing at ThousandEyes said:

"When businesses need to decide which cloud provider best meets their needs, one metric that's notably missing from their assessments has been performance data, mainly because it’s never been available or has, at best, been myopic. The second annual Cloud Performance Benchmark gives businesses that comparative data. Understanding cloud performance is essential for planning and for ongoing measurement so you can be assured that you're providing customers and employees with the best possible performance." 

ThousandEyes is an enterprise software platform that measures how internet, cloud and other third-party dependencies impact end user digital experiences. The ThousandEyes platform was leveraged to produce this research, measuring network performance to global cloud regions and within cloud regions. Results were derived from an analysis of over 320 million data points collected from 98 global metro locations over the course of 30 days, using both end-to-end performance and in-depth path measurement techniques. Organizations should consult the entire report to find the regional and cloud-provider specific metrics relevant to them.