By Lloyd Bloom, Senior Product Manager, Compuware
Page Delivery Performance Risks in the Cloud
Compuware recently published its first annual “Cloud Provider Global Performance Ranking – 12 Month Average.” Where do these rankings come from, what do the numbers mean, and how can you use them?
To put these questions in context, it is helpful to emphasize the importance of measuring performance from an end-user perspective. Availability, portability and data security have long dominated the discussion around cloud computing risks. But end-user performance – the speed by which cloud-based applications are delivered to end users around the world – represents another significant and potentially costly threat.
Specifically, as page load times increase, so does the rate at which end users abandon pages. Recent studies like the one below illustrate how a poor end-user experience has a direct impact on revenue and ROI. A page load time of just six seconds results in a page abandonment rate of about 20 percent. If the page in question is a shopping cart checkout, then you will be losing 20 percent of your customers – and your revenue – simply because of the page load time.
The End User’s Perspective Is the Only One That Matters
When application owners and operations teams think about application “performance,” they often focus exclusively on the health of applications and how transactions execute in their multi-tiered back-end infrastructures. The implicit expectation is that if the application performs well in the data center, all is well. This is as true for cloud-hosted applications as it has been for more traditional data center-hosted applications.
End users know better. Although they don’t know anything about back-end performance, they are closet experts on end-user performance. Without understanding (or caring about) the impact on your business, they are making decisions to continue with their transactions, or not, based upon their subjective understanding of performance. When all lights are green at the cloud service provider’s facility, end users can still find page delivery times way too slow, and … click … “let’s see what a competing site can do for me.”
According to the Aberdeen Group, application performance issues — including internal app performance — can hurt overall corporate revenues by up to nine percent (Aberdeen Group, “Application Performance Management: The Lifecycle Approach Brings IT and Business Together.”) This brings us back to the question of the origin, meaning, and usability of the Provider Rankings.
Understanding Performance Delivered Over the Internet
Why does the end user see performance differently than the cloud service provider? A number of factors come into play, but they largely boil down to “distance.” This can be the physical distance between an application end user at the internet’s “edge” and the cloud host. The longer the distance, the more the delay (latency) in page delivery. But, even when geographic distance is short, “network distance,” or the effect of routing conversations between clients and servers, can add to the geographic distance with potentially significant performance impact.
The high-speed backbone links from the cloud service provider to the internet backbone data centers, and the “last mile” connections via local ISPs from the backbone to the end user are generally well-optimized, but exceptions do occur that allow network distance to introduce delays that kill application responsiveness. For example, we have seen instances where network connections from Singapore to Tokyo have been routed through San Francisco, tripling page delivery times.
When looking at performance of cloud applications, it’s important to understand the role of the various players that impact performance. The cloud service provider’s key roles are to provide an application execution environment (servers, load-balancing, databases, storage, etc.) and connectivity to the internet. The application owner then deploys an application and is responsible for having contracted sufficient application environment from the provider. The application design and selection of infrastructure are the key elements that impact the ability of a site to create web pages in a timely manner, assuming that the infrastructure meets its performance SLAs.
Once the application does its work, the network takes over and page delivery time becomes critical. Poor routing along the backbone network or congested last mile ISP links can negate the best attention to application design or cloud data center infrastructure. As an application owner, it is difficult to have influence over which ISPs your users use. But, you can search for cloud service providers that have the best level of page delivery times over the backbone network to the geographies of your most important target end users.
Compuware’s Cloud Provider Global Performance Ranking was created to highlight the importance of the relationship between the backbone network and cloud service providers in effectively delivering pages to end users. The measurements come from Global Provider View, an application hosted on Compuware’s CloudSleuth.net, a community portal for learning about cloud players and performance in the cloud.
In truth, a global 12-month average has a limited role in making practical decisions. Rolling up that much data into a few numbers tends to mask important realities. So, look at these global numbers with a grain of salt and be prepared to dive deeper, because the data behind these numbers is really useful.
If you were to open the Global Provider View (go ahead, you’ll have fun following along), you’ll see a map of the region that you are sitting in (Oceania, North America, etc.) and a set of dots that represent backbone data centers. From these locations, Compuware measures the response time of a two-page reference application hosted in 30 cloud providers’ sites. The color of each dot indicates the best possible page delivery time from the cloud providers. Click on a dot, and you’ll see a list of cloud providers, ordered by their performance. Use these performance numbers to identify the cloud providers with the best page delivery times for the region of interest.
Summary: Don’t ignore your cloud provider’s network performance
The cloud offers distinct advantages in many aspects of application deployment, administration, and management, but it is not a panacea for performance. There is no guarantee that end-user experience will be better when delivered from the cloud than from the enterprise data center. Rather, the performance problem is roughly the same, and has to be addressed with SLAs focused on the key measure of end-user performance. And to achieve great end-user experiences, application owners have to make performance-based decisions at all levels of the end-to-end application delivery chain, not just in the data center. The Global Provider View targets a key part of that path, the backbone network.
Lloyd Bloom, a senior product manager at Compuware, has worked in hardware and software development for the past 26 years. Recognizing the difficulty in delivering well-performing applications, he joined Compuware in 1995 to work directly on APM challenges. Lloyd started as an APM field consultant, then joined the APM product management team, where he has focused on end-user and network performance for the past 11 years. Most recently, Lloyd assumed leadership of CloudSleuth, Compuware’s partner-driven community that educates users about performance in the cloud.
