Welcome Login

For current information from GSA about COVID19 please click HERE

You are here

Cloud Service Evaluation Series, Feature # 1 - Agility Metrics


Metrics Categories


  • Accountability

   ⇨  Agiliity

  • Assurance
  • Financial
  • Performance
  • Security and Privacy
  • Usability


Elasticity, Scalability, Flexibility & Portability 

The primary feature of cloud computing is its ability to quickly spin up or down based on resource needs, constantly responding to customer demand. Two Agility Metrics are at play here:

  • Elasticity, which relates how quickly a resource can be added at any given time, and
  • Scalability, which refers to how much of a resource can be added at any given time.

The Federal Government uses the five-pronged NIST definition of Cloud found in SP 800-145. Two of the five aspects are applicable here: 

  • On-demand self-service.

A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider.

  • Rapid elasticity

Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time.


So what constitutes ‘rapid’ elasticity? There are two types: Horizontal and Vertical.

  • Horizontal elasticity measures the speed at which more of the same resource can be provisioned. 
  • Vertical elasticity measures how quickly a new type of resource can be provisioned. 

Generally, we are concerned with horizontal elasticity, as it is the easier of the two. 

So, given this, ask the following questions:

  • How important is elasticity to my application in the cloud? Will I need a high elasticity due to regular fluctuations in resource need? Is low elasticity satisfactory because my known user base has a steady workload?
  • Is my goal cost savings? Can I use the public cloud to maximize savings if I need high elasticity? Is the automation in place so that when resources are no longer required, they can be rapidly and automatically released to save money?

Your answers affect how you’ll weigh the Elasticity metric. For all his metrics, Feuless uses a 0-10 scale. He suggests testing the service provider’s elasticity directly if at all possible. If not, the provider should detail elasticity measurements in a proposal provided to the agency. Consult the people who develop applications going into the cloud about its relative importance.


The Scalability metric, just like the Elasticity metric, has horizontal and vertical variations as well. 

  • Vertical scalability is how much of a new type of resource can be provisioned before capacity is reached;
  • Horizontal scalability measures how much of the same type of resource can be provisioned before capacity is reached. 

The scalability metric must assume that service performance remains satisfactory, costs remain in an acceptable range to the customer while scaling, and Service Level Agreements (SLAs) aren’t reduced while scaling. Where any of these assumptions erode is the point of maximum scalability. 

Feuless recommends executing real-world capacity tests if at all possible. If not, third parties and customer testimonials will have to attest to the cloud service provider’s scalability. Discuss with the technical team that deals with the scalability requirements of the cloud-ready or in-development applications to appropriately weight the scalability metric.


Flexibility references the capacity of a CSP to alter the functionality to match customers’ needs. The more service options a CSP offers, the more flexibility it has. A service option menu includes everything from compute instances and storage types to operating system versions and SaaS user types.

Customers’ needs are bound to change over time. While it is best to anticipate future needs and judge a CSP’s value based on them, this is often impossible. Hedge your bets as a cloud customer and select a CSP that offers more rather than less. 

Start with what features and functions you know you want now. Are configurations easily shifted and manipulated? How long and expensive are they? Are there other barriers that need to be negotiated? Secondarily, what will your organization want and need in the near future? Plan for, as much as possible, more long-term desires. 


Portability refers to the level of ease a customer can transfer their ecosystems, data, applications, and more from one Cloud Service Provider to another. The two relevant types of portability that we are concerned with are Technical Portability and Legal Portability. 

Technical Portability 

CSP’s often use different technologies to implement their cloud service, so it makes sense that it would be difficult to easily move from one to another seamlessly. Programming interfaces, templates that control cloud resources, and underlying code are just a few examples of such incompatibility.

Some CSP’s make it easy to upload your migrating non-cloud applications, but difficult to move anywhere else.

Beware ‘vendor lock-in’ when evaluating a CSP. Feuless emphatically states here that “portability is your insurance against negative experiences with service providers [as well as] changes in your own organization.” 

Score the technical portability metric by first ballparking the number of CSP’s that offer acceptably compatible environments to the CSP being evaluated. Invite technical staff to help critique CSP’s for portability, as they deal with application and data requirements for these cloud applications on a regular basis.

Ask questions like:

  • How usable and complete will the migrated data be? How much will you need to ask of your developers to ensure application compatibility?
  • How long would it take to move everything elsewhere? How expensive?

Legal Portability

On the legal side, portability is often limited by the terms and conditions that Cloud Service Providers present to their customers. As consumers of cloud services, you must explicitly protect yourself and shield your intellectual property and data.

If your organization would like to switch providers, is your data going to be immediately returned to you? Will the CSP drag the migration process out or will they be offering active assistance? Require written clauses in your contract with your provider that your organization retains sole ownership of your data and that should you decide to move, that they’ll help identify and move all assets off their network.

It is often tempting to engage with a cloud service provider for the long-term. CSP’s offer lower prices to those organizations willing to commit to using their infrastructure for longer periods. As said in the technical portability section, don’t sacrifice portability to save money unless you’ve adequately discussed and considered the length of time you’re willing to commit to a CSP. 

Technically savvy legal counsel is indispensable to help resolve any issues that may arise with this aspect of cloud service evaluation. Counsel should help explain any data ownership or transition assistance needs that should be made explicit, and extensively review relevant sections of the actual contract.


Agility metrics allow agencies leeway in their cloud acquisition strategy, giving them wiggle room when their technological, functional or acquisition needs inevitably change.

In our next feature, Performance metrics of cloud computing services will be covered. We will answer questions such as:

  1. How do you rank your different functionality needs when choosing a cloud service provider?
  2. How do you prioritize the functionality you are acquiring against the relative ease or difficulty you’ll have in integrating it? How interoperable is this Cloud Service?
  3. How important is service response time to your agency’s cloud need?

Views: 1406

NEW.... the GSA Cloud Information Center hosted on the Federalist!  Get answers to your cloud questions! Federal government users can conduct market... More

This Information Hub:


  • Explains what Cloud Computing is and the benefits of federal cloud adoption 


  • Provides cloud computing best practices, guidance and acquisition templates 


  • Assists Government users conduct market research on cloud service providers


  • Supports the cloud adoption journeys of all Government agencies through direct GSA customer support!



For information about getting on GSA Schedules or general direct cloud support (for agencies or vendors), contact the GSA Cloud Acquisition Team at Cloudinfo@gsa.gov.
  • greg.price's picture
  • TParker's picture
  • sleigh1906's picture