Call centers are never short on metrics, but is the business utilizing the right metrics to ensure they are fit for business? Should those metrics change in an agile environment? The agile methodology for software development has definitely taken center stage, however, it struggles in large organizations with long standing processes and considerable investments in six sigma processes and methods to measure quality and defects. Some may attempt to re-brand the old-six-sigma methods into lean six sigma for agile ™ methods, but Almost all will be lost in the flood of metrics floating around in today’s contact center environment. Organizations need to focus on critical, business impacting metrics to determine if it is moving in the right direction.
Embracing agile requires more than new titles: it requires a culture change. I have already outlined these changes in a previous blog post titled, Top 10 Ways to be a Test Automation Hero. It highlights that “providing value to operations” is a key component. As such you’ll need operational data to guide the changes you are making in the software development processes.
So what metrics should you measure? I have compiled a list that has proven to be effective in determining if your contact center is fit for business. In working with numerous companies deploying a contact center assurance strategy, the ability to tie together operational numbers and QA numbers has been instrumental to show the benefits of an agile environment and developing with improved customer experience in mind. The top 10 metrics include:
10: Number of defects reported by operations/customers in the first 8 weeks after a major application change
This is not just a “how bad are we” report for the QA team. This really is about:
- How good is the operations team or your customers at finding our bugs
- Learning not only how they were found, but more importantly how they were “experienced”
This is important because it will help you track the next few metrics
What is the mean time to understand the defect in operations? It’s a subset of mean time to repair (MTTR).
8: Revenue/cost impact
What was the revenue impact based on those defects? (which contains many different sub-metrics)
In terms of customer satisfaction, access to purchase or upsell opportunities, and market conditionality, these should be weighted more heavily if the customer was in the first 90 days of onboarding. This revenue impact can be spun out in many different ways by accurately capturing the sub-metrics associated with the defect/outage.
7: What is your current self-service application development regression test coverage as percentage?
Some find this almost impossible to gauge because they don’t have a good way of measuring all the possible permutations of the application to begin with, especially in natural language applications.
Include “invalid and negative path” tests for re-prompting and timeouts; typically, we see customers with 5-15% coverage of their application before they implement Empirix Test Automation solutions to rapidly automate both test case generation as well as test scripting and execution. However after implementing the automated test case generation capabilities, they achieve more than 85% application coverage on positive and negative test paths
6: How long does a full regression test cycle take?
This indicates how fast you can deploy new builds into production and will set the tone for what your yearly release cycle may look like.
To find out numbers 5-1 and to understand the “WHY” for #10-6 above, Subscribe to my blog for the next installment:
Erik is Empirix’s Test automation expert, with over 15 years in the telecommunications data networking and call center industry, Erik has architected, coded, executed, and managed some of the world’s largest and most complex stress tests for the top global service providers and enterprises. His experience includes the synchronization of web, voice, email, and screen pop testing enabling customers to test real world conditions before they launch new platforms.