Part 19 (1/2)

Final Product: The Metrics Report Card

Chapter 9 introduced the metrics I developed to answer the leadershi+p's question for our organization If only the service provider and the executive leader were to be viewers of the results, I could havethe orous review throughout the ement chain, and also be seen by customers, I had to find a way to make the results usable (if not readable) at each level

I worked with a team of consultants and the service providers to develop what beca the metrics allows for it to be viewed at different levels, and at different degrees of aggregation, while still preserving the concept of expectations

Now that basic measures were developed, we (the team of consultants- the service providers and me) needed a way to make them into a Report Card What we had was a set of charts But we needed to find a way to report on the overall service health, while still preserving the individual e, and Custo at an acadeical to use the Report Card concept as a template

In the case of a Report Card, students can take totally disparate courses-everything frorades obtained can be based on totally different evaluation criteria, but the grades are still understandable An Avery well; a B ; below a Cto meet expectations Quizzes, tests, research papers, and classroorade Other less normal evaluation methods can be used-like reviews of art produced, presentations to committees, and panel reviews of materials produced in the course of the class

In all cases, the student gets a letter grade that can be transferred to a nu sirasp iure out how to evaluate and deter Expectations, Meeting Expectations, or an Opportunity for Ie (of abandoned calls or abandoned calls less than 30 seconds) So we could smoothly transition this to a common measurement view

Delivery

Remember, for this discussion I've broken delivery into ”availability,” ”speed,” and ”accuracy”

Availability

For each measure, expectations have to be identified Table 10-1 shows the expectations for Availability

Thisthe rade” of O, M, or E This could then be used to develop a grade point average We could roll up the Grades at the Availability level We could also roll up the grade at the Delivery level (Availability, Speed, and Accuracy together) And hopefully ould be able to roll up the grade to an overall Service grade

The first decision we had to rades We opted to give each grade (E, M, or O) a value It was important to us to have our results be beyond reproach Since we knew errors ht seep in from many different quadrants, we had to ensure our intentions were never in question Trying to eable, if not simple, orked off a 10-point value scale An E orth 10 points; an M orth 5 points; and an O orth zero We then averaged the numerical values So if we had two values to use, as in the case of Availability ould get the following: E = 10, M = 5, O = 0

An E and an M averaged 75 points An E and an O averaged 5 points An M and an O averaged 25 points Noe took the calculated grade and turned it back into an evaluation against expectations A grade of 8 or greater would be an E A grade between 5 and 8 would be an M A grade beloould be an O So, another way to look at it is as follows: An E and an M averaged 75 points, which was an M An E and an O averaged 5 points, which was an M An M and an O averaged 25 points, which was an O We liked the way this worked You had to exceed in a measure to balance out an Opportunity for I on the side of excellence” as my friend Don would say

Err on the side of excellence

A possible drawback was itand the Exceeding Expectations a ”good” thing As I covered earlier, they are both anomalies What ant are Meets Expectations Another drawback was that the corades could hide anomalies If you had an equal number of Es and Os it would roll up to Ms and look like you were doing just anted- expectations

The positives were that we could show an overall grade, giving a ”feel” for the health of the iterades by si any Ms or Es that had Os buried in the data That would allow the metric customer to knohere to dive deeper to find the Os and see as happening in those cases

The bad vs good thing we could not overcome as easily In the end we decided to deal with it on a case by case basis, ensuring that we stressed that both were anomalies We accepted this because, noExpectations, in the end, the fact that expectations were exceeded wasn't in itself a bad thing It was only ”bad” based on how you achieved the grade-like if you neglected other important work/services or applied too rade by a process i properly on a different area, it was not only a good thing, but we could change the customers' expectations because ould be able to deliver at this higher level consistency It could beco point for our services over our competitors Opportunity for Improvement, the other anomaly, couldn't be said to be the same If you failed to meet expectations,the custo Even if you found that it was due to natural disasters or things out of your control, the custoative So while I wanted both anoation, the purpose of that investigation was clearly different

One investigation was essentially conducted to see if the occurrence could be avoided in the future, while the other was to see if it could be replicated

This led to the decision that ould roll up the values using the translation, and if there were Os below the level we shoe rade with an icon (Note: As we continually worked to improve our tools and processes, we adapted the icon when there was any anoure 10-1 is the Translation Grid we used to convert a grade to a letter grade and back again We originally colored the values as Green for Exceeds, Blue for Meets, and Red for an Opportunity This made it very hard to convince anyone that Opportunity for I investigation as an Exceeds Expectations (green)

Figure 10-1 Translation Grid So looking at the Availability charts we added the expectations, so that visually we could tell where ere in terms of health of availability This visual depiction happens at the rades with other rade for the category) and before we look to roll up grades into delivery Figures 10-2 and 10-3 show the abandoned rate and calls abandoned in less than 30 seconds, with expectations

Figure 10-2 Abandoned call rate with expectations

Figure 10-3 Percentage of calls abandoned in 30 seconds or less

Speed

Speed wasn't as simple as Availability We could measure how many cases were responded to (or resolved) faster than expected, within expectations, and slower than expected The probleood? We could have said any case that fell out of the Meets Expectations range (above or beloas an anoical, but since there were thousands of cases, it was not practical And when I interviewed the department, it was clear that anomalies would happen from time to tier to respond than expected And other ti This was a natural byproduct of the nature of the work and the environmental factors that influenced performance as well as workload

So for these cases, we decided to determine as expected by the custoe of cases does the custoe of cases does the custoe of cases does the customer feel is acceptable to fail to meet expectations?

So we looked to define the expectations in the forth of time to respond and the time to resolve

Time to Respond Exceed: Responds in less than 5 seconds Meets: Responds in 6 to 30 seconds Opportunity for Ireater than 30 seconds Time to Resolve Exceed: Resolved in one hour or less Meets: Resolved in 24 hours or less Opportunity for Improvement: Resolved in five days or less For each we needed to detere of the cases would the custoories listed, as shown in Table 10-2

Figure 10-4 shows percentage of cases resolved in less than one hour While this is a good et the full picture Looking at only this ive a skewed view of how healthy the service was (in tere of calls resolved in less than one hour This second level of expectations allowed us to use percentages, and allowed us to look at anonificant (as defined by the expectations) amount of cases It's hile to note that the third measure for Speed: Time to Respond, moves in the opposite direction of the other measures This will also be the case with rework, where less is better

Accuracy