Your cart is currently empty!
Human Metrics in Six Sigma
Six Sigma teaches us to view everything as a process. We should take an objective look at the system, measure the values, form a model, enact changes on the process, and observe the effects of these changes. However, we sometimes want to measure objectively something that is intrinsically subjective, the opinions of people. These fall under the category of what I like to call “Human Metrics.” Some common examples of these metrics are customer satisfaction, perception of quality, and ease of use.
How Do We Accurately Measure These Values?
Surveys are regularly used as the catchall for human metrics. This has several flaws, though. First, people are not consistent graders, and a 7/10 for one person may be an 8/10 for another. Ultimately, though, this is not a problem because it represents the same sort of variation you will see in any measurement.
The second problem is self-selection of responders. This is well documented, and can create extreme disparities between real and actual numbers. A good example of this can be found with call-in surveys. In many cases, only those with a chip on their shoulder will be compelled to call, creating a clearly biased sample. To counter this, sometimes incentives are offered to get a higher response rate, but this brings us to the third problem with surveys, non-response. Where some people may choose to simply not respond to a survey, others will purposely subvert the survey itself by ignoring the questions and answering in some sort of pattern. This is common with incentives, like contests for surveys, because it is so easy to simply hit the number one repeatedly without hearing the questions being asked. It can be hard to pick out this group, and one can only hope that over the long term the collective contributions of these non-responders will balance each other out.
How Do You Deal With These Problems?
A common solution is to try to address and eliminate the issues inherent to surveys. Select the sample by hand, normalize the scores across respondents, and remove surveys with obvious patterns or very unusual scoring habits. Ultimately though, even if these methods were perfect at eliminating their respective flaws, you would still be left with the fact that people are bad judges of their own opinions. Thus, you cannot use the responses to predict future behavior. This brings us to the best method for retrieving human metrics, using a proxy.
Measurement by Proxy
Measurement by proxy is a method that has existed for millennia. If you know the angle of the sun, you can measure the height of a flagpole by measuring the length of its shadow. You can apply this to human metrics and find values from behaviors that reflect certain opinions. Often, these are the values you care about most. Repeat sales, for example, is a good measurement of customer satisfaction. However, while repeat sales a metric that you can find using existing data, some require testing. For example, if you want a metric for ease of use, then you can look at the time taken to use the product. To test this, one might take a group of people with little or no familiarity with a product and ask them to use it. The time taken does not exactly demonstrate ease of use, but the two are related enough to make this a reasonable proxy in certain situations.
That is not to say that surveys cannot be useful, or don’t have a place. Simply put, they are very easy to implement, and consistent surveys allow the tracking of trends in user opinions. Additionally, measurement by proxy is far from perfect. Other factors in your system can seep into your measurement and taint the values. For example, the number of repeat sales may be artificially increased by having a product with a short lifespan. Naturally, the short lifespan is bad for customer satisfaction, but the proxy would insist otherwise.
Conclusion,
As a Six Sigma expert, you should look to be quantitative in your assessment of systems. Human metrics are no exception to this rule. You cannot abandon the Six Sigma philosophy just because “Everyone is different.” Instead, look to measure how they are different.
2 responses to “Human Metrics in Six Sigma”
-
Another concern with survey taking as a measurement system lies in cultural norms. While some cultures encourage complaints and indications of problems (Anglo-American cultures do this well in my experience), others do just the opposite and encourage highlighting of the positive and a distinct lack of comment on the negative (Some Asian cultures are good examples of this).
Don’t make the mistake that national or racial cultures are the scope to be aware of, either; company or location-based culture can have this effect (the small office that just “goes along” with all the decisions of the parent company no mater how painful, or the office that willfully disregards edicts from above are two typical examples).
As such, surveys in contained and defined populations are usable as you describe; cross-cultural populations, however, require some cultural sensitivity – even if the survey technique is made as objective as possible.
-
Another approach to consider if you are a supplier of raw materials, tooling or similar:
1) Define a customer population of interest. An example is “all customers that buy our PCD wire drawing dies”.
2) Randomly select a sample of said population
3) Send someone with an open mind to visit the customer, but don’t ask for input from Purchasing people. Instead, talk to the people doing the actual work with your product (material handlers, machine operators, etc). They’ll often give you the straight story. Ask for their honest opinion and assure them that you’ll fix any problems.
4) Beware, some Purchasing folks will try to hide problems fearing their solution will result in a price increase. Try to talk to people at Gemba without Purchasing or Management people hovering about. Use diversionary tactics, if needed. 🙂
5) Listen carefully, be sure you understand properly.
6) Buy the floor workers the best local pizza pies you can find. Avoid chain pizza.Then make inferences on the entire population from your sample. Follow up on any problematic areas discovered. Repeat with a new sample in 6-12 months or thereabouts.
It’s not a perfect approach, but by demonstrating your desire to help and getting the story straight from the at-Gemba workers, you can often get useful feedback.
Leave a Reply