>if executives want a simple metric that is better than NPS, a random number generator is the fastest and cheapest way to achieve it
>In fact, NPS measures nothing in particular.
These types of sweeping statements aren't a helpful way for the author to advocate his broader point.
The strengths of the article lie in the more detailed points which bring to light some great examples of how NPS, and surveys in general are mis-used (and some examples that don't actually manifest themselves as real problems in daily use very often).
My thoughts on the examples chosen above:
>50% or more of their data filled with arbitrary numbers
I'm not sure what this refers to, but the way the NPS equation works, every respondent's score matters and mathematically impacts the overall NPS (each one feeds in as either a promoter, neutral, or detractor). Some of the author's own recommended questions include only 3 answer inputs also.
>Not collected from state-licensed specialists
Almost all operational data collected by companies for management reporting is not collected by state-licensed specialists, but is still useful.
>Easily gamed
All survey questions could be gamed in the ways similar to the article examples. A good executive will make sure the survey is asked in the same way of his own organization as of those of peers, and in the same manner over time. He or she won't let agents do things like cherry pick which customers to survey, otherwise the money he invests in the survey wont' actually help him run his company.
>Shareholders should insist the numbers be corroborated by a neutral third party
All survey questions can be tainted by preceding questions. When writing a survey it is fairly straightforward to A/B test the order to make sure this isn't a major factor.
These types of sweeping statements aren't a helpful way for the author to advocate his broader point.
The strengths of the article lie in the more detailed points which bring to light some great examples of how NPS, and surveys in general are mis-used (and some examples that don't actually manifest themselves as real problems in daily use very often).
My thoughts on the examples chosen above:
>50% or more of their data filled with arbitrary numbers
I'm not sure what this refers to, but the way the NPS equation works, every respondent's score matters and mathematically impacts the overall NPS (each one feeds in as either a promoter, neutral, or detractor). Some of the author's own recommended questions include only 3 answer inputs also.
>Not collected from state-licensed specialists
Almost all operational data collected by companies for management reporting is not collected by state-licensed specialists, but is still useful.
>Easily gamed
All survey questions could be gamed in the ways similar to the article examples. A good executive will make sure the survey is asked in the same way of his own organization as of those of peers, and in the same manner over time. He or she won't let agents do things like cherry pick which customers to survey, otherwise the money he invests in the survey wont' actually help him run his company.
>Shareholders should insist the numbers be corroborated by a neutral third party
Correct, companies who are serious about measuring NPS accurately do often hire third parties to run surveys (see example of JD Power running NPS: http://www.jdpower.com/press-releases/bain-certified-net-pro...)
>NPS recommendation question taints followup questions
All survey questions can be tainted by preceding questions. When writing a survey it is fairly straightforward to A/B test the order to make sure this isn't a major factor.