When Yes and No are Not Enough

Imagine a world where all the people inhabiting it viewed everything as “black-or-white”.  A person is very satisfied with life until, at some point, they became instantly very unsatisfied.  Children in school either have no knowledge over a subject or they know everything about it.  And neighborhoods are either so safe that the residents have never even heard of a lock before or are so dangerous that armed guards must be hired each time a trip to the store is needed.  Indeed this would be a strange world to live in, so it may be surprising that often times the way we collect data makes it appear that we live in this sort of world.

Data of this type is called “binary” meaning only a yes or no type answer is recorded.  While this sort of data is useful for questions like: “Do you live in Kansas?”, “Did you vote for Bayes last election?”, or “Are you male or female?”, it is not as appropriate when there are varying degrees for the answer such as; “Where you satisfied with the seminar?”.  Instead we should use a rating scale when we want to collect this type of data.  Let's take a look at an example to see why this is.

Say we are presenting a seminar on technology and we want to assess whether the participants have learned about using Excel.  We will ask the people taking the class two questions before and after the seminar to measure the amount they learned.  The questions will be “Could you use Excel if needed?” and “On a scale of 1-10 rate how well could you use Excel if needed? (10 being better)”.  After the seminar we take a look at our results (it seems the seminar turnout was poor since there was only three participants).

Now looking at just the yes/no responses we may be disheartened -only one person improved from a No to a Yes, and it also seems that the class wasn't useful to another since they already could use Excel.  However, looking at the rating scale we could see that in actuality the seminar was a success, all of the participants learned, with an average learning of 3 on the scale.  The seminar was the same, yet just because of the way we collected our data our conclusions could be quite different.

So why is a rating scale better for data like this?  It is because it allows us to measure slight changes in the data easier.  It is relatively difficult to switch a response from a No to a Yes, but much easier to move a rating up or down one or two points.  Further, we could always convert our rating scale data back to a binary type by grouping the responses (say 1-5 = No and 6-10 = Yes), but we can not turn binary data into rating data.

When designing surveys or intake data, take a minute to think about what kind of data you are collecting, and consider if a rating scale can reasonably be used.  The extra “shades of gray” that you will find in your data will make it easier to keep you pointed towards the truth.