Which ranking of order you prefer?
We are creating a survey questionnaire. One of the questions is to indicate the order of preference (strength of belief) about what intervention works (“Indicate the order in which you believe the students benefit from the following list of six interventions”).
There is some order already inherited or known or researched, but we want to see whether the experts feel the same.
The question is, what system for ordering to use. We came up with two: from 1 to 6 (lowest preference/no benefit to highest/best benefit) or from 1 to 6 (meaning first best – worst, last best). These two schema are opposite, so getting ’1′ in the first is the same as getting ’6′ in the second.
Which one do you feel is more intuitive? Do you use some other way of asking the order?
My first thought is that 6 should be for the highest/best benefit if it is for a scale assessing the strength of a feeling or a preference. If it is for a scale where people need to rank the order of importance of different items, then I think 1 should be for the most important/highest benefit item. But I’m curious what others think too?
Of course, you can always use words in a Likert-style question instead of numbers: e.g. strongly agree, agree, neutral, disagree, strongly disagree, (does not apply).
Here’s one online resource that seems helpful for designing data-collection instruments: http://www.idrc.ca/en/ev-28258-201-1-DO_TOPIC.html
It seems to be about order, so I would definitely say 1 for the most important. At least for me, the opposite would seem rather strange. Unless the question was something like “Grade the usefulness of different interventions (from 1 to 5)”.
I’m with Clint. 6. My experience is that scales usually have the greatest number associated with the highest degree of something on a survey or if nominal categories are used, the most positive is on the right-most side of the scale . There has been a lot of research about the effects of different types of scales and their effects on survey response. I don’t have it now, but I can find a reference for you if you are interested.
Also, a number suggests that you agree, but then you quantify agreement. The word system (e.g., from strongly disagree to strongly agree) let’s you disagree. In some cases, I have seen numbers used in the actually survey to use space, but the numbers are associated with a nominal key (e..g, 1 = strongly disagree, 2 = disagree, . . . )
Just guessing,
Jus
I meant, “the survey to SAVE space, but . . . .”
However hard we tried, saying “Use each number only once”, we still got a few of answers of the format: “3,4,2,3,4″. That’s what some call 1 to 6.