Posts Tagged ‘multiple-item measure’

The series of Market Research Methods 101 continues! Today we are going to discuss another basic but crucial concept: Reliability.

Before we head to today’s theme, let’s review the relevant concept, validity, which I introduced in my last post. In the context of market research, validity means that when you want to measure a specific topic or idea, you are exactly measuring what you are supposed to measure and do not include anything else. Going back to my favorite weight scale example, if you are 100% sure your bathroom weight scale can accurately shows your weight, so we can say the scale is valid. Then how can the scale example shed light on what reliability is? Here we go! Let’s assume you keep the good habit of measuring your weight 3 times a day by your bathroom weight scale. In the morning, the weight shows that you are 100 pounds; at noon, it says 80 pounds; in the evening, you stand on the scale the third time and this time you found your “weight” is 120 pounds. If so, do you think your bathroom weight scale is reliable?

Of course not. We know that, regularly, it’s impossible for one’s weight to fluctuate dramatically within a day. Therefore, the three weight giving by the scale should be consistent. If not, then we can tell it is lack of reliability. With respect to market research methods, since currently most of market research still relies on the self-report approach and we all know human beings are extremely complicated and sometimes capricious (well, it hurts but it’s true), it is especially important to evaluate whether the answers to a survey by a respondent is reliable. Well, the core question is, HOW?

Actually, the answer is already in the weight scale example. An easy way to identify lack of reliability is to see whether a respondent’s reply to the same question is consistent. When designing a survey, for a specific concept that we would like to measure, we can intentionally set multiple-item measures at different places in a questionnaire. Therefore, after we collect all the data and information, we are able to see the consistency of a respondents in terms of a specific idea. However, when design multiple-item measures, you don’t want to simply repeat the same question with different words. Remember, human beings are not as constant as rocks, every single detail on a survey will influence their minds, which may prevent them from giving real answer to a question. Imagine you are answering a questionnaire asking you the same questions three times in different words, how do you feel? Probably not very happy. You might think it’s a waste of your time and may start to make up answers or even stop taking the survey.

The ideal form of multiple-item measures is to have 3-5 questions asking about the same topic without the respondents’ awareness that those questions are asking the same topic. To be honest, it is extremely hard to achieve. Nevertheless, we do have techniques and tips to make it happen. In my future posts, I’ll introduce those guidelines little by little. If you are interested in this topic, please follow or RSS my blog so you can receive my updates in the first time.

Do you understand what reliability is after reading this post? Now we know both what validity and reliability are. Can you tell the difference between them? The target picture below can help you review and  differentiate the two concepts.

In the next post, I’m going to introduce the last but not lease basic measurement concept that I think is crucial, level of measurement. It’s coming soon!

Advertisements