Market Research Methods 101: What is Validity?

Posted: March 8, 2013 in Market Research Methods 101
Tags: , , , ,

This is the second post of my new series: Market Research Methods 101. From now on in this series, I’ll introduce specific market research method as well as some relevant examples and cases I’ve done. Across the methods I’ll discuss,there are some general terms representing the basic, logic and magic of scientific market research methods.They can help you make sure your research is techniquely correct. The three crucial and most-mentioned terms are Validity, Reliability and Level of Measurement. They will be the topic of my next three posts in Market Research Methods 101. Today, let’s begin with Validity.

Instead of displaying various definitions, I’d like to share with you a simple but inspiring example that helped me understand what validity is. You’ll find this example quite engaging because it is an issue that almost all the girls (possibly also guys) care about in their entire life: Weight!

Weight Scale Example helps you understand what Validity is!

Let’s assume (well, probably it is true for you : -b) that you measure your weight 3 times a day by a bathroom weight scale. In the morning, the weight shows that you are 100 pounds (Hooray!). At noon, you weigh yourself again with the same scale: still about 100 pounds (plus or minus 2 or 3 depending on whether you do it before or after lunch). In the evening, you stand on the scale the third time and again you find yourself about 100 pounds. Now you may be pretty sure your REAL weight is 100 pounds. But, wait a minute, are you 100% right? Probably not. What if you go to a gym and weigh yourself with another scale and find you are 110 pounds? Are you still sure about you are 90 pounds? I guess you might want to find a third scale to measure your weight again. If it tells you 110, you may start to question your own bathroom weight scale and think there might be  something wrong with it.

So, how does this relevant to validity? In market research, when we want to measure or evaluate some issue or topic, a common mistake is that researchers may simply assume the measurement they use (e.g. the questions in the survey, interview or focus group) can indeed measure the issue or topic they are supposed to measure.  Back to the weight scale example, it’s just like you assume that your bathroom weight scale can accurately measure your real weight at first. However, you didn’t check if the number shown by the scale is actually “your weight”, not something else. This “dangerous” assumption, most often without our awareness, led you to a wrong conclusion about what your weight is. This kind of situation happened in market research, too. Say, we would like to research students’ satisfaction of a professor in a course. If we simply reply on the change of how many students chose this course, we probably won’t get an accurate evaluation. What if the course is a required one and only one session is available for the students? Then though the professor is not doing a good job, there will still be a lot of students register for it. A more appropriate approach to measure students’ satisfaction might be directly ask their attitudes and opinions toward the course and the professor’s performance.

It might be correct, but not valid.

Now do you understand what validity is? To summarize, in the context of market research, validity is to which a test measures what it claims to measure (the real issue or topic). It means that the items on the test represent the entire range of possible items the test should cover and it covers nothing other than those. It is crucial since you can consider it as the essence of a research project. If you would like  to go east but are actually running to the west, then how can you finally get to your destination?

I hope my weight scale example help you gain a good understanding of validity. If you find it interesting and want to know more, please follow my following post. The weight scale will continue and I’ll talk about Reliability in the next post!

Advertisements
Comments
  1. Cynthia says:

    The weigh scale example is interesting, lol! I am looking forward your next post regarding the reliability and I hope I will be able to know how to tell which scale is the accurate one then.)

  2. LI Mengyu says:

    i like the discussion about measurements. Bad tests leads to useless founding

    p.s. in my area we usually argue the validity of measurement by referring a lot famous works which also used the same measurement…….orz, seems a bit unreasonable.

    • Hi Mengyu, thanks for your comment. I agree with you. Simply repeating the “authoritative” measurement doesn’t make any sense to me.

      In the market research field, each topic or case has its unique attributes or premises, so the measurement should always be “customized”. I had a class all about measurement pretest at BU. Typically, we use Chronbach’s alfa to test reliability and factor analysis for validity test.

      I guess in the world of computer science, you might have different approaches for measurement test, right?

  3. Xinyan says:

    Hi, Yajie, finally I found your blog~
    The example you use here warmly reminds me of the Prof.’s lectures~ I think it is really very smart of you to use it to explain the difference between validity and reliability!
    Guess I will say “I want more” as my closing~

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s