Archive for November, 2012

From the comments of my previous posts, I noted that it is not the basic process, but the seemingly complicated methods and techniques that stem people from “loving” market research. So, I decided to begin a series “Market Research Methods 101” to introduce market research techniques and methods I learned and practiced. Hopefully, I can find a simple and entertaining way to help you at least have a knowledge of the common market research methods, if you have not already become a fan or even an expert.

With respect to primary market research, generally, there are two kinds of methods: qualitative and quantitative. Qualitative market research methods include focus groups, in-depth interviews, content analysis, ethnography, evaluation and semiotics, etc. While quantitative methods are related to statistics, all the way from percentages, multiple regression to factor analysis. The graphic from the article Qualitative Vs. Quantitative Research- When to use which can serve as a cheat sheet when we need to make the decision of which type of methods to use for a particular research project.

Before diving into specific methods, which I plan to do in my future posts of this series, I’d like to first clarify some misunderstanding of qualitative and quantitative research methods.

1. In the context of market research, though these two categories possess different approaches, they share the same ultimate goal: exploring issues, understanding causes and effects, and finally, solving the problems.  When you hesitate about which kind of methods to use, always remember to go back to the origin and ask yourself: What problem(s) I would like to solve? What are my research questions? And what methods are practical to apply?

2.  Whether data is involved is not the attribute to differentiate qualitative and quantitative methods. Both of the two categories use data for measurement. However, qualitative methods reply on unstructured data while quantitative methods prefer structured data. The essential difference between qual. & quan. is generalization, meaning whether the research findings are representative and projectable to the general population. Basically, qualitative research is on a case-by-case base, so the results are always limited to specific contexts; while quantitative research mostly aims at identifying a model or even a theory based on great amount of data, therefore, its results can be utilized in a broader context.

3. There are no good or bad methods. There are no so-called “best” methods. There are only most appropriate methods. In my opinion, the debate of which kind of methods surpass the other is meaningless. Going back to the graphic above, we can find that these two types of methods both have expertise that its counterpart doesn’t have. In industry, it is rare that a market research project sole relies on one type of methods.

For example, assume you want to research people’s attitude about a new ice-cream flavor that Ben&Jerry plan to launch shortly. First of all, you may want to hold focus groups to hear about people’s opinion about ice-cream generally as well as their specific opinion on this new flavor. From the focus group, you might find out what worked and what didn’t work, and then you wonder if it is the same case in the general public, especially Ben&Jerry’s customers. That’s where a survey comes in. Based on the findins from the focus group, we can conduct a survey among a large group of people (usually above 500) to gather data and information for quantitative analysis. From the analysis, you may find specific patterns of people’s ice-cream consumption, but you might not understand the reason behind some of the patterns. At this stage, a series of post focus groups can help you probe those specific questions and concerns. For the sake of fun, I call this kind of approach “hamburger research approach” (pre-qualitative+quantitative+post-qualitative). It is quite common in the real market research industry because it enables business decision makers to gain maximum consumer insights and reduce the uncertainty to minimum extent.

Hope you find this introduction helpful. If you have specific market research methods that you’d like me to discuss in my future posts, please feel free to let me know!


The series of Market Research Methods 101 continues! Today we are going to discuss another basic but crucial concept: Reliability.

Before we head to today’s theme, let’s review the relevant concept, validity, which I introduced in my last post. In the context of market research, validity means that when you want to measure a specific topic or idea, you are exactly measuring what you are supposed to measure and do not include anything else. Going back to my favorite weight scale example, if you are 100% sure your bathroom weight scale can accurately shows your weight, so we can say the scale is valid. Then how can the scale example shed light on what reliability is? Here we go! Let’s assume you keep the good habit of measuring your weight 3 times a day by your bathroom weight scale. In the morning, the weight shows that you are 100 pounds; at noon, it says 80 pounds; in the evening, you stand on the scale the third time and this time you found your “weight” is 120 pounds. If so, do you think your bathroom weight scale is reliable?

Of course not. We know that, regularly, it’s impossible for one’s weight to fluctuate dramatically within a day. Therefore, the three weight giving by the scale should be consistent. If not, then we can tell it is lack of reliability. With respect to market research methods, since currently most of market research still relies on the self-report approach and we all know human beings are extremely complicated and sometimes capricious (well, it hurts but it’s true), it is especially important to evaluate whether the answers to a survey by a respondent is reliable. Well, the core question is, HOW?

Actually, the answer is already in the weight scale example. An easy way to identify lack of reliability is to see whether a respondent’s reply to the same question is consistent. When designing a survey, for a specific concept that we would like to measure, we can intentionally set multiple-item measures at different places in a questionnaire. Therefore, after we collect all the data and information, we are able to see the consistency of a respondents in terms of a specific idea. However, when design multiple-item measures, you don’t want to simply repeat the same question with different words. Remember, human beings are not as constant as rocks, every single detail on a survey will influence their minds, which may prevent them from giving real answer to a question. Imagine you are answering a questionnaire asking you the same questions three times in different words, how do you feel? Probably not very happy. You might think it’s a waste of your time and may start to make up answers or even stop taking the survey.

The ideal form of multiple-item measures is to have 3-5 questions asking about the same topic without the respondents’ awareness that those questions are asking the same topic. To be honest, it is extremely hard to achieve. Nevertheless, we do have techniques and tips to make it happen. In my future posts, I’ll introduce those guidelines little by little. If you are interested in this topic, please follow or RSS my blog so you can receive my updates in the first time.

Do you understand what reliability is after reading this post? Now we know both what validity and reliability are. Can you tell the difference between them? The target picture below can help you review and  differentiate the two concepts.

In the next post, I’m going to introduce the last but not lease basic measurement concept that I think is crucial, level of measurement. It’s coming soon!

Plan to develop your career in market research industry? Interested in what your future bosses are looking for when hiring entry-level market analysts? You will like this post.

In my market research class last week, we had a pannel with three experienced market research professionals: Allison Drummond, Market Insights & Analytics Manager at Liberty Mutual Group; Sharon Quigley, Sr. Manager Customer Insights at Vistaprint; and John Zarrella,President of Research Results, Inc. They kindly accepted my request for a short video interview. So we have a video clip for today’s post! I have to admit that I’m a bad photographer, but the content provided by the three market research experts are really inspiring and valuable! So, check it out!