Carbonview Insights

Market Research - A quick survey design tip

Great article on response substitution.  While there are some great research methods and designs regarding the appropriate research for the question at hand, sometimes it pays to also look at the simple things that could help in getting the most appropriate answers from your audience. 

http://adage.com/cmostrategy/article?article_id=145120

 

Online Market Research - Services and Technology…without outsourcing.

Oddly we have had quite a few inquiries lately about whether we utilize any outsourcing services.  We do not and for the foreseeable future, we will not.  We don’t have anything against those organizations who do, it’s just not in line with the value we give to our clients.  For that reason, we have decided to make it a little more formal by registering with ‘Offshoring Transparency’  to certify what we have been doing since 2005.

 

It’s fairly simple for us, our back office activities (invoicing, etc.)  are handled through our parent company.  So, the only activities left are those that we view as our primary offerings/services (i.e. research, programming, etc.).   Our clients come to us for our ability to assist them in their market research needs, outsourcing their requests to someone focused specifically on fulfilling the order doesn’t seem right to us.  We started Carbonview with a focus to creating a consultative approach to the ever evolving online methodologies.  Our staff makes it a priority to specifically understand that client’s definition of a successful project.  We have always viewed our custom solutions in understanding what combination will answer to this challenge regarding the people, process and technology.  Inevitably the ‘people’ factor becomes one of the most important areas.  That fact can’t just be the individuals speaking to the client, it’s their initial contact, the project manager, the programmer and most importantly the focused and intentional thinking about how to answer to that specific challenge; cohesive focused curiosity.  Outsourcing inherently breaks that chain.  That outsourcing agent is focused on one piece of the total puzzle.  As such, that one area will always look, feel, and act different from the cohesive solution that has been created aside it.   

 

In the few years we’ve been around, we have taken on many challenges that only succeeded due to this approach.  Many of our products/services are answers to challenges that we and our clients faced.   

 

That said, I applaud the growing concept of ‘transparency’ within our industry.  I don’t think we have an option to not be transparent.  All of our client’s trust us to provide them with information that is unbiased and properly designed and administered at minimum.  While both those terms deserve their own discussions, transparency of how we as researchers have allowed these principals to guide us seems to be common sense.

For those who are interested in understanding more about the Foundation for Transparency in Offshoring…

Respondent Engagement - Question Types

How is the current respondent engagement movement impacting and changing the normal question types?

Over the past 2 years, Carbonview has kept the usability of the survey one of our main points of distinction.   It seems somewhat basic to the outsider of market research that the page being displayed needs to communicate its objective clearly and in doing so be enjoyable, fun and easy to interact with.   The ‘respondent’ is truly the beginning to all of our services and since our payment to them can be as little as nothing and at the very least non-biasing we are selling the respondent on giving their time.  As such there is inherently a correlation to how much time you get from that respondent vs. their enjoyment of that survey instrument.  If that activity is analogous to filling out a form we are guaranteeing exhaustion in a number of minutes.  If the interaction is enjoyable and clear regarding the objective of the page (questions) it is only natural to assume higher engagement of the survey.

 

While the first days of the ‘respondent engagement’ movement focused on more stylized versions of the basic input (single response, multi-response, numeric, etc.) this has now given way to many new ‘question types’ not previously offered.  The benefits of these new question types are multi-faceted, but the basic improvements span from simply more engaging surveys to a more immersive instrument that allows for those opinions collected to be done so in a much more comprehensive way.

This entry discusses three of the many new types of ‘questions/format’ now accepted as inherently better than the form-input approach they replace. 

Qualitative One/One Chatting

Replacement of:  Online Chatting replaces larger initiatives within the same research.  Historically, there has been no merging of the qualitative and quantitative methodologies.

Carbonview created ‘CVEngage’ as our instrument to handle this new and growing trend to speak directly with the respondent.  When creating this service, we focused on the following areas…

• Allow the survey to utilize this new service in the same manner as a question.  This includes allowing for skip logic to only allow those folks that you want to speak with directly to get the CVEngage screen. 
• Allow the client to choose the way they would like to engage with the respondent.  They can chat with the respondent directly, choose their own online moderator or look to Carbonview to provide the online moderator for them.
• Full Integration with the survey and the data collected inclusive of allowing the moderator/client to be able to see the answers to previous questions in the survey while chatting with that respondent.

 

Compound question types (Creating the conversation)

                Replacement of:  More complicated skip logic and reference points for the respondent.  Contextual questioning has historically needed in-depth reference points and in most cases could not follow a natural thought process, but yet needed to adhere to a restrictive survey design/process

                Carbonview utilizes many of our survey engagement widgets to create that conversation.  One of those utilizes ‘lightbox’ technology to immediately overlay the question needing follow-up in such a way that allows the person to respond in a more conversational approach.  Another example is utilizing any number of drag-drop methods for response entry.  These types of questions such as store-shelf displays, dial tests for commercials, image zoning for print media all store the data in traditional form input question types (i.e. single response, multi-response, numeric, etc.), but allow the respondent the ability to engage with the survey in a more cohesive manner.

 

Page-Flip

                Replacement of: Clunky non-immersive form-input questions that are administered in linear format thus keeping adhering to the form-input restrictions.  In many cases, the print media being tested was physically sent to the respondent’s house so they can read and review in the form it was packaged in.

                While many have created the page-flip technology for print media testing within Market Research, Carbonview has created a page-flip that is highly adaptable.   Currently, Carbonview can display total magazines and/or menus and measure such interactions as time on page, recall, specific advertisement content measurement, immediate and immersive overlay’s giving the research process a more immersive approach.

 

Three basic rules of the Game with these new types…

• Pay attention to the format of the data - None of these question types are of value if they can’t be brought back with some effectiveness to a data structure that can be effectively aggregated and reported on. 
• Do not require the respondent to download anything when interacting with these new question types. 
• Give clear instructions and if possible an example tutorial video on how to interact with these new types of questions.   If the respondent isn’t used to the format and is left to work out the interactive nature the research can be suspect.

 

 

 

 

Common Sense

 

Common Sense

At what point does common sense enter into our market research process?  Rhetorical, yes, but it seems to be a question that needs to be answered quite frequently.  A most recent project is the impetus for this entry as it seemed that the further we progressed, the more separated we became from ‘common sense'.

How did we get here?

As with many of these types of projects, the client didn't come to us with the specifications of ‘I would like to answer a question I have, but I would like to throw common sense out the window and create a qualification that filters out 99% of the individuals who represent this universe.'    Our research started quite normal with; this is what we need to test, these are the individuals we need to interview.  The problem arose when a typing tool was applied as a standard procedure the client had as part of the screening criteria.  There was an obvious issue when the qualifying audience dropped from the estimated 60% to 1.5%.  While we attempted to get our client to see that something was seriously incorrect, they persisted in the name of standards.   Normally, the study would pause for clarity, the client pushed to continue.

 

"Everybody gets so much information all day long that they lose their common sense" (Gertrude Stein)

"Have you lost your common sense?"  Common sense can be that reality check when you have made decisions that any other person would not make when presented with the same exact scenario, details and all.   So, when the client persisted with believing that their criteria was correct, I asked a simple question;  Are you looking to get your answer from collecting the opinions of those who represent 1% of your total consumer base that would purchase this product?   Common sense should have prevailed, it didn't.   In the end, the client kept to their ‘standard' and we delivered results and findings on that 1% segment making sure to state the criteria of the study upfront. 

What does your gut say?

We have all heard this phrase; ‘My gut says....'  I think this is a step that should be taken for every market research project.  If common sense, the popular and most reasonable thought,  is not central to the design then we (as an industry) should acknowledge that fact.  Many advances in every aspect of our life have been made by bucking the ‘common sense' rule.  I agree that if one wants to step out on that daring limb, they should focus solely on seeing it through.  Where would we be if Bill Gates would have considered selling code as a move separated from common sense?When lack of common sense is created by non-decision making or persisting in the name of standards, now that is a different story all together.

At the end of the day, we need common sense...

Common sense is that barometer that is needed in every aspect of life.  Food smells bad, common sense says don't eat it.  When dealing with market research, knowing why the research is needed is one of the main drivers of that common sense approach.   I speak to this issue on a regular basis with Carbonview and hope that in doing so our services are better for it.  Our job is to make sure the findings are attained in the most efficient, cohesive and representative approach available.  The assumption and foundation for those items to have a chance at success, is...common sense.

 

Market Research Checklist

 

Over the past few years, I've seen many checklists within the Market Research space.  Mainly these checklists have been focused on execution, but recently NGMR (Next Generation Market Research) on Linkedin had a great discussion on a checklist during the onset of the research process.   While the discussion yielded amazing insights and thoughts to the items that should be on a checklist for Market Research, I wanted to create a more concise and usable checklist from that discussion.  Thanks to all the  NGMR members who contributed to this list. 

Below is a concise and effective approach to bringing clarity to your next project.  While I don't presume this list to be all encompassing, it's a great start to your own more focused list.  Feel free to email me with your checklist and/or items to add ( rich.ratcliff@carbonview.com )

Let's start with the beginning...

What is the Study (and Stakeholders') Objectives?

What business problem is the study trying to address?

How do the stakeholders of the study think the research will help them address their needs?

Will the research provide recommendations that are actionable?

Are the benefits of the research recommendations quantifiable?

What decisions will the results of this study influence?

What are the client or stakeholder's expectations?  

How research-savvy are they?

 

Continuing with some research basics...

If this project goes perfectly, what will the client be able to do that they can't do today?

Do the stakeholders need results for an event, budget cycle, or strategic planning process?

Who will use this research (sales department, customer care, advertising team?)

Is there existing data from past research or other sources, such as web analytics?

What is the context of the research?  The context stems directly from the reason the research is being funded, in most cases, client decisions.

 

 

This is an opportunity...or is it?

If this is a new client, is this a competitive RFP? 

Why did this new client seek you out? 

Who else is competing for this research? How will they respond?  

Is your contact the decision-maker for this project? 

How committed is the decision-maker to actually acting on the results?

How much budget does the client have to spend?  For planning, rather than reworking the study later.

Are the client's expectations reasonable and attainable given the budget and timing?

 

Don't get into trouble...

What are the legal limitations for doing research in this country and state? This includes laws about using auto-dialer, privacy, intellectual property rights, unsolicited email and/or phone calls, etc.

Are there any ethical issues to be addressed while designing the study.

Are there any MR code of conduct issues to be addressed before considering this project?

 

Keeping it real...

Is there a geographical context and/or requirement?

What is the extent you or the client can generalize the research findings (i.e. population of interest).

Is the target audience for the study feasible? 

Once the above questions are discussed, the study should start to form regarding tactical project parameters.  Below are some parameters to discuss if you haven't already...

• Research Approach
• Qualitative or Quantitative?
• Methodology
• Specific Exercises needed (Conjoint, Maxdiff, etc.)
• Length of Interview
• What kind of sample will suit this study best?
• Type of respondent (e.g., CxO, consumers, Developers). This needs to have as much clarity as possible.
• Number of market segments (e.g., company sizes, industries, adoption criteria).
• If possible, any understanding of the incidence of this audience
• What is the targeted timeframe for completing the project? (i.e., # of days, weeks, or months)

 

Market Research Solutions - Clarity and Superior Results

 

Superior results can't happen without clarity.  On the other hand while someone can have clarity, superior results aren't guaranteed.  Execution of that clear vision is imperative, but that is of course another subject for another day.

The concept of ‘clarity and superior results' has been a focus of mine lately.  While it seems novel and obvious, listening to a client that validly isn't clear with the objective of ‘selling' will only put both parties on a path to being dissatisfied with the process and more than likely the results.  As I sat in a call with a potential client today I realized ‘they aren't clear regarding the question they need to answer', my objective changed to convincing them not necessarily to purchase our services, but to find clarity.  Clarity in the objective of the research, clarity in the options available, and most importantly clarity in the question(s) they need to ask themselves.   An hour later, clarity reached and subsequent solution is in the process of getting designed. 

Clarity and Superior Results, maybe I'm a little late on my New Year's resolution and if so I'll make an exception and add another to my list.  If you want to experience superior results, achieving clarity is the starting point.  Nothing should happen beforehand.  If we start any project as market researchers or solution developers pursuing a nebulous vision, our objectives will be less focused and even worse, those objectives may be completely incorrect thus creating ineffective research or solutions.   One thing I have always been clear about, client's purchase or commission market research for clarity in the hopes that research will assist them in superior results.  If we are clear about how we can bring that clarity to them, the rest is easy.

Respondent Engagement

Most recently we had the opportunity to work with a very innovative organization that posed a few questions to us regarding our philosophy in a few different but similar areas of respondent engagement.  What I found interesting is how thought provoking their very simple questions were.  My intention in posting these questions here is to allow you the reader to ask yourself the same question(s).  I pride myself on the very real notion that Carbonview can make a difference in the quality of the research we provide in our own small way.  The intention of these questions for anyone who ventures to answer them gives insight into an approach and thus what a great way to communicate how Carbonview lives day by day, month by month, year by year, moving forward in both market research and technology because we embraced technology as an enabling agent to better research.

 

What are the limitations/downside of current on-line surveys?

Even after the mass-acceptance of this methodology more than 15 years ago, so much of today's online research still attempts to mirror what was done with non-online opinion collection methods. Carbonview is one of the few companies in the Market Research industry that embraces technology to deliver against the research needs of modern companies. We look at each individual project as a unique opportunity to supply the optimum blend of today's latest technology and solid market research techniques, enabling us to deliver the best possible respondent and client experience.

How does the Carbonview technology give richer, more conversational and emotional involvement and data from the respondent?

Carbonview's primary focus is on the collection of information from a respondent with the most appropriately engaging instrument. Whether you are testing a concept by utilizing our 'Image Zoning' or researching the usability of a website utilizing C2View, our focus is to create a more conversational approach and thus a more engaging and interesting interview process. All of our developments are based on creating a natural and more efficient approach to collecting information.

The result creates an interactive, conversational, two-way exchange of information. Our 'standard' online approach creates a built-in intuitive exchange of question and answers and thus increased acceptance by the individuals we need the most; The respondents.

What are some specific examples of how Carbonview has helped our clients garner a deeper understanding and/or appreciation of their consumer/end-user?

Example 1: Carbonview created a custom virtual shelf display with drag/drop of the actual images of the products instead of just names, to allow the appropriate connection to awareness, usage and perception of those brands to come through. The vertical for this research stakes a large part of its value on the unique branding within each competitor. Given the inherent unique branding within each competitor for this task, we made sure to keep that branding value by not introducing plain text into the measurement process. Better recall, better results, better understanding of the consumer.

Example 2:  Carbonview most recently created the ability to measure the user interaction for any website our clients want to measure. The website does not need to load anything on their pages and the respondent does not need to load anything on their computer. Additionally, our clients receive videos of the interaction with the website. Our ability to allow the respondent to be in the environment on their computer on a normal website with no barriers has allowed our clients to get the insight that has never before been available.