Is your multicultural research misleading marketing decisions?

Despite the dramatic growth of multicultural populations in the U.S., many survey companies continue to use outdated assumptions and practices in the design and execution of surveys in communities that are linguistically and culturally diverse. Following are some of the more problematic practices that may warrant your attention, whether you are a survey practitioner or a buyer of survey research.

1. Is your survey team culturally sterile?

If your survey team lacks experience conducting surveys in diverse communities, you may  already be dead on arrival. Since most college courses on survey or marketing research do not address the problems that are likely to occur in culturally-diverse communities, mistakes are very likely to happen.  An experienced multicultural survey team member is needed to assess the study challenges and resources. Really, how else will you know if something goes wrong?

2.  Are you planning to outsource to foreign companies?

So your firm has decided to outsource its Latino or Asian surveys instead of hiring your own bilingual interviewers. Think twice about this.  If you have ever monitored interviews conducted by foreign survey shops, you are likely to discover several issues that impact survey quality: language articulation problems, and a lack of familiarity with U.S. brands, institutions, and geography.  The money that you save by outsourcing will not fix the data quality issues that will emerge from these studies. Better to use an experienced, U.S. based research firm with multilingual capabilities that does not outsource to foreign survey shops.

3. Are you forcing one mode of data collection on survey respondents?

Think about it --  mail surveys require reading and writing ability; phone surveys require one to speak clearly; and online surveys require reading ability and Internet access. Forcing one mode of data collection can exclude important segments of consumers that can bias your survey results. Increasingly, survey organizations are using mixed-mode methods (i.e., combination of mail, phone and online) to remove these recognized limitations, and achieving improved demographic representation and better quality data.

4. English-only surveys make little sense in a multicultural America.

Of course, everyone in America should be able to communicate in English, and most do. But our own experience confirms that two-thirds of Latino adults and 7 in 10 Asians prefer a non-English interview when given a choice. The reason is simple: Latino and Asian adults have large numbers of immigrants who understand their native language better than English – which translates to enhanced comprehension of survey questions,, more valid responses, and improved response rates.  Without bilingual support, the quality of survey data is increasingly suspect in today’s diverse communities.

5. Are you still screening respondents with outdated race-ethnic labels?

Multicultural persons dislike surveys that use outdated or offensive race-ethnic labels that are used to classify them – which can result in the immediate termination of the interview, misclassification of survey respondents, or missing data. Published research by the Pew Research Center and our own experience suggests that it is better to use multiple rather than single labels in a question: that is, “Do you consider yourself Black or African American, Hispanic or Latino, Asian or Asian American, white or Anglo American?” Since Latinos and Asians identify more strongly with their country of origin, it is a good idea to record their country of origin or provide a listing of the countries represented by the terms Latino or Asian.  Use of the label Caucasian is often used along with the white label, but should be avoided because the Caucasian category also includes Latinos.

6.  Are your survey respondents consistently skewed towards women?

A common problem is that multicultural males are considerably more reluctant than white males to participate in surveys, which often results in survey data that is overly influenced by female sentiments and behaviors. The imbalance often results from the poor management of interviewers who dedicate less effort to getting males to cooperate. Rather than improve data collection practices that create such imbalances, survey analysts will typically apply post-stratification weights to correct the imbalance even when large imbalances are found – a practice that can distort the survey results.  It is always a good practice to review both un-weighted and weighted survey data to judge the extent of this problem.

7.  Online panels are not the solution for locally-focused multicultural studies.

With high anxiety running throughout the survey industry from the recent FCC settlement of $12 million with the Gallup Organization, many survey companies will likely replace their telephone studies with online panels.  For nationally-focused surveys, online panels may be an adequate solution to reach a cross section of multicultural online consumers. For local markets, however, the number of multicultural panel members is often insufficient to complete a survey with a minimum sample of 400 respondents. Worse yet, the majority of multicultural panel members are the more acculturated, English-speaking, higher income individuals – immigrants are minimal on such panels. Online panel companies will have to do a better job of expanding their participants with multicultural consumers. In the meantime, don’t get your hopes too high.

8.  Translators are definitely not the last word on survey questionnaires.

So your questionnaire has just been translated by a certified translator, and you are confident that you are ready to begin the study of multicultural consumers. After a number of interviews, however, you learn that the survey respondents are having difficulty understanding some of the native language vocabulary being used, and interviewers are having to “translate-on-the-fly” by substituting more familiar wording – a major problem in multicultural studies. It is obvious that the survey team placed undue confidence on the work of the certified translator, and did not conduct a pilot study of the translated questionnaire to check for its comprehension and relevance among the target respondents.  A good pilot study can save you time, money and headaches.

These tips represent only a partial listing of the many ways in which a survey can misrepresent multicultural communities.  Industry recognition of these types of problems is a first step towards their elimination, although survey practitioners are slow to change their preferred ways of collecting data. Raising the standards for multicultural research will perhaps pick up steam once higher education institutions require the study of these issues in their research courses, and buyers of research require higher standards from research vendors.

You can reach Dr. Rincón at edward@rinconassoc.com

© Rincón & Associates LLC 2015