December 18, 2013

On Immigration Policy, Deportation Relief Seen As More Important Than Citizenship

Appendix B: Methodology of surveys

Survey data in this report are based on two Pew Research Center surveys conducted with a nationally representative sample of either Hispanics or Asian Americans.

Differences between groups or subgroups, such as foreign-born and native-born Hispanics or foreign-born and native-born Asian Americans, or between Asian Americans and Hispanics, are described in this report only when the differences are statistically significant and therefore unlikely to occur by chance. Complex survey designs and weighting procedures affect variance estimates and, as a result, influence tests of significance and confidence intervals surrounding such tests. Statistical tests of significance take into account the complex sampling design used for these surveys and the effect of weighting.

Survey Data: Hispanics

Pew Research Center Survey of HispanicsThe nationally representative survey of Hispanics was conducted in both English and Spanish on cellular as well landline telephones with 701 Hispanic adults, ages 18 and older, living in the United States. The survey was conducted from Oct. 16 to Nov. 3, 2013, in all 50 states and the District of Columbia. Interviews were conducted for the Pew Research Center by Social Science Research Solutions (SSRS) with a staff of bilingual interviewers; respondents could choose to be interviewed in English or Spanish or switch between languages during the interview. About half of the interviews (49%) were conducted in English and about half (51%) in Spanish.

The margin of sampling error for results based on the total sample of 701 Hispanic adults is plus or minus 4.4 percentage points at the 95% level of confidence. This means that in 95 out of every 100 samples drawn using the same methodology, estimated proportions based on the entire sample will be no more than 4.4 percentage points away from their true values in the population. Results based on subgroups will have larger margins of error. In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.

Sample Design

The sample was drawn from two sources. One sample source came from recontacting respondents or other Hispanics in households who completed the 2013 Pew Research Center survey of Hispanics that was conducted from May 24 to July 28, 2013. Eligible respondents or other household members were recontacted and asked to complete this survey after rescreening the household for an eligible Hispanic adult and for those on landlines, a random selection of respondent to interview. Eligible respondents/households are those from all strata that used random digit dial sampling in the originating survey (N=2,000). The originating survey used a stratified sampling design with oversampling (i.e., geographic-based disproportionate sampling) in areas with higher incidence of Latinos. For more details on the methodology of the originating survey, see Appendix A in “Three-Fourths of Hispanics Say Their Community Needs a Leader.

A second sample source came from Hispanics who participated in the EXCEL omnibus survey conducted by SSRS. The EXCEL omnibus survey is a weekly survey of U.S. adults covering a variety of topics. The sample design is a fully replicated, stratified, single-stage random digit dialing sample of telephone households with random selection of an eligible respondent in each household. Each week a minimum of 1,000 interviews are completed and at least 40% of completed interviews are done with respondents on their cellphone. Hispanics completing the EXCEL omnibus survey also completed this survey.

Weighting

Several stages of statistical adjustment or weighting are used to account for the complex nature of the sample design and to ensure an accurate representation of the national Hispanic population. The first stage of weighting accounted for the base weights of respondents/households in the originating survey and a propensity weight adjustment associated with recontacting previously interviewed households. The weighting procedures also included an adjustment for unequal probability of selection for those in the EXCEL omnibus sample found to possess both a landline and a cellphone and an adjustment for the likelihood of within household selection for those with multiple adults in the EXCEL omnibus landline sample.

In addition, the data were put through a post-stratification sample balancing routine to population totals for the U.S. Hispanic adult population based on the 2012 U.S. Census Bureau’s Current Population Survey, March Supplement and for phone use parameters, the 2010 National Health Interview Survey, projected to 2013. Iterative proportional fitting technique, or raking, corrects for differential nonresponse that is related to particular demographic characteristics of the sample. This weight ensures that the demographic characteristics of the sample closely approximate the demographic characteristics of the population. The variables matched to population parameters were: gender, education, age, Census region, heritage, U.S. born or years in the U.S., phone use (i.e., cellphone only, cellphone mostly, mixed/landline only/landline mostly), and density of the Hispanic population. Following raking, the weights were trimmed to control the variance created by the weight.

Survey Data: Asian Americans

Pew Research Center Survey of Asian AmericansThe nationally representative survey of Asian Americans was conducted with 802 Asian-American adults, ages 18 and older, living in the United States. The survey was conducted Oct. 16-31, 2013, in all 50 states and the District of Columbia. Interviews were conducted for the Pew Research Center by Abt SRBI. The survey was conducted in English as well as Vietnamese, Korean, Mandarin and Cantonese.

Respondents who identified as “Asian or Asian American, such as Chinese, Filipino, Indian, Japanese, Korean, or Vietnamese” were eligible to complete the survey interview, including those who identified with more than one race and regardless of Hispanic ethnicity. The question on racial identity also offered the following categories: white, black or African American, American Indian or Alaska Native, and Native Hawaiian or other Pacific Islander.

The margin of sampling error for results based on the total sample of 802 Asian-American adults is plus or minus 5.0 percentage points at the 95% level of confidence. This means that in 95 out of every 100 samples drawn using the same methodology, estimated proportions based on the entire sample will be no more than 5.0 percentage points away from their true values in the population. Results based on subgroups will have larger margins of error. In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.

U.S. Asian groups, subgroups, heritage groups and country-of-origin groups are used interchangeably in this report to reference respondents’ self-classification into “specific Asian groups.” This self-identification may or may not match respondents’ country of birth or their parents’ country of birth. Self-classification is based on responses to an open-ended question asking for a respondent’s “specific Asian group.” Asian groups named in this open-ended question were “Chinese, Filipino, Indian, Japanese, Korean, Vietnamese, or of some other Asian background.” Respondents self-identified in more than 20 Asian heritage groups; this self-classification was unrelated to eligibility for the survey.

Sample Design

The survey was conducted using a probability sample based on recontacting eligible respondents or other Asian Americans in households from previous Pew Research Center surveys. There were three sources of recontact cases for this survey.

One sample source came from recontacting respondents or other Asian Americans in households that completed the Pew Research Center 2012 Survey of Asian Americans. Eligible respondents or other household members were recontacted and asked to complete this survey after rescreening the household for an eligible Asian-American adult. Eligible respondents/households are those from all strata using random digit dial sampling in the originating survey. For more details on the methodology of the originating survey, see Appendix 1: Survey Methodology in the “The Rise of Asian Americans.

A second source of recontact cases came from Asian Americans interviewed as part of a brief screener survey for a Pew Research Center survey of American Jews in 2013. The third source of recontact cases came from Asian Americans interviewed in other prior Pew Research Center surveys conducted during 2012 or 2013. Eligible respondents or other household members were recontacted and asked to complete this survey after rescreening the household for an eligible Asian-American adult.

Surveying in Multiple Languages

The survey was conducted in English as well as Vietnamese, Korean, Mandarin and Cantonese. Approximately 86% of Asian Americans in the random digit dial sampling strata of the originating survey (the 2012 Survey of Asian Americans) were interviewed in English. And all of those in the other recontact sample sources completed at least a brief interview in English. Thus, a high share of English-proficient adults was expected to complete the current survey and, indeed, 95% of the completed interviews were conducted in English. To adjust for differential nonresponse of Asian Americans with limited English proficiency, the survey included two questions also asked on the American Community Survey about the use of multiple languages in the household and the respondent’s self-reported English-language skills. As noted below, these responses were used to balance the sample to match national Asian-American adult population parameters on English-language skills.

Weighting

Several stages of statistical adjustment or weighting are used to account for the complex nature of the sample design and to ensure an accurate representation of the national Asian-American population. The weights account for numerous factors, including (1) differential rates of selection in the original surveys in which each recontact case was sampled; (2) an adjustment for numbers with unknown eligibility for the present survey; (3) an adjustment for nonresponse to the original survey; (4) an adjustment for unequal probability of selection for those with multiple cellphones used by adults in the household; and (5) an adjustment for the likelihood of within household selection in the landline sampling frames with multiple eligible adults. An additional adjustment was made to account for the overlap between cellphone and landline random digit dial sampling frames.

In addition, the data were put through a post-stratification sample balancing routine to population totals for the U.S. Asian adult population based on the 2011 American Community Survey public use microdata (ACS PUMS). Iterative proportional fitting technique, or raking, corrects for differential nonresponse that is related to particular demographic characteristics of the sample. This weight ensures that the demographic characteristics of the sample closely approximate the demographic characteristics of the population. The variables matched to population parameters were: heritage group x nativity, heritage group x gender x age, heritage group x gender x education, heritage group x census region, English-language skills, gender x age, gender x education, and education x age. Heritage group included seven categories: Chinese, Filipino, Asian Indian, Japanese, Korean, Vietnamese, and other Asian or multiple Asian heritage. For some parameters, Vietnamese heritage was combined with other Asian/multiple Asian heritage. In addition, the sample was balanced to match Asian-American adult telephone service estimates based on the 2012 National Health Interview Survey. After raking, the weights were trimmed to control the variance created by the weights and to improve the precision of the weighted survey estimates.