Muslim American Survey, 2017

DOI

10.17605/OSF.IO/HMRWK

Citation

Mohamed, B., & Smith, G. A. (2023, March 16). Muslim American Survey, 2017.

Summary

This is the third national probability survey of American Muslims conducted by Pew Research Center (the first was conducted in 2007, the second in 2011). Results from this study were published in the Pew Research Center report 'U.S. Muslims Concerned About Their Place in Society, but Continue to Believe in the American Dream.' The report is included in the materials that accompany the public-use dataset.

The survey included interviews with 1,001 adult Muslims living in the United States. Interviewing was conducted from January 23 to May 2, 2017, in English, Arabic, Farsi and Urdu. The survey employed a complex design to obtain a probability sample of Muslim Americans. Before working with the dataset, data analysts are strongly encouraged to carefully review the 'Survey Methodology' section of the report.

In addition to the report, the materials accompanying the public-use dataset also include the survey questionnaire, which reports the full details on question wording. Data users should treat the questionnaire (and not this codebook) as the authoritative reflection of question wording and order.

The ARDA has added six additional variables to the original data set to enhance the users' experience on our site.

Data File

Cases: 1001
Variables: 222
Weight Variable: WEIGHT

The sample design featured 10 strata - four strata in the landline frame and six strata in the cellphone frame. The first weighting adjustment corrected for the fact that telephone numbers in some strata were sampled at a relatively high rate (oversampled) and telephone numbers in other strata were sampled at a relatively low rate (undersampled). Specifically, if Nh is the total count of telephone numbers on the sampling frame in stratum h and nh is the count of numbers sampled from that stratum, then the first weighting adjustment was computed as w1 = Nh/nh.

The next adjustment corrects for the fact that the response rate was higher in some strata and lower in other strata. Specifically, if Sh is the count of phone numbers sampled in stratum h that yielded a completed screener interview, Eh is the count of sampled phone numbers in stratum h that were known to be eligible for screening (e.g., working and residential), and E(sample mean)h is the count of sampled phone numbers in stratum h for which eligibility was unknown but are estimated to be eligible for screening, then the second weighting adjustment was computed as w2 =(Eh +E(sample mean)h)/Sh.

The next adjustment corrected for the fact that the non-Muslims in the dataset represent only half of all of the non-Muslim adults screened. To avoid unnecessary data collection, only 50% of the non-Muslims (n=20,102) were administered the demographics module. To correct for that subsampling, the third weighting adjustment (w3) multiplies the weights for the non-Muslim cases by 2 and does nothing to the weights for the Muslim cases (multiplication by 1).

The fourth weight adjustment concerned only landline cases. It adjusted for the fact that landline respondents who live in a household with multiple adults were less likely to be interviewed than landline respondents who live alone. For example, if a household with a landline sampled for this study had four adults living in it, then all four adults were eligible for the survey, but we only interviewed one of them, meaning that three were not selected. By contrast, when we called a landline number used by someone who lives alone, that person was selected every time. In order to avoid underrepresenting people who live in multi-adult households, the fourth adjustment multiplied the weight for landline cases by the number of adults living in the household (Ai).

The next step was to account for the overlap between the landline and cellphone sampling frames. Adults with both a residential landline and a cellphone ('dual service') could potentially have been selected for the survey in both frames. In other words, they had a higher chance of being selected for the survey than adults with just a landline or just a cellphone. To correct for that, the weighting adjusts down dual users so that they are not overrepresented in the survey. The specific approach used in this survey, composite estimation,38 combines the interviews with dual users from the landline sample and the interviews with dual users from the cellphone sample using a weighted average. The two groups of dual users were weighted equally, a common specification to reduce the variance of the weighted estimates. The adjustments detailed above were applied to create the full base weight, which was used as the input weight for the raking procedure.

Due to the complex design of the Muslim American study, formulas commonly used in RDD surveys to estimate margins of error (standard errors) are inappropriate. Such formulas would understate the true variability in the estimates. Accordingly, analyses in this report used a repeated replication technique, specifically jackknife repeated replication (JRR), to calculate the standard errors. Repeated replication techniques estimate the variance of a survey statistic based on the variance between subsample estimates of that statistic. The subsamples (replicates) were created using the same sample design, but deleting a portion of the sample, and then weighting each subsample up to the population total. A total of 100 replicates were created. A statistical software package designed for complex survey data, Stata, was used to calculate all of the standard errors and test statistics in the study.

Data Collection

January 23 - May 2, 2017

Original Survey (Instrument)

Pew Research Center 2017 Survey of U.S. Muslims

Funded By

Support for this project was provided by the Pew Charitable Trusts.

Collection Procedures

Once the strata were defined at the county level and through the use of the flags as described above, the next step in the sampling process involved employing an optimization algorithm to allocate the interviewing across the six geographic strata and the flagged stratum. The algorithm maximized the expected precision of the survey (specifically the effective sample size) factoring in the different Muslim incidence levels in various strata, as well as the loss in precision stemming from weighting adjustments.

At this point, a sample of cellphone numbers was drawn from within each geographic stratum (except the lowest density stratum). The numbers were drawn from the list of all residential cellphone numbers in the United States. These numbers were then compared to the entire set of cellphone numbers from the flagged stratum. Any numbers that appeared in both a geographic stratum and the flagged stratum were removed from the former, and were available to be sampled only as part of the flagged stratum.

This process was then repeated for landline numbers, except that no telephone numbers were drawn that were associated with the low, very low, or lowest strata within the landline frame. The decision to refrain from dialing landline numbers in these strata was based on the extremely low rate at which interviews done on landlines in these areas have yielded interviews with Muslim respondents. In recent Pew Research Center surveys, for instance, just one in 1,323 respondents interviewed on a landline in the lowest stratum has been Muslim.

The strength of this research design was that it yielded a probability sample. That is, each adult in the U.S. had a known probability of being included in the study. The fact that some persons had a greater chance of being included than others (e.g., because they live in places where there are more Muslims) is taken into account in the statistical adjustment described below. In total, 40,987 screening interviews were completed as part of this study. Of these, 1,001 respondents identified themselves as Muslim and completed the full interview.

Overall, the estimated coverage rate for Muslim Americans provided by the study, accounting for the excluded strata, is over 90%.

Sampling Procedures

One of Pew Research Center's goals in this study was to interview a sample of at least 1,000 American Muslims. In random-digit dial (RDD) surveys of the English-speaking U.S. population, roughly 1% of respondents typically identify as Muslim in response to a question about their religious identity. This means that if the Center had relied exclusively on national RDD sampling techniques, it would have had to interview and screen roughly 100,000 people in order to identify and recruit a sample of 1,000 Muslims. Such an approach is impractical. Instead, researchers used existing data on the Muslim American community and on telephone users more generally to design a sampling plan that reached and interviewed a nationally representative sample of Muslim Americans more efficiently than a simple RDD approach would have done. The first step in designing the sampling plan involved using several sources of data to estimate the share of the population that is Muslim for each county (or county equivalent) in the U.S. One key resource in this effort was the Pew Research Center database of more than 150,000 telephone interviews conducted between 2011 and 2016. Another resource was data from the American Community Survey (ACS), which is an annual survey conducted by the U.S. Census Bureau. The Census Bureau does not collect information about religion, but the ACS does include measures of ancestry, national origin for immigrants, and languages spoken. These measures were used to analyze the geographic distribution of adults who are from (or whose ancestors are from) countries with significant or majority Muslim populations, or who speak languages commonly spoken by Muslims.

The other data sources used to create county-level estimates were the 2010 Religious Congregations and Membership Study, county-level counts of flagged likely Muslim telephone numbers provided by Survey Sampling International, and a dataset of county-level official statistics (e.g., educational attainment, housing stress, economic activity) archived by the Inter-university Consortium for Political and Social Research. Survey designers at Abt Associates used a statistical approach known as small-area estimation to take these various data and estimate the density of Muslims in each county.

The next step was sorting all of the counties in the U.S. into six different groups, or geographic strata, based on the estimated incidence of Muslims. Eight counties were placed in the 'very high' geographic stratum. These eight counties are home to just 2% of all U.S. adults (according to Census data), but to 13% of all U.S. Muslim adults (according to Pew Research Center surveys conducted since 2011), and Muslims account for nearly 5% of the population in these counties.35 The second highest density stratum, the 'high' stratum, consists of 25 counties that are home to just 4% of all U.S. adults but to 14% of all U.S. Muslims; Muslims account for nearly 3% of the population in these counties.

At the other end of the spectrum, the lowest density geographic stratum includes 785 primarily rural counties which are home to 21% of all U.S. adults, but just 4% of all U.S. Muslims and where Muslims account for just 0.1% of the overall population. These counties were excluded from the geographic strata, though the study includes partial coverage of Muslims living in this stratum through the use of flagged sample (described below) and because some people (including some Muslims) have phone numbers associated with one of the higher strata but actually reside in the lowest density stratum.

Principal Investigators

Besheer Mohamed, Senior Researcher, Pew Research Center
Gregory A. Smith, Associate Director of Research, Pew Research Center

Related Publications

Pew Research Center. July 26, 2017. "U.S. Muslims Concerned About Their Place in Society, but Continue to Believe in the American Dream."

Pew Research Center. Aug 15, 2017. "How Pew Research Center Conducted Its 2017 Survey of Muslim Americans." YouTube.

Pew Research Center. Apr 17, 2018. "Being Muslim in the U.S." YouTube.