Many factors are considered in determining the size of Utah’s BRFSS sample. These factors include cost, number and size of populations for which estimates are desired, acceptable level of confidence in estimates, the tolerable range of error, the prevalence of the behavior, and the effect of the sampling design.
Initially, the Utah sample size was designed to allow state-level estimates along with analysis by some common demographic subgroups such as age and sex. Since 1995 the sample has been stratified by Utah’s 12 local health districts. From 1995 to 2003, the sample size in each district was determined so that a minimum of 500 interviews were collected every three years. This process lead to an over-sampling in Utah’s rural health districts. The larger sample size allows for estimates with an approximate 5% range of error at a 95% confidence level within the health district. However, for some questions that are asked of only subpopulations of respondents or are asked only some years, the range of error may be larger.
In 2004 the sample was increased within the 12 health district strata in order to obtain 100 completed interviews in each of Utah’s 61 small areas by combining three years of data. Because the urban health districts are further divided into small areas based on population size, this change in the sample design mostly lead to increases in the sample for Utah’s urban health districts. In 2009, the sample size increased again to accommodate a set of health insurance and access questions. By doubling the sample size, the new questions could be asked of half of respondents without compromising the questionnaire space already used by other programs for state-added questions.
In 2009, the sample size increased again to accommodate a set of health insurance and access questions. By doubling the sample size, the new questions could be asked of half of respondents without compromising the questionnaire space already used by other programs for state-added questions.
Because Utah has not used simple random sampling, some adjustment is needed in determining sample size. The non-random sampling design causes the precision of the estimates to be lower. This means that a larger sample size must be obtained than would be required with simple random sampling. This adjustment factor is called a design effect (DEFF). It must be estimated in advance in order to determine sample size. In general, a reasonable DEFF estimate for the BRFSS is 1.3. The following formula can be used to estimate sample size for the BRFSS.
N=[((1.96^2)(0.25))/tolerable range^2]DEFF *
*NOTE: This calculation assumes a 95% confidence level; if the tolerable range was +/-0.05 (5%) and the estimated design effect was 1.3, then N=[((1.96^2)(0.25))/0.05^2]1.3 or 499 respondents.
BRFSS Sampling Designs
From 1984 through 1998, Utah used a cluster sampling technique - a three-stage procedure based on the Mitofsky-Waksberg method of random-digit dialing. In the Mitofsky-Waksberg sampling design, as implemented in the BRFSS, telephone numbers are randomly selected from blocks of 100 telephone numbers generated from the set of all existing area codes and prefixes in the state. Then, sampling is carried out in three stages. In the first stage, selected blocks of 100 randomly ordered numbers are screened to determine the household status of the first (i.e. defining) number in each block. Blocks remain in the sample only if a residence is reached. In the second stage, the 100 numbers in the accepted block are randomly dialed to identify additional households. In the third stage, individual respondents are randomly selected from all adults aged 18 and older living in a household and are interviewed in accordance with BRFSS calling protocol until the target number of three interviews is completed. The completed interviews from a given block constitute a cluster, also known as a primary sampling unit or PSU.
Since 1999 the BRFSS has used the Disproportionate Stratified Sampling (DSS) design. In years 1999 through 2002, in the DSS design, all the telephone numbers in each health district were disproportionately stratified by telephone blocks. A block consists of 100 phone numbers that differ only by their last 2 digits (e.g. (801)-538-1100 to (801)-538-1199). One-plus block (high-density stratum) are computer-generated listings of 100 consecutive telephone numbers containing at least one published household telephone number. Zero-blocks (low-density stratum) are listings of 100 consecutive telephone numbers containing no published household telephone numbers. To be representative, both one-plus and zero block numbers were randomly sampled from each health district, but at a disproportionate rate of 4:1.
Beginning with the 2003 data year, zero block numbers were not included in the sample frame. In the DSS design since 2003, telephone numbers were drawn from two strata (lists) that are based on the presumed density of known telephone household numbers. In this most recent DSS design, telephone numbers are classified into two strata that are either high density (listed 1+ block telephone numbers) or medium density (not listed 1+ block numbers). High density versus medium density numbers are sampled at a rate of 1.5 to 1.
Although BRFSS was designed to produce state-level estimates, Utah began geographical stratification of the sample by local health district in 1995 in order to be able to analyze the data by Utah’s 12 local health districts approximately every three years. The telephone sample is drawn independently for each local health district. The use of stratified random sampling requires knowledge about the composition of subpopulations; because telephone prefixes generally follow geographic boundaries, stratified random sampling can be used in telephone surveys to generate probability samples within geographic areas.
In 2009, Utah began calling cell phones. The sample supplier uses a sampling frame based on the Telecordia database of telephone exchanges and 1,000 banks (e.g., 617-492-000 to 617-492-0999). They use dedicated cellular 1,000 banks, sorted on the basis of area code and exchange within a state. An interval, K, is formed by dividing the population count of telephone numbers in the frame, N, by the desired sample size, n. The frame of telephone numbers is divided into n intervals of size K telephone numbers. From each interval, one 10-digit telephone number is drawn at random. Currently, the cell phone sample is not stratified by geographic area.
New analysis was done by the CDC on previously existing data from the 2002 BRFSS to calculate estimates for selected Metropolitan and Micropolitan Statistical Areas (MMSAs) and counties. This analysis included two MMSAs in Utah: Ogden-Clearfield, UT Metropolitan Statistical Area and the Salt Lake City, UT Metropolitan Statistical Area, and four counties: Davis, Weber, Salt Lake and Tooele. In order to enable the CDC to include the Provo-Orem UT Metropolitan Statistical Area in any future analysis, the sample was increased in the Utah County Health District in 2004 and that Metropolitan Statistical Area has been included in this analysis since then.
In order to facilitate reporting of data at the community level, Utah has been divided into 61 small areas. “Small areas” refers to a set of 61 geographic areas in Utah with population sizes ranging from approximately 20,000 to 60,000 persons. These geographic areas are especially useful for doing public health assessment in communities within Utah’s urban counties. The Utah BRFSS has included a ZIP code question since 2001. The ZIP code and county information allow the data to be analyzed by Utah’s 61 Small Areas. Small areas were determined based on specific criteria including population size, political boundaries of cities and towns, and economic similarity. A few of the small areas are single county but most are determined using ZIP code. Starting in 2001 the Utah BRFSS has asked respondents for their ZIP code. The data for 2001 and 2002 were combined and analyzed in order to determine the number of interviews obtained in each of Utah’s small areas. Using this information, the Utah BRFSS sample was increased and adjusted by health district for 2004 so as to obtain at least 100 completed interviews in each small area over a three year period. This required an increase in completes state-wide from 4,000 to 5,000 per year. More information about Utah’s small areas is available at: http://health.utah.gov/opha/IBIShelp/sarea/SmallAreaAnalysis.htm.
Behavioral Risk Factor Surveillance System Operational and User's Guide, U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Available on line at http://ftp.cdc.gov/pub/Data/Brfss/userguide.pdf.
Further information about Utah’s BRFSS may be obtained by contacting Jennifer Wrathall, Utah BRFSS Coordinator, in the Office of Public Health Assessment, Utah Department of Health, P.O.Box 142101, Salt Lake City, UT 84114-2101. Phone: (801) 538-6434 FAX: (801) 538-9346. Email: firstname.lastname@example.org.