Information and Advice on 2020 Federal Data Quality and Use

How has the COVID-19 pandemic disrupted data collection and what are the ramifications for data quality?

August 3, 2022 Alicia VanOrman Senior Research Associate
PDF. Information and Advice on 2020 Federal Data Quality and Use

While the COVID-19 pandemic has disrupted daily routines, schooling, work, health, and the economy around the world, it has also significantly impacted U.S. data collection for annual federal surveys and the release of statistics on the well-being of children and families. The disruptions in data collection limit our understanding of how the pandemic impacted families in 2020 and will have ramifications for data quality and availability for the next several years.

All major federal surveys implemented methodological changes in response to the COVID-19 pandemic. However, federal agencies faced different challenges and choices based on how their surveys were originally designed. Some agencies continued data collection efforts by shifting from in-person to phone or online interviewing modes while others suspended operations or delayed or extended data collection periods. New surveys, such as the Household Pulse Survey, were also developed to provide up-to-date information about how the pandemic was affecting the U.S. population, families, and children.

This brief summarizes changes to several U.S. Census Bureau surveys and programs that provide data on children and families—the American Community Survey (ACS), Current Population Survey (CPS), 2020 Census, and Population Estimates—as well as how to access data, evaluate data quality and usability, and what to do if the data you typically rely on are not available. A short outline of these changes, as well as information about many additional data sources, is available in a companion table, The COVID-19 Pandemic’s Impact on Federal Statistical Systems Data Collection and Data Quality.

Key Takeaways and Recommendations

American Community Survey (ACS)

• The Census Bureau did not release the standard 2020 ACS 1-year data products. Instead, it produced experimental estimates.
• The 2020 ACS 1-year experimental data release included selected tabulated data for the United States, the 50 states and the District of Columbia, and a 1-year Public Use Microdata Sample (PUMS) data set with the experimental weights.
• The 2020 ACS 1-year experimental data should not be compared to any other ACS data or decennial census data.
• Estimates for Public Use Microdata Areas (PUMAs), which can combine or split counties, should be used with caution as the experimental weights are not optimized to produce estimates for these areas.
• The 2016-2020 ACS 5-year data were released under a waiver process because they did not meet the Census Bureau’s quality standards.
• ACS 5-year data users need to pay close attention to the margins of error, which may be substantially larger than usual for all data releases that include the 2020 data (that is, the 2016-2020 through the 2020-2024 data release).
• Data users who require a 1-year estimate should use 2019 data.
• Data users who require a 5-year estimate can use 2016-2020 data. These data can be compared to prior non-overlapping 5-year data (that is, 2011-2015).

Current Population Survey

• Data users should use caution when comparing results from the 2020 Annual Social and Economic Survey (ASEC) with prior years and can refer to the 2020 ASEC documentation for more information.

2020 Census

Redistricting data file (P.L.-94) recommendations include:

• Data users should consider aggregating small geographic areas to larger geographic areas with population sizes of 500 or more (for example, combine multiple census block groups together).
• Data users should not divide data across tables (for example, do not divide the population by the number of housing units to obtain the average household size).
• Data users may subtract data across tables (for example, subtract adult population from the total population to obtain a count of children).
• 2020 Census results can generally be compared to prior censuses and ACS data.

Demographic and Housing Characteristics Tables

• Those who are looking for data that have not yet been released (for example, data for 5-year age groups by sex and race) can first look to the Population Estimates Program to see if the desired data are available in the population estimates.
• If the geographic area or characteristics are not available in Population Estimates, then data users can use the 2016-2020 ACS 5-year data. Please note that these data are period estimates covering 2016-2020 and data users should pay close attention to the margins of error.

U.S. Population Estimates

• Vintage 2021 population estimates were produced using a blended base that incorporates data from multiple sources.
• Data users should pay attention to trends over time in case the Vintage 2021 estimates are substantially higher or lower than the 2020 Census data or Vintage 2020 estimates.

READ THE FULL REPORT

Frequently Asked Questions

Where can I find estimates of poverty rates, income, and other measures of economic well-being for 2020?

• For state-level estimates, use the 2021 CPS ASEC estimates. These data should provide reliable estimates at the state-level for large populations (all people, all children, etc.). Note that the poverty estimates from the CPS ASEC are not the same as from the ACS because of differences in how information about income is collected. PRB wrote an article summarizing the key differences between CPS ASEC estimates and ACS data.
• Economic data are included in the 2020 ACS 1-year experimental estimates, though data users will likely need to use the PUMS data file to develop estimates. These data cannot be compared to prior years, so they are not useful for understanding trends in poverty rates and other measures of economic well-being.
• The 2016-2020 ACS 5-year estimates are available for small geographic areas and can be used to disaggregate data by race and ethnicity. These data can be used for trend analyses when compared to another non-overlapping 5-year period and are available in tables on census.gov and through the PUMS data file.
• The 2019 ACS 1-year estimates can continue to be used.
• Small Area Income and Poverty Estimates (SAIPE) for 2020 are available. The U.S. Census Bureau did not provide updated methodological information, so the 2020 ACS data were likely incorporated into the model as usual.
• The Household Pulse Survey provides data on economic well-being during the COVID-19 pandemic. These data should be used with caution as they are experimental and do not have a pre-pandemic benchmark.

Where can I find estimates of health insurance coverage for 2020?

• For state-level estimates, use the 2021 CPS ASEC estimates. Note that health insurance estimates from the CPS ASEC are not comparable to those from the ACS. See a summary of differences between CSP ASEC estimates and ACS data by PRB for more information.
• Health insurance estimates are included in the 2020 ACS 1-year experimental estimates, though data users will likely need to use the PUMS data file to develop estimates. These data cannot be compared to prior years, so they are not useful for understanding trends.
• The 2016-2020 ACS 5-year estimates are available for small geographic areas and can be used to disaggregate data by race and ethnicity. These data can be used for trend analyses when compared to another non-overlapping 5-year period and are available in tables on census.gov and through the PUMS data file.
• The 2019 ACS 1-year estimates can continue to be used.
• Small Area Health Insurance Estimates (SAHIE) are planned to be released in summer 2022.
• The State Health Access Data Assistance Center (SHADAC) produced state-level estimates for 2020 using the CPS ASEC data.

Where can I find 2020 birth data?

• The pandemic did not impact the collection of birth data. Use usual sources and methods, such as your state Department of Health office, National Center for Health Statistics reports, and CDC Wonder.

Where can I find 2020 mortality data?

• The pandemic did not impact the collection of mortality data. Use usual sources and methods, such as your state Department of Health office, National Center for Health Statistics reports, and CDC Wonder.

Where can I find additional child health data?

• One option is to use data from the National Survey of Children’s Health (NSCH). For state-level estimates, data users should pool two to three years of data together.
• The COVID-19 pandemic had no impact on response rates or data quality for the 2020 data.

Where can I access education-related statistics in 2020?

• For updating trends, data users will need to determine if the original data provider made any changes in 2020.For state-provided administrative data or state test scores obtained through states’ Departments of Education, data users will need to find out how the state handled attendance records, state-testing, etc., when schools were closed, in remote-only learning modes, and hybrid-learning modes.
• For data submitted to the federal government and released through the National Center of Education Statistics (NCES): NCES has not announced what data will be available or any concerns about data quality; however, some 2019-2020 school-year data are delayed relative to prior year release dates.
• For achievement data through National Assessment for Educational Progress (NAEP): The 2021 assessments were delayed until 2022. New data are expected in late 2022.
• The Household Pulse Survey included some questions about the delivery and receipt of education over the course of the pandemic. Data users should be aware that these data are experimental and that the Household Pulse Survey has a very low response rate (less than 5%). The questions also changed and have not been asked in all phases of the survey.
• Data from the ACS and the CPS can also be used for school enrollment and educational attainment data.