Challenges in a Federal Statistical Agency Ecosystem: The U.S. Census Bureau

This paper describes the Census Bureau investment of resources in a new endeavor, the Open Census Project. The Census Bureau aims to become a leader among Federal statistical agencies in the US Government’s platforms for open science through its Open Census project. The Open Census Project will increase transparency, accessibility, collaboration, and trust for all public, federally funded Census Bureau statistical products, research, and software. Doing so will require the development of an intuitive and secure ecosystem for Census Bureau researchers, including software engineers and statisticians, to disseminate their work and associated processes, driving the highest standards of research integrity and transparency.

Recent developments in data visualization, developer Application Program Interfaces (APIs), and web services reinforce a long American tradition of statistical mapping and innovation at the US Census Bureau. Consistent with other international statistical agencies, the Census Bureau has used contemporary innovations in statistical mapping and data visualization to disseminate national census results for over 14 decades. The US Census Bureau’s data products and analyses have enabled decision makers and the public to access census results quickly and easily. The new information technology environment requires the Census Bureau to more rapidly expedite these results and deliver mapping products to new customers as well as to its traditional data consumers. The Census API has empowered developers and commercial companies to test the limits of a new emerging world of big data solutions. This article presents some of the most recent data visualization products from the Census Bureau, including an expanding Data Visualization Gallery to merge geospatial information and statistics on the map.

Information distribution practices of federal statistical agencies: The census bureau example

Towards Unrestricted Public Use Business Microdata: The Synthetic Longitudinal Business Database

Our purpose here is to set forth certain principles for the design and appraisal of statistical information systems which describe economic and social organizations and their environments-households, industrial and commercial firms. governments. As such we are primarily concerned with the evaluation of systematic procedures for producing descriptive statistics in and of organizations; we concentrate attention on Federal statistics, though the principles are not limited to this field of application. This is a broad area to which the disciplines of probability and mathematical statistics have made significant contributions for more than two centuries-theoretical contributions which have been regularly tested by experience. There is, therefore, little need to expound on the secure but limited contributions of the principles of probability sampling, statistical inference, and experimental design to the area of descriptive statistics. In actual fact, it is easy to show that descriptive statistics (of economic and social organizations) also involve certain issues of system compatibility and complexity which are not customary topics in probability and mathematical statistics. Some of the issues may be quickly illustrated in terms of a statistical allegory: one is handed a copy of the very valuable Statistical Abstract of the United States, published by the Bureau of the Census on behalf of the statistical agencies of the Federal Government, and is asked to explain how the various tables might be rationally fitted together into coherent and consistent components of one or more information systems. The problem here is not whether rational processes of administration led to the production of a given set of statistical tables, since this can be readily established in almost all cases. The characteristic problem is rather one of the following character: given the basic properties of the data underlying the tables, which have been supplied by a number of statistical reporting systems of one or more governmental agencies (Federal, state, or local), do there exist one or more conceptual frameworks or models which would systematically organize and interrelate the whole set of statistical tables belonging to a well-defined subject matter field in a constructive way; in such a subject matter field, do there exist feasible methods of transformation for reconciling one statistical table with another statistical table? As a case in point, how does one transform a frequency distribution of consumer income based on a sample survey of households into a frequency distribution of taxable income classified by taxpayers? The technical answers to problems of this type usually take the form that even when such transformations are held to be feasible by some scientific community, the computations involved may be massive or elaborate or sometimes both at once and, consequently, costly or timeconsuming. We have illustrated by means of a fable, the fundamental problem typically confronting the serious user of Federal statistics-in industry, in the academic world, or in Federal, state, and local governments: the conversion of resources of time, money, knowledge, and influence into a rational fitting together of available statistical data from diverse sources to meet the constraints of the problem, case, or presentation. Where such efforts in reorganization or transformation of available statistical data must fail on technical grounds, the serious user can undertake to sponsor and/or support the required organization of data by means of a new statistical reporting system. This may be the most reasonable solution of the potential user's immediate problem; especially so, if the totality of all such new reporting systems can be so designed as to advance the feasibility of practicable reorganizations of statistical data in the future.

Census statistics play a key role in public policy decisions and social science research. However, given the risk of revealing individual information, many statistical agencies are considering disclosure control methods based on differential privacy, which add noise to tabulated data. Unlike other applications of differential privacy, however, census statistics must be postprocessed after noise injection to be usable. We study the impact of the U.S. Census Bureau’s latest disclosure avoidance system (DAS) on a major application of census statistics, the redrawing of electoral districts. We find that the DAS systematically undercounts the population in mixed-race and mixed-partisan precincts, yielding unpredictable racial and partisan biases. While the DAS leads to a likely violation of the “One Person, One Vote” standard as currently interpreted, it does not prevent accurate predictions of an individual’s race and ethnicity. Our findings underscore the difficulty of balancing accuracy and respondent privacy in the Census.

The topic of the role of privacy and confidentiality in the use of administrative records for statistical purposes is an important one. In providing comments, I will draw extensively from my experience in working with fifty states and six federal agencies in creating the LEHD (Longitudinal Employer-Household Dynamics) program at the Census Bureau between 1997 (when I started my ASA fellowship) and 2004 (when the program became a fully-fledged Census Bureau program). However, I will also draw on my experience in building the LEED (Linked Employer-Employee Data) program at Statistics New Zealand, as well as my experience working with the data producers from many countries (both statistical agencies and administrative agencies) while in my World Bank consultancy. The final context is my experience with cyberinfrastructure initiatives within the United States, as well as in Europe.

Morris Hansen was the most influential statistician in the evolution of survey methodology in the twentieth century. Early in his career at the Census Bureau he put together and directed a staff of mathematical statisticians and other survey methodologists whose aim was to systematically define the principal problems in the conduct of surveys, carry out research on these problems, and develop the statistical methods necessary to overcome them. This work included the development of sampling theory necessary for the efficient conduct of large-scale national surveys, the establishment of formal quality control methods for surveys, and the derivation of theory and models for analyses of nonsampling errors. Hansen prodded the Census Bureau into accepting such innovations as the purchase of the first computer for statistical purposes, the development of optical scanning equipment, the introduction of self-enumeration and mail in both demographic and economic censuses, and other survey techniques now commonly used by both government and private organizations. He anticipated the concern that would arise over the completeness of coverage in decennial population censuses and, long before the current interest in the subject, he persuaded the bureau to adopt procedures designed to improve coverage; and he directed research on coverage problems. Both the statistical methods he and his staff developed and the form of research staff he organized had a profound effect on statistical agencies all over the world.

The state of the art in signal extraction gradually evolved from the use of a mechanical form of moving average filters to the present sophisticated model-based techniques capable of performing automatic modeling and signal extraction involving hundreds or even thousands of time series in one production run. The leading edge of technology is being shared by two ARIMA model-based systems: ARIMA X12 of the US Bureau of Census and the twin programs TRAMO-SEATS developed at the Bank of Spain. These specialized expert systems have been adopted by most statistical agencies of advanced OECD countries and the European community. The Philippines on the other hand is using the ARIMA X11 system modified by the Bank of Canada in its routine seasonal adjustment and time series decomposition tasks. This study is an attempt to implement the ARIMA model-based (AMB) approach of extracting unobserved signals from 194 quarterly national accounts statistics of the Philippines using the TRAMO-SEATS system in a fully automatic modeling mode. The successful result of the application adequately demonstrates the feasibility of adopting a system being used routinely by countries in more advanced economies.

SummaryNational statistical agencies lack statistical methodology to express uncertainty in their released estimated overall rankings. For example, the US Census Bureau produced an ‘explicit’ ranking of the states based on observed sample estimates during 2011 of mean travel time to work. Current literature provides measures of uncertainty in estimated individual ranks, but not a direct measure of uncertainty for the estimated overall ranking. We construct and visualize a joint confidence region for the true unknown overall ranking that provides a measure of uncertainty in the estimated overall ranking.

Publicly available statistics from government agencies that are credible, relevant, accurate, and timely are essential for policy makers, individuals, households, businesses, academic institutions, and other organizations to make informed decisions. Even more, the effective operation of a democratic system of government depends on the unhindered flow of statistical information to its citizens. In the United States, federal statistical agencies in cabinet departments and independent agencies are the governmental units whose principal function is to compile, analyze, and disseminate information for such statistical purposes as describing population characteristics and trends, planning and monitoring programs, and conducting research and evaluation. The work of these agencies is coordinated by the U.S. Office of Management and Budget. Statistical agencies may acquire information not only from surveys or censuses of people and organizations, but also from such sources as government administrative records, private-sector datasets, and Internet sources that are judged of suitable quality and relevance for statistical use. They may conduct analyses, but they do not advocate policies or take partisan positions. Statistical purposes for which they provide information relate to descriptions of groups and exclude any interest in or identification of an individual person, institution, or economic unit. Four principles are fundamental for a federal statistical agency: relevance to policy issues, credibility among data users, trust among data providers, and independence from political and other undue external influence. Principles and Practices for a Federal Statistical Agency: Fifth Edition explains these four principles in detail.

The Committee on National Statistics at the National Academy of Sciences/National Research Council first published Principles and Practices for a Federal Statistical Agency in 1992 in response to requests from Congress and others about what constitutes an effective statistical agency. Beginning in 2001, the committee has regularly updated P&P every 4 years so that a new edition is available for incoming political officials. The document helps protect the independence and objectivity of federal statistics in the highly decentralized U.S. statistical system. This article begins with a brief history of the federal statistical system and the difficulties that its decentralized structure presents for innovation and coordination, followed by an overview of the committee. The article then discusses the why and what of the first through fifth editions of P&P, highlighting major changes in each edition to respond to changes in society, the legal environment, and statistical methods and data collection techniques. The article concludes that P&P has been of value to the federal statistical agencies, to the Office of Management and Budget, and in congressional oversight.

When a statistical agency, such as the United States Bureau of the Census, publishes tabular data, it must withhold certain data elements that contain confidential information associated with the data respondents. Cell suppression is a technique commonly used in the publishing of economic data in tabular formats. The sensitive entries, which are called primary suppressions, need to be suppressed. However, the suppression of the primary cells alone still allows one to estimate a range for each of the missing values by considering the published entries. Additional entries in the table must be suppressed to ensure that these ranges are not too narrow. Traditionally, the protection required is defined by an interval centered around the value of each primary suppression. This paper formulates and develops solution techniques for the traditional problem as well as for a relaxation of the problem where sliding protection ranges are allowed. In the relaxed problem, the protection ranges have fixed widths but are free to slide; the only restriction is that each range must contain the value of the primary suppression. We present network flow‐based heuristics for both versions of the cell suppression problem and use a lower‐bounding procedure to evaluate the performance of the heuristics. Extensive computational results based on real‐world and randomly generated tables demonstrate that sliding protection ranges can significantly reduce the total amount of suppressed data, as compared to the traditional suppression scheme. Furthermore, the heuristics produce optimal or near‐optimal solutions for real‐world problems.

Federal statistical agencies are keenly aware of the need for data objectivity, which requires that data be accurate, reliable, and unbiased and that they be is presented in an accurate, clear, and unbiased manner. To be trusted, they need to be presented in a clearly nonpartisan manner. No one realized this more nor did more to advance the reputation for objectivity of the critical employment, price and wage data produced by the U.S. Bureau of Labor Statistics (BLS) than its tenth commissioner, Janet L. Norwood. Dr. Norwood passed away from complications of Alzheimer’s disease on March 27, 2107 in Austin, Texas at the age of 91. During her career at BLS and in years after her service as Commissioner, Janet Norwood had many platforms for advancing the quality, integrity, reliability and objectivity of Federal statistics. She came up through the ranks at BLS, mainly in the Bureau’s price programs and was Deputy Commissioner when, in May 1979 after 16 years in BLS, she was confirmed as Commissioner following the death in office of Julius Shiskin and a short period of service as Acting Commissioner. Only one other commissioner had previously come up through the ranks, and until Janet Norwood, no Commissioner had been a female – though one female had served as acting commissioner in the 1960s. Other firsts marked her career. She was appointed by a Democratic President (Carter) and twice reappointed by a Republican (Reagan). As Commissioner of Labor Statistics, with service from 1979 to 1991, her public contribution to the objectivity of the data centered around monthly open testimony on the jobs report, usually on the first Friday of each month, before the Joint Economic Committee of Congress. These monthly employment reports are among the earliest, most closely watched and politically charged of the federal government’s indicators of the health of the economy. The hearings had an ignoble beginning, introduced in 1971 during the tenure of her predecessor, Geoffrey Moore when the then-Secretary of Labor dictated the end of monthly press conferences at which the Commissioner explained the data. The Joint Economic Committee stepped into the breach and began this series of monthly testimonies to give the Commissioner a platform for an impartial discussion of the statistics. Janet Norwood took full advantage of the opportunity to go to the Congress and the public each month, eventually logging 137 hearings over her 12-year tenure. Her analysis in these hearings was precise and crisp and she took pains to avoid interpretation that might be perceived as political in tone, often side stepping efforts by members of one party or the other to add flavor to the data releases. Many considered these monthly appearances as being dry, technical affairs, but the jousting over the meaning of the data often made for high theater. She excelled at walking a fine line and sticking to the story that the data told. Her nonpartisanship and demeanor engendered trust, and that trust extended to the data themselves, even though on a monthto-month basis, the estimates sometime looked wacky and were hard to explain. More than surviving and enhancing the hearings however, she made her mark on the Bureau by initiating programs to protect and enhance the quality and scope of the Bureau’s core programs. She did it despite widespread budget cuts duringmost of her tenure. Her major contribution was in the significant improvements made to the sensitive Consumer Price Program (CPI) during her years. It was a time when inflation was rising substantially and great attention was paid to the CPI. There was much controversy over the techni-

In 2011, the European Statistical System Committee – a committee of the European Commission (EC) – adopted the European Statistics Code of Practice (http: //ec.europa.eu/eurostat/web/products-manuals-and-gui delines/-/KS-32-11-955) becoming the first of three major organizations to issue new policy guidance in as many years for national statistical agencies. The United Nations approved its Fundamental Principles of Official Statistics (http://unstats.un.org/unsd/dnss/gp /fundprinciples.aspx) in 2013 and the United States White House Office of Management and Budget Office became the third in November 2014 with its issuing of Fundamental Responsibilities of Federal Statistical Agencies and Recognized Statistical Units. (http: //www.gpo.gov/fdsys/pkg/FR-2014-12-02/pdf/201428326.pdf). These three actions collectively affirm the importance of official statistical data to the governance, economy, and democracy of countries around the world. They also affirm the responsibilities of the dedicated statisticians, economists, demographers, sociologists and other professionals working to produce official statistics. Each of the three documents represents an updating of previous guidance. The 2011 European Statistics Code of Practice is a revision of the original 2005 document. The U.N. first adopted its Fundamental Principles of Official Statistics in 1994 but revised and updated the preamble for the 2013 release. The U.S. document – also referred to as Statistical Policy Directive Number 1 – combines previous guidance with new guidance into a unified policy statement, while also expanding its application to three agencies beyond the thirteen primary U.S. federal agencies. The documentation accompanying the U.S. directive’s issuance cites both the UN Fundamental Principles of Official Statistics and European Statistics Code of PracTable 1 Ten (10) UN Fundamental Principles of Official Statistics