A practical, one-stop reference on the theory and applications of statistical data editing and imputation techniques Collected survey data are vulnerable to error. In particular, the data collection stage is a potential source of errors and missing values. As a result, the important role of statistical data editing, and the amount of resources involved, has motivated considerable research efforts to enhance the efficiency and effectiveness of this process. Handbook of Statistical Data Editing and Imputation equips readers with the essential statistical procedures for detecting and correcting inconsistencies and filling in missing values with estimates. The authors supply an easily accessible treatment of the existing methodology in this field, featuring an overview of common errors encountered in practice and techniques for resolving these issues. The book begins with an overview of methods and strategies for statistical data editing and imputation. Subsequent chapters provide detailed treatment of the central theoretical methods and modern applications, with topics of coverage including: Localization of errors in continuous data, with an outline of selective editing strategies, automatic editing for systematic and random errors, and other relevant state-of-the-art methods Extensions of automatic editing to categorical data and integer data The basic framework for imputation, with a breakdown of key methods and models and a comparison of imputation with the weighting approach to correct for missing values More advanced imputation methods, including imputation under edit restraints Throughout the book, the treatment of each topic is presented in a uniform fashion. Following an introduction, each chapter presents the key theories and formulas underlying the topic and then illustrates common applications. The discussion concludes with a summary of the main concepts and a real-world example that incorporates realistic data along with professional insight into common challenges and best practices. Handbook of Statistical Data Editing and Imputation is an essential reference for survey researchers working in the fields of business, economics, government, and the social sciences who gather, analyze, and draw results from data. It is also a suitable supplement for courses on survey methods at the upper-undergraduate and graduate levels.

A practical, one-stop reference on the theory and applications of statistical data editing and imputation techniques Collected survey data are vulnerable to error. In particular, the data collection stage is a potential source of errors and missing values. As a result, the important role of statistical data editing, and the amount of resources involved, has motivated considerable research efforts to enhance the efficiency and effectiveness of this process. Handbook of Statistical Data Editing and Imputation equips readers with the essential statistical procedures for detecting and correcting inconsistencies and filling in missing values with estimates. The authors supply an easily accessible treatment of the existing methodology in this field, featuring an overview of common errors encountered in practice and techniques for resolving these issues. The book begins with an overview of methods and strategies for statistical data editing and imputation. Subsequent chapters provide detailed treatment of the central theoretical methods and modern applications, with topics of coverage including: Localization of errors in continuous data, with an outline of selective editing strategies, automatic editing for systematic and random errors, and other relevant state-of-the-art methods Extensions of automatic editing to categorical data and integer data The basic framework for imputation, with a breakdown of key methods and models and a comparison of imputation with the weighting approach to correct for missing values More advanced imputation methods, including imputation under edit restraints Throughout the book, the treatment of each topic is presented in a uniform fashion. Following an introduction, each chapter presents the key theories and formulas underlying the topic and then illustrates common applications. The discussion concludes with a summary of the main concepts and a real-world example that incorporates realistic data along with professional insight into common challenges and best practices. Handbook of Statistical Data Editing and Imputation is an essential reference for survey researchers working in the fields of business, economics, government, and the social sciences who gather, analyze, and draw results from data. It is also a suitable supplement for courses on survey methods at the upper-undergraduate and graduate levels.

A comprehensive guide to automated statistical data cleaning The production of clean data is a complex and time-consuming process that requires both technical know-how and statistical expertise. Statistical Data Cleaning brings together a wide range of techniques for cleaning textual, numeric or categorical data. This book examines technical data cleaning methods relating to data representation and data structure. A prominent role is given to statistical data validation, data cleaning based on predefined restrictions, and data cleaning strategy. Key features: Focuses on the automation of data cleaning methods, including both theory and applications written in R. Enables the reader to design data cleaning processes for either one-off analytical purposes or for setting up production systems that clean data on a regular basis. Explores statistical techniques for solving issues such as incompleteness, contradictions and outliers, integration of data cleaning components and quality monitoring. Supported by an accompanying website featuring data and R code. This book enables data scientists and statistical analysts working with data to deepen their understanding of data cleaning as well as to upgrade their practical data cleaning skills. It can also be used as material for a course in data cleaning and analyses.

Missing data pose challenges to real-life data analysis. Simple ad-hoc fixes, like deletion or mean imputation, only work under highly restrictive conditions, which are often not met in practice. Multiple imputation replaces each missing value by multiple plausible values. The variability between these replacements reflects our ignorance of the true (but missing) value. Each of the completed data set is then analyzed by standard methods, and the results are pooled to obtain unbiased estimates with correct confidence intervals. Multiple imputation is a general approach that also inspires novel solutions to old problems by reformulating the task at hand as a missing-data problem. This is the second edition of a popular book on multiple imputation, focused on explaining the application of methods through detailed worked examples using the MICE package as developed by the author. This new edition incorporates the recent developments in this fast-moving field. This class-tested book avoids mathematical and technical details as much as possible: formulas are accompanied by verbal statements that explain the formula in accessible terms. The book sharpens the reader’s intuition on how to think about missing data, and provides all the tools needed to execute a well-grounded quantitative analysis in the presence of missing data.

ADMINISTRATIVE RECORDS FOR SURVEY METHODOLOGY Addresses the international use of administrative records for large-scale surveys, censuses, and other statistical purposes Administrative Records for Survey Methodology is a comprehensive guide to improving the quality, cost-efficiency, and interpretability of surveys and censuses using administrative data research. Contributions from a team of internationally-recognized experts provide practical approaches for integrating administrative data in statistical surveys, and discuss the methodological issues—including concerns of privacy, confidentiality, and legality—involved in collecting and analyzing administrative records. Numerous real-world examples highlight technological and statistical innovations, helping readers gain a better understanding of both fundamental methods and advanced techniques for controlling data quality reducing total survey error. Divided into four sections, the first describes the basics of administrative records research and addresses disclosure limitation and confidentiality protection in linked data. Section two focuses on data quality and linking methodology, covering topics such as quality evaluation, measuring and controlling for non-consent bias, and cleaning and using administrative lists. The third section examines the use of administrative records in surveys and includes case studies of the Swedish register-based census and the administrative records applications used for the US 2020 Census. The book’s final section discusses combining administrative and survey data to improve income measurement, enhancing health surveys with data linkage, and other uses of administrative data in evidence-based policymaking. This state-of-the-art resource: Discusses important administrative data issues and suggests how administrative data can be integrated with more traditional surveys Describes practical uses of administrative records for evidence-driven decisions in both public and private sectors Emphasizes using interdisciplinary methodology and linking administrative records with other data sources Explores techniques to leverage administrative data to improve the survey frame, reduce nonresponse follow-up, assess coverage error, measure linkage non-consent bias, and perform small area estimation. Administrative Records for Survey Methodology is an indispensable reference and guide for statistical researchers and methodologists in academia, industry, and government, particularly census bureaus and national statistical offices, and an ideal supplemental text for undergraduate and graduate courses in data science, survey methodology, data collection, and data analysis methods.

Missing data form a problem in every scientific discipline, yet the techniques required to handle them are complicated and often lacking. One of the great ideas in statistical science—multiple imputation—fills gaps in the data with plausible values, the uncertainty of which is coded in the data itself. It also solves other problems, many of which are missing data problems in disguise. Flexible Imputation of Missing Data is supported by many examples using real data taken from the author's vast experience of collaborative research, and presents a practical guide for handling missing data under the framework of multiple imputation. Furthermore, detailed guidance of implementation in R using the author’s package MICE is included throughout the book. Assuming familiarity with basic statistical concepts and multivariate methods, Flexible Imputation of Missing Data is intended for two audiences: (Bio)statisticians, epidemiologists, and methodologists in the social and health sciences Substantive researchers who do not call themselves statisticians, but who possess the necessary skills to understand the principles and to follow the recipes This graduate-tested book avoids mathematical and technical details as much as possible: formulas are accompanied by a verbal statement that explains the formula in layperson terms. Readers less concerned with the theoretical underpinnings will be able to pick up the general idea, and technical material is available for those who desire deeper understanding. The analyses can be replicated in R using a dedicated package developed by the author.

The research and its outcomes presented here focus on spatial sampling of agricultural resources. The authors introduce sampling designs and methods for producing accurate estimates of crop production for harvests across different regions and countries. With the help of real and simulated examples performed with the open-source software R, readers will learn about the different phases of spatial data collection. The agricultural data analyzed in this book help policymakers and market stakeholders to monitor the production of agricultural goods and its effects on environment and food safety.

This volume brings together a selection of papers presented at the 2017 European Establishment Statistics Workshop, which have been revised and expanded here. Several contributions will serve to deepen the reader’s understanding of the unit problem in business statistics, while further chapters showcase recent advances in business survey methodology and practice in areas such as linking and data integration, sampling and estimation, data collection from businesses, measurement and mitigation of response burden in business surveys, among others. Written by leading experts in business statistics, the volume offers detailed and up-to-date findings to survey methodologists and practitioners working with business statistics. It will also be useful for readers in official statistics, academia and the private sector.

"This is a valuable reference guide for readers interested in gaining a basic understanding of probability theory or its applications in problem solving in the other disciplines." —CHOICE Providing cutting-edge perspectives and real-world insights into the greater utility of probability and its applications, the Handbook of Probability offers an equal balance of theory and direct applications in a non-technical, yet comprehensive, format. Editor Tamás Rudas and the internationally-known contributors present the material in a manner so that researchers of various backgrounds can use the reference either as a primer for understanding basic probability theory or as a more advanced research tool for specific projects requiring a deeper understanding. The wide-ranging applications of probability presented make it useful for scholars who need to make interdisciplinary connections in their work. Key Features Contains contributions from the international who′s-who of probability across several disciplines Offers an equal balance of theory and applications Explains the most important concepts of probability theory in a non-technical yet comprehensive way Provides in-depth examples of recent applications in the social and behavioral sciences as well as education, business, and law Intended Audience This Handbook makes an ideal library purchase. In addition, this volume should also be of interest to individual scholars in the social and behavioral sciences.

Data privacy technologies are essential for implementing information systems with privacy by design. Privacy technologies clearly are needed for ensuring that data does not lead to disclosure, but also that statistics or even data-driven machine learning models do not lead to disclosure. For example, can a deep-learning model be attacked to discover that sensitive data has been used for its training? This accessible textbook presents privacy models, computational definitions of privacy, and methods to implement them. Additionally, the book explains and gives plentiful examples of how to implement—among other models—differential privacy, k-anonymity, and secure multiparty computation. Topics and features: Provides integrated presentation of data privacy (including tools from statistical disclosure control, privacy-preserving data mining, and privacy for communications) Discusses privacy requirements and tools for different types of scenarios, including privacy for data, for computations, and for users Offers characterization of privacy models, comparing their differences, advantages, and disadvantages Describes some of the most relevant algorithms to implement privacy models Includes examples of data protection mechanisms This unique textbook/guide contains numerous examples and succinctly and comprehensively gathers the relevant information. As such, it will be eminently suitable for undergraduate and graduate students interested in data privacy, as well as professionals wanting a concise overview. Vicenç Torra is Professor with the Department of Computing Science at Umeå University, Umeå, Sweden.