This open access book describes and reviews the development of the quality control mechanisms and methodologies associated with IEA’s extensive program of educational research. A group of renowned international researchers, directly involved in the design and execution of IEA’s international large-scale assessments (ILSAs), describe the operational and quality control procedures that are employed to address the challenges associated with providing high-quality, comparable data. Throughout the now considerable history of IEA’s international large-scale assessments, establishing the quality of the data has been paramount. Research in the complex multinational context in which IEA studies operate imposes significant burdens and challenges in terms of the methodologies and technologies that have been developed to achieve the stated study goals. The demands of the twin imperatives of validity and reliability must be satisfied in the context of multiple and diverse cultures, languages, orthographies, educational structures, educational histories, and traditions. Readers will learn about IEA’s approach to such challenges, and the methods used to ensure that the quality of the data provided to policymakers and researchers can be trusted. An often neglected area of investigation, namely the consequential validity of ILSAs, is also explored, examining issues related to reporting, dissemination, and impact, including discussion of the limits of interpretation. The final chapters address the question of the influence of ILSAs on policy and reform in education, including a case study from Singapore, a country known for its outstanding levels of achievement, but which nevertheless seeks the means of continual improvement, illustrating best practice use of ILSA data.

This handbook is the first of its kind to provide a general and comprehensive overview of virtually every aspect of International Large Scale Assessment (ILSA). It includes historical, economic, and policy perspectives, theoretical foundations, methodology, and reviews of findings from analyses of ILSA data. After decades, during which ILSAs have generated knowledge within central areas of education research and gained increased and substantial impact on educational policy, practice and research, such a broad overview for a wide-ranging audience is much needed. With contributions from authors and editors from all continents, this handbook appeals to an international audience and keeps a neutral perspective, not favoring one ILSA over another. The handbook is suitable to be read by politicians, researchers and stakeholders who are seeking an overview of ILSAs, their history and development, and both potential benefits and limitations with regard to policy implications. The reviews of findings from studies analyzing ILSA data will be of interest to stakeholders, teachers, researchers, and policymakers. Considering that the reviews extend to all fields pertaining to educational research, the book will be valuable to all researchers interested in education. Students may use the book to learn about ILSAs in the context of policy, theoretical underpinnings, or research. Moreover, the methodology section is written in a manner that is understandable and accessible for students, stakeholders, or researchers not familiar with these data. This methodology part, however, is also a valuable resource for researchers who are familiar with ILSA data, as it provides overviews of the design and sampling procedures of several ILSAs, and includes advice on methods of analysis.Even the owners of the ILSAs may find the book valuable, as it contains overviews and insights into a number of ILSAs, provides information how the data is used by the research community, and includes recommendations for future instruments.

The volume represents presentations given at the 86th annual meeting of the Psychometric Society, held virtually on July 19–23, 2021. About 500 individuals contributed paper presentations, symposiums, poster presentations, pre-conference workshops, keynote presentations, and invited presentations. Since the 77th meeting, Springer has published the conference proceedings volume from this annual meeting to allow presenters to share their work and ideas with the wider research community, while still undergoing a thorough review process. This proceedings covers a diverse set of psychometric topics, including item response theory, Bayesian models, reliability, longitudinal measures, and cognitive diagnostic models.

This book explores the often controversial international large-scale assessments (ILSAs) in education and offers research-based accounts of international testing as a social practice. Assessment exercises, such as the Organisation for Economic Co-operation and Development's Programme for International Student Assessment (PISA), produce comparable international statistics and rankings on educational performance, and are influential practices that shape educational policy on a global scale. The chapters in this volume, written by expert researchers in the field, take the reader behind the scenes to document a broad range of ILSA practices – from the recruitment of countries into ILSAs, to the production and performance of large-scale testing, and the management, media reception and use of test data. Based on data that is only available to expert researchers with inside access, the international case study material includes examples from Australia, Ecuador, Germany, Japan, Mexico, Norway, Russia, Scotland, Slovenia, Sweden, the UK and the USA. The volume provides important insights for teachers, researchers and policy-makers who use and study assessment data and who wish to evaluate its significance for educational policy and practice.

Technological and statistical advances, along with a strong interest in gathering more information about the state of our educational systems, have made it possible to assess more students, in more countries, more often, and in more subject domains. The Handbook of International Large-Scale Assessment: Background, Technical Issues, and Methods of Data Analysis brings together recognized scholars in the field of ILSA, behavioral statistics, and policy to develop a detailed guide that goes beyond database user manuals. After highlighting the importance of ILSA data to policy and research, the book reviews methodological aspects and features of the studies based on operational considerations, analytics, and reporting. The book then describes methods of interest to advanced graduate students, researchers, and policy analysts who have a good grounding in quantitative methods, but who are not necessarily quantitative methodologists. In addition, it provides a detailed exposition of the technical details behind these assessments, including the test design, the sampling framework, and estimation methods, with a focus on how these issues impact analysis choices.

The OECD has initiated PISA for Development (PISA-D) in response to the rising need of developing countries to collect data about their education systems and the capacity of their student bodies.

Des pays du monde entier se sont lancés dans de grandes évaluations nationales ou ont participé pour la première fois à de grandes évaluations régionales ou internationales dans le but d’améliorer leur systèmed’éducation. Lorsque les évaluations sont bien conçues, les informations crédibles qu’elles f ournissent au sujet de la performance des élèves peuvent contribuer à mieux répartir les ressources entre les établissements et à améliorer la qualité de l’enseignement et le rendement de l’apprentissage. Les services de la Banque mondiale ont rédigé le présent Manuel pour les évaluations à grande échelle des acquis scolaires dans le but de proposer un guide complet à ceux qui veulent en apprendre davantage sur la conception et l’administration de ce type d’évaluations et sur l’analyse et l’utilisation de leurs résultats. Cet ouvrage répond à des questions que se posent souvent les équipes chargées de grandes évaluations et les responsables politiques qui souhaitent prendre des décisions les concernant en toute connaissance de cause. Chaque chapitre aborde une étape du processus d’évaluation, prodigue des conseils, propose des orientations et donne des exemples de ce qui se fait dans des pays. Par ailleurs, cet ouvrage décrit les nouvelles tendances qui se dessinent dans les grandes évaluations et donne des informations à jour sur de grandes évaluations régionales et internationales. DIRK HASTEDT, Directeur exécutif de l’Association internationale pour l’évaluation du rendement scolaire (IEA) « Ce qu’il y a d’unique dans cet ouvrage, c’est qu’en plus de passer des spécifications techniques en revue, il donne des exemples de la façon dont des pays du monde entier administrent des évaluations à grande échelle, en tirent des enseignements et en utilisent les résultats. En ce sens, ce Manuel pour les évaluations à grande échelle des acquis scolaires est un excellent guide pratique à consulter pour obtenir une vue d’ensemble des grandes évaluations et mieux comprendre comment et pourquoi les administrer. » SILVIA MONTOYA, Directrice de l’Institut de statistique de l’UNESCO (ISU) « C’est l’ouvrage idéal pour les responsables d’évaluations scolaires à la recherche d’informations complètes, mais lisibles, sur le sujet. Il est extrêmement bien rédigé et parfaitement structuré, de sorte qu’il est facile à consulter et que tous les aspects y sont décrits clairement et avec concision. C’est l’ouvrage de référence par excellence pour tous ceux qui veulent explorer les étapes à enchaîner pour mener une grande évaluation, d’autant qu’il foisonne d’exemples sur toutes les grandes évaluations internationales. » ANDREAS SCHLEICHER, Directeur de la Direction de l’éducation et des compétences et conseiller spécial du Secrétaire général, chargé de la politique de l’éducation, de l’Organisation de coopération et de développement économiques (OCDE) « De nombreux pays ont décidé d’administrer des évaluations internationales pour comparer la qualité, l’équité et l’efficience de leur système d’éducation à des normes de référence. Mais retirent-ils vraiment de leurs efforts de planification, d’administration et d’analyse de quoi aider les élèves à mieux apprendre, les enseignants à mieux enseigner et les établissements à travailler plus efficacement ? Ce Manuel pour les évaluations à grande échelle des acquis scolaires aidera les responsables politiques et leurs équipes techniques à répondre à ces questions. » ANDREI VOLKOV, Directeur de l’Institut de stratégie publique, École de gestion de Moscou SKOLKOVO « En 2008, c’est en grande partie dans l’objectif d’améliorer la qualité de l’enseignement fondamental que le Fonds fiduciaire de l’assistance russe à l’éducation pour le développement (READ) a été créé. Le Fonds continue de donner l’exemple, c’est d’ailleurs la plus grande initiative russe visant à promouvoir l’évaluation des acquis scolaires. Les approches conçues à l’aide du Fonds, que ce soit celles visant à renforcer les capacités institutionnelles et spécialisées ou à orienter les réformes de l’éducation, ont prouvé leur efficacité dans de nombreux pays. Le Manuel pour les évaluations à grande échelle des acquis scolaires rassemble sous un format pratique les meilleures études de cas au sujet des évaluations à grande échelle. Cet ouvrage a un atout particulièrement important, celui de proposer un système intégré de renforcement des capacités, de sorte que c’est un guide pratique, facile à utiliser dans des contextes culturels différents. Nous espérons que cet ouvrage contribuera à faire connaître largement l’expérience collective rassemblée par le Fonds et nous rapprochera de la réalisation de l’objectif relatif à l’éducation dans le Programme de développement durable. »

To improve their education systems, countries around the world have increasingly initiated national largescale assessment programs or participated in international or regional large-scale assessment studies for the first time. Well-constructed large-scale assessments can provide credible information on student achievement levels, which, in turn, can promote better resource allocation to schools, stronger education service delivery, and improved learning outcomes. The World Bank developed this Primer on Large-Scale Assessments of Educational Achievement as a firststop resource for those wanting to understand how to design, administer, analyze, and use the results from these assessments of student achievement. The book addresses frequently asked questions from people working on large-scale assessment projects and those interested in making informed decisions about them. Each chapter introduces a stage in the assessment process and offers advice, guidelines, and country examples. This book also reports on emerging trends in large-scale assessment and provides updated information on regional and international large-scale assessment programs. DIRK HASTEDT, Executive Director of the International Association for the Evaluation of Educational Achievement (IEA) “A special feature of the publication is that it not only gives an overview of technical specifications, but also includes examples from around the world on how countries are conducting large-scale assessments, what they found, and how the results were used. With this perspective, the Primer on Large-Scale Assessments of Educational Achievement is an excellent and easy-to-read publication to get a comprehensive overview of large-scale assessments and how and why they are conducted.†? SILVIA MONTOYA, Director of UNESCO Institute for Statistics (UNESCO UIS) “If you are responsible for learning assessment in a country and are searching for a comprehensive, yet readable, guide on large-scale assessment, this is your book. Extremely well structured and written, this primer is easy to follow, and makes points clearly and concisely. It is an excellent resource that explores the steps for a good large-scale assessment with examples from all international large-scale assessment programs.†? ANDREAS SCHLEICHER, Director for the Directorate of Education and Skills and Special Advisor on Education Policy to the Organization for Economic Cooperation and Development’s (OECD) Secretary- General “Many countries have joined international educational assessments to benchmark quality, equity, and efficiency in their education systems. But what does it take to design and implement those efforts well and to draw value from this to help students learn better, teachers teach better, and schools to work more effectively? This Primer on Large-Scale Assessments of Educational Achievement helps policy makers and their technical teams to find answers to these questions.†? ANDREI VOLKOV, Director of the Institute for Public Strategy, Moscow School of Management SKOLKOVO “In 2008, when the Russia Education Aid for Development (READ) Program was launched, we determined its main goal as the improvement of the quality of basic education. Today, the READ Program keeps setting trends as the largest Russian initiative promoting educational assessment. Approaches developed within the READ Program, from building institutional and expert capacity to influencing educational reforms, have proven their efficacy in many countries. The Primer on Large-Scale Assessments of Educational Achievement brings together in a practical format the best experience and case studies in conducting assessments under the READ Program. An especially important feature of the book is an integrated capacity building component, which makes it a practical tutorial ready for use in different cultural contexts. Through this book, we hope that our collective experience gathered during READ will be widely shared, bringing us closer to achievement of the Sustainable Development Goal on education.†? evelopment Goal on Education.†?