Howard Wainer

This authoritative guide directs consumers and users of test scores on when and how to provide subscores and how to make informed decisions based on them. The book is designed to be accessible to practitioners and score users with varying levels of technical expertise, from executives of testing organizations and students who take tests to graduate students in educational measurement, psychometricians, and test developers. The theoretical background required to evaluate subscores and improve them are provided alongside examples of tests with subscores to illustrate their use and misuse. The first chapter covers the history of tests, subtests, scores, and subscores. Later chapters go into subscore reporting, evaluating and improving the quality of subscores, and alternatives to subscores when they are not appropriate. This thorough introduction to the existing research and best practices will be useful to graduate students, researchers, and practitioners.

This authoritative guide directs consumers and users of test scores on when and how to provide subscores and how to make informed decisions based on them. The book is designed to be accessible to practitioners and score users with varying levels of technical expertise, from executives of testing organizations and students who take tests to graduate students in educational measurement, psychometricians, and test developers. The theoretical background required to evaluate subscores and improve them are provided alongside examples of tests with subscores to illustrate their use and misuse. The first chapter covers the history of tests, subtests, scores, and subscores. Later chapters go into subscore reporting, evaluating and improving the quality of subscores, and alternatives to subscores when they are not appropriate. This thorough introduction to the existing research and best practices will be useful to graduate students, researchers, and practitioners.

This celebrated primer presents an introduction to all of the key ingredients in understanding computerized adaptive testing technology, test development, statistics, and mental test theory. Based on years of research, this accessible book educates the novice and serves as a compendium of state-of-the-art information for professionals interested in computerized testing in the areas of education, psychology, and other related social sciences. A hypothetical test taken as a prelude to employment is used as a common example throughout to highlight this book's most important features and problems.Changes in the new edition include: *a completely rewritten chapter 2 on the system considerations needed for modern computerized adaptive testing; *a revised chapter 4 to include the latest in methodology surrounding online calibration and in the modeling of testlets; and *a new chapter 10 with helpful information on how test items are really selected, usage patterns, how usage patterns influence the number of new items required, and tools for managing item pools.

How can admissions officers, employers, and scholarship committees maximize the accuracy of prediction of individual performance while minimizing adverse impact due to group differences? Testing offers a straightforward solution to the first half of this problem. Tests are the best way to predict how someone will perform in school, in the military, in medicine, or while controlling airline traffic and flying a plane. Tests are also useful beyond personnel selection, such as for selection of a college major or courses. However, the other side of this problem is more complex. Using tests is always accompanied by group differences that could result in continued systemic discrimination by limiting opportunities for those who are marginalized. This book charts an approach to using tests that incorporates evidence, transparency, and societal values to maximize efficiency and fairness.

How can admissions officers, employers, and scholarship committees maximize the accuracy of prediction of individual performance while minimizing adverse impact due to group differences? Testing offers a straightforward solution to the first half of this problem. Tests are the best way to predict how someone will perform in school, in the military, in medicine, or while controlling airline traffic and flying a plane. Tests are also useful beyond personnel selection, such as for selection of a college major or courses. However, the other side of this problem is more complex. Using tests is always accompanied by group differences that could result in continued systemic discrimination by limiting opportunities for those who are marginalized. This book charts an approach to using tests that incorporates evidence, transparency, and societal values to maximize efficiency and fairness.

A comprehensive history of data visualization—its origins, rise, and effects on the ways we think about and solve problems.With complex information everywhere, graphics have become indispensable to our daily lives. Navigation apps show real-time, interactive traffic data. A color-coded map of exit polls details election balloting down to the county level. Charts communicate stock market trends, government spending, and the dangers of epidemics. A History of Data Visualization and Graphic Communication tells the story of how graphics left the exclusive confines of scientific research and became ubiquitous. As data visualization spread, it changed the way we think.Michael Friendly and Howard Wainer take us back to the beginnings of graphic communication in the mid-seventeenth century, when the Dutch cartographer Michael Florent van Langren created the first chart of statistical data, which showed estimates of the distance from Rome to Toledo. By 1786 William Playfair had invented the line graph and bar chart to explain trade imports and exports. In the nineteenth century, the “golden age” of data display, graphics found new uses in tracking disease outbreaks and understanding social issues. Friendly and Wainer make the case that the explosion in graphical communication both reinforced and was advanced by a cognitive revolution: visual thinking. Across disciplines, people realized that information could be conveyed more effectively by visual displays than by words or tables of numbers.Through stories and illustrations, A History of Data Visualization and Graphic Communication details the 400-year evolution of an intellectual framework that has become essential to both science and society at large.

Exploring the intersection of ethics and statistics, this comprehensive guide illustrates the proper use of probabilistic and statistical reasoning in the behavioral, social, and biomedical sciences. Lauded for their contributions to statistics, psychology, and psychometrics, the authors make statistical methods relevant to readers' day-to-day lives by including real historical situations that demonstrate the role of statistics in reasoning and decision making. In addition, seven U.S. Supreme Court decisions reflect the influence of statistical and psychometric reasoning and interpretation/misinterpretation.

Teacher tenure is a problem. Teacher tenure is a solution. Fracking is safe. Fracking causes earthquakes. Our kids are over-tested. Our kids are not tested enough. We read claims like these in the newspaper every day, often with no justification other than 'it feels right'. How can we figure out what is right? Escaping from the clutches of truthiness begins with one simple question: 'what is the evidence?' With his usual verve and flair, Howard Wainer shows how the sceptical mindset of a data scientist can expose truthiness, nonsense, and outright deception. Using the tools of causal inference he evaluates the evidence, or lack thereof, supporting claims in many fields, with special emphasis in education. This wise book is a must-read for anyone who has ever wanted to challenge the pronouncements of authority figures and a lucid and captivating narrative that entertains and educates at the same time.

To function in modern society complex data must be absorbed and understood at a breakneck pace. The most efficient way to do this is through data-based graphics. This book is an exploration and celebration of graphical methods of data presentation. Visual Revelations' principal purpose is to enlighten, inform, and amuse the reader regarding the shortcomings of common graphical practices; particularly how they can misinform while simultaneously providing models of wonderful graphics. There are many examples of the best graphic practice, graphs that go beyond conveying, facts, and structure to be able to carry emotion as well. Aimed at an educated, lay audience, this volume benefits anyone who must either convey or receive quantitative information, including designers, statisticians, and people in the media.

Is it sensible to screen for breast or prostate cancer? Should the locations of cancer clusters be made available to the general public? When a doctor wants to perform major surgery and there's no chance for a second opinion, do you agree? The answers to these questions are not as black and white as they may first appear. Medical Illuminations presents thirteen contemporary medical topics, from the diminishing value of mammograms to how to decide if a hip needs to be replaced, to understanding cancer maps. In each case it illustrates how modern tools of statistical thinking and statistical graphics can illuminate our understanding. The goals are to solve some vexing problems that seem perplexing, and to make both the problems and their solutions clear to a non-technical audience. The aim is to ignite in the reader an understanding of statistical thinking, which, though subtle, can be learned without going through arcane mathematics. And, moreover, that learning about how to think in this way provides a huge payoff in the deeper understanding of our complex world.

Exploring the intersection of ethics and statistics, this comprehensive guide illustrates the proper use of probabilistic and statistical reasoning in the behavioral, social, and biomedical sciences. Lauded for their contributions to statistics, psychology, and psychometrics, the authors make statistical methods relevant to readers' day-to-day lives by including real historical situations that demonstrate the role of statistics in reasoning and decision making. In addition, seven U.S. Supreme Court decisions reflect the influence of statistical and psychometric reasoning and interpretation/misinterpretation.

In his entertaining and informative book Graphic Discovery, Howard Wainer unlocked the power of graphical display to make complex problems clear. Now he's back with Picturing the Uncertain World, a book that explores how graphs can serve as maps to guide us when the information we have is ambiguous or incomplete. Using a visually diverse sampling of graphical display, from heartrending autobiographical displays of genocide in the Kovno ghetto to the "Pie Chart of Mystery" in a New Yorker cartoon, Wainer illustrates the many ways graphs can be used--and misused--as we try to make sense of an uncertain world. Picturing the Uncertain World takes readers on an extraordinary graphical adventure, revealing how the visual communication of data offers answers to vexing questions yet also highlights the measure of uncertainty in almost everything we do. Are cancer rates higher or lower in rural communities? How can you know how much money to sock away for retirement when you don't know when you'll die? And where exactly did nineteenth-century novelists get their ideas? These are some of the fascinating questions Wainer invites readers to consider. Along the way he traces the origins and development of graphical display, from William Playfair, who pioneered the use of graphs in the eighteenth century, to instances today where the public has been misled through poorly designed graphs. We live in a world full of uncertainty, yet it is within our grasp to take its measure. Read Picturing the Uncertain World and learn how.

This extraordinary book provides a journey through the jungle of good and bad graphical devices used to illustrate and obfuscate data. The diversity of examples is tremendous: ranging from Napoleon's retreat from Moscow to the O-rings on the space shuttle and from mortalities in hospitals to the schedules of buses. The author's aim is to illustrate how effectively a well-chosen graphic can reveal in an instant the essential truth behind some data whilst a poorly designed representation can conceal an awful truth. As well as providing numerous examples of both, the author includes plenty of helpful ideas to make us all better producers and consumers of graphics.

Uneducated Guesses challenges everything our policymakers thought they knew about education and education reform, from how to close the achievement gap in public schools to admission standards for top universities. In this explosive book, Howard Wainer uses statistical evidence to show why some of the most widely held beliefs in education today--and the policies that have resulted--are wrong. He shows why colleges that make the SAT optional for applicants end up with underperforming students and inflated national rankings, and why the push to substitute achievement tests for aptitude tests makes no sense. Wainer challenges the thinking behind the enormous rise of advanced placement courses in high schools, and demonstrates why assessing teachers based on how well their students perform on tests--a central pillar of recent education reforms--is woefully misguided. He explains why college rankings are often lacking in hard evidence, why essay questions on tests disadvantage women, why the most grievous errors in education testing are not made by testing organizations--and much more. No one concerned about seeing our children achieve their full potential can afford to ignore this book. With forceful storytelling, wry insight, and a wealth of real-world examples, Uneducated Guesses exposes today's educational policies to the light of empirical evidence, and offers solutions for fairer and more viable future policies.

Good graphs make complex problems clear. From the weather forecast to the Dow Jones average, graphs are so ubiquitous today that it is hard to imagine a world without them. Yet they are a modern invention. This book is the first to comprehensively plot humankind's fascinating efforts to visualize data, from a key seventeenth-century precursor--England's plague-driven initiative to register vital statistics--right up to the latest advances. In a highly readable, richly illustrated story of invention and inventor that mixes science and politics, intrigue and scandal, revolution and shopping, Howard Wainer validates Thoreau's observation that circumstantial evidence can be quite convincing, as when you find a trout in the milk. The story really begins with the eighteenth-century origins of the art, logic, and methods of data display, which emerged, full-grown, in William Playfair's landmark 1786 trade atlas of England and Wales. The remarkable Scot singlehandedly popularized the atheoretical plotting of data to reveal suggestive patterns--an achievement that foretold the graphic explosion of the nineteenth century, with atlases published across the observational sciences as the language of science moved from words to pictures. Next come succinct chapters illustrating the uses and abuses of this marvelous invention more recently, from a murder trial in Connecticut to the Vietnam War's effect on college admissions. Finally Wainer examines the great twentieth-century polymath John Wilder Tukey's vision of future graphic displays and the resultant methods--methods poised to help us make sense of the torrent of data in our information-laden world.

The measurement models employed to score tests have been evolving over the past century from those that focus on the entire test (true score theory) to models that focus on individual test items (item response theory) to models that use small groups of items (testlets) as the fungible unit from which tests are constructed and scored (testlet response theory, or TRT). In this book, the inventors of TRT trace the history of this evolution and explain the character of modern TRT. Written for researchers and professionals in statistics, psychometrics, and educational psychology, the first part offers an accessible introduction to TRT and its applications. The second part presents a comprehensive, self-contained discussion of the model couched within a fully Bayesian framework. Its parameters are estimated using Markov chain Monte Carlo procedures, and the resulting posterior distributions of the parameter estimates yield insights into score stability that were previously unsuspected.

The measurement models employed to score tests have been evolving over the past century from those that focus on the entire test (true score theory) to models that focus on individual test items (item response theory) to models that use small groups of items (testlets) as the fungible unit from which tests are constructed and scored (testlet response theory, or TRT). In this book, the inventors of TRT trace the history of this evolution and explain the character of modern TRT. Written for researchers and professionals in statistics, psychometrics, and educational psychology, the first part offers an accessible introduction to TRT and its applications. The second part presents a comprehensive, self-contained discussion of the model couched within a fully Bayesian framework. Its parameters are estimated using Markov chain Monte Carlo procedures, and the resulting posterior distributions of the parameter estimates yield insights into score stability that were previously unsuspected.

To function in modern society complex data must be absorbed and understood at a breakneck pace. The most efficient way to do this is through data-based graphics. This book is an exploration and celebration of graphical methods of data presentation. Visual Revelations' principal purpose is to enlighten, inform, and amuse the reader regarding the shortcomings of common graphical practices; particularly how they can misinform while simultaneously providing models of wonderful graphics. There are many examples of the best graphic practice, graphs that go beyond conveying, facts, and structure to be able to carry emotion as well. Aimed at an educated, lay audience, this volume benefits anyone who must either convey or receive quantitative information, including designers, statisticians, and people in the media.

This celebrated primer presents an introduction to all of the key ingredients in understanding computerized adaptive testing technology, test development, statistics, and mental test theory. Based on years of research, this accessible book educates the novice and serves as a compendium of state-of-the-art information for professionals interested in computerized testing in the areas of education, psychology, and other related social sciences. A hypothetical test taken as a prelude to employment is used as a common example throughout to highlight this book's most important features and problems. Changes in the new edition include: *a completely rewritten chapter 2 on the system considerations needed for modern computerized adaptive testing; *a revised chapter 4 to include the latest in methodology surrounding online calibration and in the modeling of testlets; and *a new chapter 10 with helpful information on how test items are really selected, usage patterns, how usage patterns influence the number of new items required, and tools for managing item pools.