Rethinking the science of politics – Multiple methods strengthen scientific inference

Why do political theories so often fail the test of common sense? And why do individual political studies often seem to stop short of providing general guidance about political matters?

James Granato and Frank Scioli, National Science Foundation (NSF), managers of the political science program, write in the newly published June issue of Perspectives on Politics that the separation of theory and real-world tests often sharply limit the usefulness of each. They identify three methods commonly used in political science studies – formal models, case studies, and applied statistical models – any of which, used alone, they say, may produce faulty results.

Theories need factual tests, Granato and Scioli say. Case studies need to inform theories. And statistical results need to be corrected by case study analyses and based on sound theory. Problems arise when researchers use only one method and do not compensate for its shortcomings, a challenge that plagues not only political science but the other social sciences as well.

What to do?

In a research program initiative called Empirical Implications of Theoretical Models (EITM), Granato and Scioli are managing new studies across all the social and behavioral sciences to give researchers who couple theoretical models and real-world tests of these models a chance to reveal findings that can provide help, for example, to new governments in developing constitutions, to economic policy makers in estimating the impacts of trade agreements, and to educators in facilitating learning in groups.

So far, 26 awards have been made through this NSF program.

“We think these awards will spark a new direction in political science – and in other social sciences as well,” says Scioli. “These projects are not only good science, but they point toward a future in which an increasingly broad range of consequential policy decisions can be reliably informed by social science evidence.”

In the EITM initiative, researchers are required to use or develop a formal model and to test that model with real data. The combined analysis leads to two desirable outcomes, say Granato and Scioli. First, using both theoretical models and empirical tests will result in research findings can lead to better-specified and more reliable understandings of what causes the political, economic, or social phenomena studied. Such understanding is crucial for the wise application of social science findings to public policy. Second, the robust results generated by this kind of analysis allow knowledge to cumulate. Scientists can discard theories that fail empirical tests and produce useful generalizations from real-world data.

For example, Carles Boix, a political scientist at the University of Chicago, is refining theoretical models with real-world data to predict how political institutions affect emerging democracies. Boix, through an EITM grant, will derive game-theory models that, first, characterize political agents with varying economic traits (i.e., amount and types of wealth) and organizational strengths; and, second, demonstrate how constitutional set-ups influence the strategies of these political agents in choosing political regimes. His empirical test comes from a survey of all sovereign nations over the past 200 years. These data allow him to assess the probability that a given democracy will collapse into a dictatorship. Early findings show that a mix of federalism and a parliamentary form of government is least likely to revert from democracy to dictatorship. The model, based on a plausible theory with real-world validation, could help in designing constitutions for new governments.

A subset of the EITM awards focuses on economic and policy questions such as:

• the effect of tax increases and tax cuts
• what effect financial structures have on economic growth and inequality
• how people adapt to gains versus losses
• how monetary policy can best respond to shocks (such as a credit crunch or technological innovation)
• whether markets are overvalued
• the effects of changes in international economic and trade policy such as those embodied in the North American Free Trade Agreement (NAFTA).

Researchers are also examining social issues. One EITM project will link data on employers and employees to understand the workplace and human performance from both perspectives. Two other projects will investigate how residential communities form from the housing and neighborhood choices people make. Still others propose to increase understanding of how people learn in groups, how learners form generalizations from examples and how large social networks evolve.

The EITM initiative was part of NSF’s new priority area in Human and Social Dynamics in which formal modeling is an area of emphasis.