Methodical Problems of Comparative Studies

Plan
The problem of comparison
The problem of equivalence
The problem of universality
“Small N, large-V” problem
The Galton’s problem
The problem of measurement
The problem of interpretation

The problem of comparison
No two things in the world are identical, and therefore comparative politics runs into a difficulty when it wants to compare the ‘same phenomena’ or ‘similar objects’ in different countries.
Two strategies to deal with this problem are available:
• looking for more abstract concepts
• looking for equivalent concepts

The problem of equivalence
Stegmueller, D. (2011). Apples and Oranges? The Problem of Equivalence in Comparative Research. Political Analysis, 19(4). P. 473.

The problem of equivalence
The problem is depicted in ideal-typical form in Fig. 1 . Panel A shows a comfortable state of the survey research world where no item bias exists. That an individual j living in country k responds differently from an individual living in country к' is completely due to the fact that they have different preferences (njk != n’jk’). Panel В shows the opposite situation. Now our two individuals share the same level of preference or attitude strength (njk = njk’), but their responses differ due to the systematic country differences discussed above. Clearly, the differences between those individuals are not real but the result of country method effects, so that latent preferences cannot be compared between countries.

The problem of equivalence
In some countries, individuals are predominantly acquiescent, that is, they have a tendency to select only one side of the scale (usually the one indicating agreement). Some countries produce extreme responders, who consistently choose extreme ends of scales, whereas in other countries, individuals predominantly choose the middle part of a response scale - avoiding strong statements (Yang et al. 2010). In consequence, this means that two individuals sharing the same level of preference may answer survey questions differently, simply because one of them is from a country where an extreme response style is common. Scores from individuals from different countries are then no longer directly comparable since they are systematically biased (Millsap and Kwok 2004). In other words, the dependent variable lacks equivalence.

The problem of universality
Strategies to deal with this problem are available:
The Ladder of Abstraction
Retest

“Small N, large-V” problem
With each additional explanatory variable (V) the number of cases (n) required for comparisons grows exponentially. Therefore, only a few explanatory variables are often too many for the relatively small number of cases available, in which case an empirical test is not possible.
Strategies to deal with this problem are available:
increase the number of cases;
limiting the number of variables.

The Galton’s problem
Sir Francis Galton was an English Victorian era statistician, progressive, polymath, sociologist, psychologist, anthropologist, eugenicist, tropical explorer, geographer, inventor, meteorologist, proto-geneticist, and psychometrician.

The Galton’s problem
Lack of independence between two cases presents two problems. First, it might bias the results; second, it affects statistical analyses by making the size of the sample of independent cases uncertain.

The problem of measurement
Common to all fields of research

The problem of interpretation
Multiple meanings of the same political phenomenon
Two strategies to deal with this problem are available:
To change attitude towards history and histortical method
Scientific Realism