Data Analysis & Interpretation and the emergency budget July 1, 2010Posted by larry in budget, economics, Institute for Fiscal Studies, Osborne, social policy.
Tags: emergency budget, interpretation, Models of Data, nature of science, objectivity, Suppes
It has been common currency among a certain group of philosophers and scientific analysts that data do not “speak for themselves”, as it were. This means specifically that data, in order to be meaningful, must be interpreted. So much is obvious. But there is another aspect to this perspective. And this is that data must be collected under an interpretation. Unguided by any interpretive framework, collected data has as much utility as an unordered list of telephone numbers.
Before any data is collected, you have to know what you are going to collect and for what purposes. Otherwise, what you will end up with will be worthless. So, before you begin, you need to know the kind of information you wish to collect and the sorts of things it might tell you. That is, you introduce an interpretive framework, with its attendant assumptions, into the data collection process. The assumptions of the interpretive framework underlying the data collection process are often not spelled out explicitly. Nevertheless, they are present, and they influence both collection and analysis.
Before the data are collected, you need to have set out your program of analysis. That is, you ought to have a very good idea of what analytical procedures you are going to use before you see the data. This is to avoid any contaminating influence that might be brought about by the data on the investigator’s interpretation and analysis.
Once the data is collected, a detailed and extended version of the initial interpretive framework needs to be employed in order to assess what the data “say”. Although an interpretation may appear to be quite straightforward, this is not usually the case for economic data. Such data invariably carry political implications and these need to be made explicit. A lack of transparency here can suggest a nonexistent degree of objectivity of analysis and interpretation.
Channel 4 news does not always appear to appreciate either this distinction or its two prongs. Every time the channel introduces some analysis or comment, for instance, by the Institute for Fiscal Studies or its director, Robert Chote – if you didn’t know better, you might think that no one else worked for this organization – accompanied by a hyperbolic introduction, such as the “highly regarded” …, as if interviewing the institute itself were not encomium enough. (Although in so doing, the channel may be seeking to preempt criticisms of bias on its part, this is not the most salubrious way to exhibit lack of bias, which is impossible anyway.)
Such introductions give the impression that the analysis which is being presented is the most objective that it is possible to obtain (I will touch upon on the notion of objectivity another time – let me only say here that the notion of objectivity itself is the product of a long social and cultural history and is itself part of a set of ‘subjective’ interpretative frameworks.). That is, the impression is given that the analysis is independent of any controversial interpretations and that the analysis follows relatively straightforwardly from the data set itself. Such an impression is quite misleading. Not only can a single data set yield differing and incompatible interpretations, it can lead in economics to the promulgation of social policies that are massively differentially detrimental to different social groups that the empirical evidence does not really justify. This is no small matter.
Sometimes, the assumptions within the data collection framework are essential for understanding any assessment of the data collected. Let me take the homelessness figures once produced by the Thatcher government. The government wanted to show that as a consequence of their economic policies that homelessness had been reduced. The data they showed seemed to reflect this. It was only when it was noted how “homelessness” had been (re)defined that the real significance of the data became clear. Not only had homelessness not been reduced, it had increased. How did the Thatcher government manage this sleight of hand? By redefining the nature of homelessness. In their new definition of homelessness, anyone living in a box that did not move location for at least a week was viewed as having a home. I.e., they were not homeless. Anyone not aware of this new definition would assume that the data showed that the number that were homeless had been reduced and that this reduction had been brought about, as stated or implied, by innovative economic initiatives. Of course, nothing was further from the truth.
Such alterations have occurred when the FBI has published its crime figures. By altering the definitions of what constitutes a particular crime, the figures can be made to show that the crimes in question have either increased or decreased in a given year, independently of the events themselves. Altering definitions in this way is not inherently dishonest, so long as the given alteration is transparent.
Something similar seems to be happening in the emergency budget. Osborne has contended that the poor are not worse off as a consequence of his budget, but in order to assess this, we need to see what assumptions guided the selection of the data sets he used and the manner in which he analyzed them. We need him to be more transparent than he has been so far. None of this is trivial. And the fact that others appear to agree with his stringent budget cuts is not a valid justification of his claim that he is right and his detractors wrong. They could all be wrong. What is needed is an overarching framework, independent of the ones Osborne has utilized, which can be applied to the data that has been gathered and which can confirm or disconfirm the relevance of the data used and/or the analysis employed. Blanchflower, among others, has said that he intends to do just this.
(For a deep and enlightening discussion of the complex relations between theory and data, consult Patrick Suppes’ “Models of Data“.)