Data are Data, Analysis is Art

Most of the literature in my field comes from economists, education researchers that more often than not employ econometric models, and quantitative macro sociologists.

There is nothing wrong with that.

However, a recent rash of reading of such reports brought to mind how we conflate data with analysis. The former is something approaching empirical objectivity (huge caveats there) while analysis is the more subjective interpretation of reality.

For example, “450,000 students enrolled in for-profits in 2004” is a data point. The usual interpretation of that in research on for-profit college expansion is, “for-profits respond to unmet consumer demand.”

However, I can see the same data point and artfully interpret it to mean something like, “economic change, deregulation, and social policies constrained practical options for a half million workers to participate fully in the labor market. This created a demand for credentials to the benefit of for-profit education companies”.

I am not being precious. The difference matters qualitatively to the conclusions that are drawn and how public narratives about any kind of data are constructed.

How we construct the causal models for outcomes define the types of interventions likely to be employed.

I cannot help but think that the public policy prescriptions to regulate costs, defaults, and graduation rates at for-profit colleges are limited by the assumptions embedded in the analysis of the data on how the for-profit college sector developed. We limit our approach to gainful employment regulations and accreditation guidelines because we (falsely, in my opinion) model higher education as a closed eco-system whose infinity loop begins and ends with “consumer demand”.

Analysis that moves us back a step in the causal model to examine structural change, educational and occupational ideology and social policy could result in better prescriptions.

At the very least, it is worth reminding folks that how data are interpreted is a function of all matter of theories and beliefs, implicit and explicit, with all suggested biases implied.