With all of the reporting around College Scoreboard this week, it struck me how difficult it is for a non-academic consumer of information (like myself) to find out exactly what a large set of data like this means. So I stopped skimming all the headlines and took a closer look... for what it's worth.
ProPublica's report "Colleges Flush with Cash Saddle Poorest Students with Debt," focused on how selective institutions drive up debt for low-income students: "More than a quarter of the nation’s 60 wealthiest universities leave their low-income students owing an average of more than $20,000 in federal loans."
This seems to support the findings from the Hoxby-Avery study from 2012 (see table 1: College Costs and Resources by Selectivity). But what's striking is that the way the messages of the two sources are presented, you might come to two completely different conclusions about the data.
ProPublica = selective institutions are expensive for low-income high-achievers
Hoxby-Avery = selective institutions would cost less than non-selective schools for low-income high-achievers
I tried to look at the data myself. It wasn't easy. I went to College Scoreboard, ED's public-facing website and downloaded the raw data. I immediately realized:
- I need to use STATA to see anything useful
- I need to understand the coding enough to set up my parameters in STATA
- I actually need to have STATA (usually that means being affiliated with an organization or school that has a license)
- I need to know what STATA is. OK, I do, but still.
Comprehensiveness of Data?
And to add to the confusion, eLiterate reports that there are problems with the actual information gathered, including a big chunk of degree-granting community colleges simply overlooked.
On the numbers around completion rates, ED acknowledges that their data doesn't reflect the reality of adult education. Again, from e-Literate, a quote straight out of Dept. of Ed's report on College Scoreboard:
"The most commonly referenced completion rates are those reported to IPEDS and are included on the College Scorecard... However, they rely on a school’s population of full-time students who are enrolled in college for the first-time. This is increasingly divergent from the profile of the typical college student, particularly at many two-year institutions and some four-year schools."
What to think?
Even without considering the effects of well-documented psychological phenomena like confirmation bias and priming, it seems like a herculean challenge to get a solid grasp on this stuff.
Any strategies to share?