A small investment in time yields a big payoff.
The Validity Study Service gives Ohio State what it needs to make data-driven decisions.
“In the past, we never had numbers to prove our gut feelings about admissions. With the Validity Study Service, now we do.”
Rob Chabot
Director of Admissions and Recruiting
Fisher College of Business, The Ohio State University
Before 2007, officials at the Max M. Fisher College at Ohio State relied chiefly on their experience and gut instincts in making admission decisions. In the daily grind of admissions work, graduate programs director David Smith and associate director Rob Chabot lacked the data to make data-driven decisions.
But in the fall of 2006, the Buckeyes signed on to the Validity Study Service, a statistical analysis of their admission data that helps them predict who will succeed in their programs. They compiled GMAT scores, undergraduate and mid-point program grade-point averages for the master’s of accountancy and MBA programs. GMAC psychometrician Eileen Talento-Miller was always a phone call away to walk them through the formatting.
“I was beyond stunned at how quick the turnaround time was once the data was in a format she needed,” Smith says. “I thought it would take a month or two. I think we got the results back in a week or 10 days—really quick. We got this full report, analysis of certain data—wow!”
Probability of Success
Immediately helpful to Smith and Chabot was the Probability of Success matrix, which shows the probability that a student will fall in the bottom quarter of his class based on a combination of GMAT score and undergraduate GPA.
“It’s a godsend to see how the GPA and GMAT work together. We have a clear cutoff line as to what’s acceptable and what isn’t. We never had numbers to prove our gut feeling in the past,” Chabot says. “With the Validity Study Service, now we do.”
Perhaps 70 percent of their admissions decisions are easy either way, but the Probability of Success matrix has proved invaluable in the other 30 percent: the tough calls, the “marginal admits” who are ill-served, and don’t serve the program, if they are admitted but aren’t successful.
“It helps us because we usually don’t know where our gut stops and our mind starts,” Smith says.
Long-Term Benefits
Having the Validity Study Service’s easy-to-use charts also help Smith and Chabot articulate why they turn down applicants and give them evidence against charges of bias. “People come to us all the time about a person who’s wonderful,” Smith says. “Another valuable result is that it helps explain to external parties about our decisions. The dean’s office is also interested in what we’re doing.”
Besides helping them make immediate admissions decisions, the Validity Study Service also helps Smith and Chabot evaluate their procedures. “Now the entire office is data-oriented. That’s a change for the grad program office here,” Smith says.
The report ultimately showed Smith and Chabot precisely where their gut feelings were wrong.
“You’d be surprised how often what everybody thinks is true isn’t true,” Chabot says.
Tips for Conducting Your Own Validity Study
- Consider carefully which programs and which variables you want to analyze. What programs and subgroups have the most “marginal admits,” for which clear-cut data would be useful? GMAC will help you determine whether your program is large enough to give a statistically valid sample size.
- Set realistic expectations for how long it will take to pull the data you need, and budget enough time for it.
- Identify someone who can help you pull the data electronically. You’ll have to do some manual work to make sure the data aren’t corrupted, but start with electronic files.
- Contact us for help in deciding the most appropriate data to include.
Questions? Contact Us