[Paleopsych] CHE: Scholars Say College Admissions Offices Misuse Advanced Placement Data
checker at panix.com
Mon Jan 17 16:23:37 UTC 2005
Scholars Say College Admissions Offices Misuse Advanced Placement Data
The Chronicle of Higher Education, 5.1.21
A study finds that the high-school courses aren't always good
predictors of college success
By DAVID GLENN
College-admissions officers should be cautious when weighing Advanced
Placement courses on applicants' high-school transcripts, according to
a working paper presented here at the annual meeting of the American
Economic Association this month.
The paper asserts that the mere act of taking AP courses in high
school -- as distinct from scoring well on the official AP tests
-- does not predict that a student will perform well in college.
The paper's authors -- Kristin Klopfenstein, an assistant professor of
economics at Texas Christian University, and M. Kathleen Thomas, an
assistant professor of economics at Mississippi State University
-- analyzed the records of more than 28,000 students who graduated
from Texas high schools in 1999 and who enrolled that fall in the
state's four-year public universities.
The researchers posed two central questions: Did the Texas students
who took Advanced Placement courses in high school have higher grades
in their first year of college than their non-AP peers? And were they
less likely to drop out before their sophomore year?
The answers turned out to be no and no. In Ms. Klopfenstein and Ms.
Thomas's statistical analysis -- described in a paper titled "The
Advanced Placement Performance Advantage: Fact or Fiction?" -- all of
the variance in the students' grade-point averages and dropout rates
was explained by the familiar predictors of college performance:
high-school grade-point averages, SAT scores, parents' education and
income, and the proportion of experienced teachers in the students'
high schools. After such variables were accounted for, the AP courses
on the students' transcripts did not have any extra predictive power
of their own.
"AP experience may serve as a signal of high ability and motivation,"
the authors write, "but it does not by itself indicate superior
Quality vs. Quantity
Most of the best-known research about the Advanced Placement program
-- including studies conducted under the auspices of the College
Board, which creates and administers the official AP tests -- has
tried to answer a different set of questions. Those studies have
generally found that AP tests are a reliable equivalent to first-year
college examinations, and that institutions should feel comfortable in
awarding college credit to students who score well on the tests.
The new Texas study does not challenge such classic findings about the
AP program. Instead, the authors express concern that some school
districts and college-admissions offices behave as if simply taking an
AP course, regardless of one's performance on the AP test, will help a
student do better in college.
In an interview, Ms. Thomas said that she generally supports the
recent expansion of the AP program and similar efforts to make
high-school curricula more rigorous. She worries, however, that some
school districts may be paying too little attention to the quality of
their AP courses, and that high schools and colleges may have policies
that are not warranted by solid evidence.
In particular, Ms. Thomas cited the University of California's policy
of giving extra weight to AP courses when calculating applicants'
high-school grade-point averages. The university considers an A in an
AP course to be worth 5.0 points, as opposed to 4.0 for an A in a
standard course. The university grants such extra weight whether or
not the student chose to take the official AP test at the end of the
Ms. Thomas fears that admissions policies like California's
-- together with various incentives offered by federal and state
governments -- might be prompting some high schools to slap together
ill-designed AP programs with poorly trained teachers.
The AP program has expanded very rapidly. In 1980-81, according to the
College Board, 133,702 high-school students took Advanced Placement
exams. In 2003-4, the number was 1,017,396.
"We suspect that there's a lot of variance in the quality of AP
programs," Ms. Thomas said. "As schools move to increase access,
unless their principals and administrators really understand what goes
into developing that program, we can just envision cases where
suddenly a teacher hears, 'Guess what? You're the new AP teacher.'"
Many districts have moved to add AP courses, Ms. Thomas said, in the
wake of a 1999 study by Clifford Adelman, a senior research analyst at
the U.S. Department of Education, who found that "intensity of
high-school curriculum" is a strong predictor of students' college
Ms. Thomas does not dispute Mr. Adelman's general insight, but she
argues that for certain districts with large numbers of struggling
students, AP courses might not be the best way to make the curriculum
more rigorous. AP courses often require teachers to move rapidly
through a huge amount of material, she said, and "sometimes covering a
topic in depth is sacrificed for covering breadth."
The Texas study comes on the heels of a similar paper that was
released last month by scholars at the University of California. Saul
Geiser, a visiting scholar at the University of California at
Berkeley's Center for Studies in Higher Education, and Veronica
Santelices, a Ph.D. candidate at the University of California at
Berkeley's School of Education, analyzed the records of 81,445
students who enrolled as freshmen in the university system from 1998
Mr. Geiser and Ms. Santelices found that the admissions program's
"bonus point" for high-school AP courses "bears little or no
relationship to college performance." Like Ms. Klopfenstein and Ms.
Thomas, the California scholars found that simply taking AP courses is
not a valid predictor of college grades or persistence.
The California authors also found, however, that the subgroup of
students who took the official AP tests, and did well on them,
performed very well in college. Strong scores on AP tests, in fact,
were a better predictor of students' sophomore-year grade-point
averages than any other variable except their high-school grade-point
averages. (Ms. Klopfenstein and Ms. Thomas were unable to perform a
similar analysis because they did not have access to the Texas
students' AP-test records. The College Board might soon provide them
with those data, however.)
At the conference panel here, Ronald G. Ehrenberg, a professor of
industrial and labor relations and of economics at Cornell University,
and director of the Cornell Higher Education Research Institute, cited
the importance of Ms. Klopfenstein and Ms. Thomas's line of research.
He also said, however, that the study's design might have missed some
important potential effects of taking Advanced Placement courses. For
example, he asked, "What's the impact of having taken an AP class, or
classes, on where students go to college?"
In a telephone interview, Trevor Packer, executive director of the
College Board's Advanced Placement program, said that while he has
serious reservations about the Texas paper's methodology, he welcomes
both the Texas and California studies.
"More and more people have felt that AP classes are something on a
student's transcript that would be meaningful in the admissions
process," he said.
That belief, he continued, should be scrutinized carefully because the
College Board has established only the much narrower claim that
students who score well on AP tests should be awarded college credit
in those subjects. The Texas and California studies, he said, might be
useful steps toward assessing whether taking AP courses, per se, is an
accurate predictor of college performance.
Mr. Packer said he wished, however, that the Texas paper had
distinguished between students who did well on AP tests and those who
did not. "Until Klopfenstein and Thomas separate their sample into at
least two different groups," he said, "it's impossible for them to
make this claim that they're making that AP doesn't impact overall
Ms. Thomas said that she and Ms. Klopfenstein are eager to conduct
such a study, and hope to receive data from the College Board that
would allow them to do so. But she also said that it was valid for
them to look at AP students as a single group, without regard to their
test scores, because that is exactly what some important institutional
actors, including the University of California admissions offices,
Mr. Packer said he also worries that the Texas study's assessment of
dropout rates might be flawed because Ms. Klopfenstein and Ms. Thomas
considered all students who disappeared from the Texas records in 2000
as having dropped out. It is possible, Mr. Packer said, that some of
those students might have transferred to private colleges or
transferred out of state.
Ms. Thomas replied that national statistics suggest that very few
Texas students were mislabeled in that way. In any case, she added,
such a flaw would not affect her analysis of students' first-year
As for Ms. Klopfenstein and Ms. Thomas's concern that the quality of
AP courses is sometimes weak, Mr. Packer conceded that it is a
legitimate worry, but said that more-detailed studies were needed.
"What would really help administrators," he said, "would be to
understand what sort of AP classrooms do impact students' college
"Administrators need research-based help in understanding what
configuration of the AP classroom is helpful for expanding access and
providing increased opportunity and inclusivity, without watering down
the course," he said.
Such studies are indeed planned by Ms. Klopfenstein and Ms. Thomas.
They would like to use their Texas data to identify particular AP
programs that perform improbably well, given the school district's
resources or demographics. Ms. Thomas said that studies of such
programs could explain how they succeed, providing a lesson for other
The College Board, meanwhile, plans to publish new guidelines this
year about the training that AP teachers should receive, Mr. Packer
Ms. Klopfenstein and Ms. Thomas's paper has been submitted to a
journal but has not yet completed the peer-review process.
More information about the paleopsych