[Paleopsych] NYT: Measuring Literacy in a World Gone Digital

[Premise Checker]

Measuring Literacy in a World Gone Digital
NYT January 17, 2005
By TOM ZELLER Jr.

There was a time when researching a high school or college
term paper was a far simpler thing. A student writing
about, say, Count Ferdinand von Zeppelin, might have
checked out a book on the history of aviation from the
local library or tucked into the family's dog-eared
Britannica. An ambitious college freshman might have
augmented the research by looking up some old newspaper
clips on microfilm or picking up a monograph in the stacks.


Today, in a matter of minutes, students can identify these
and thousands of other potential resources on the Internet
- and, as any teacher will attest, they are not always
adept at sorting the wheat from the chaff.

Now the Educational Testing Service, the nonprofit group
behind the SAT, Graduate Record Examination and other
college tests, has developed a new test that it says can
assess students' ability to make good critical evaluations
of the vast amount of material available to them.

The Information and Communications Technology literacy
assessment, which will be introduced at about two dozen
colleges and universities later this month, is intended to
measure students' ability to manage exercises like sorting
e-mail messages or manipulating tables and charts, and to
assess how well they organize and interpret information
from many sources and in myriad forms. About 10,000
undergraduates at schools from the University of
California, Los Angeles to Bronx Community College are
expected to take the test during the first offering period,
which ends March 31.

Still, just what is meant by "information" or even
"technological" literacy remains a hotly debated topic in
academic circles, and there is no widespread agreement on
whether such skills can be taught, much less measured in a
test. What seems certain, however, is that a lucrative
market is emerging for testing companies that are willing
to fill the perceived need.

The initial technology test is aimed at midlevel college
students, but the Educational Testing Service says it has
also received inquiries from high schools and businesses.
And while the new assessment is not a high-stakes
requirement for academic advancement like the SAT, it seems
inevitable that most students will one day need to prove
themselves along these lines.

Part of the problem, many educators say, is that the
traditional vetting process for information is now so
easily bypassed.

"In an earlier time, information came, really, from only
one place: the university library," said Lorie Roth, the
assistant vice chancellor of academic programs for the
California State University system, one of seven school
systems that worked with the testing company over the last
two years to develop the test. "Now it is all part of one
giant continuum, and often the student is the sole arbiter
of what is good information, what is bad information and
what all the shades are in between."

But not everyone agrees that measuring information literacy
can be done, even with a standardized test.

"There is a basic problem with identifying a single set of
skills that could possibly relate to all people," said
Stanley Wilder, the associate dean of the River Campus
Libraries at the University of Rochester in New York, who
wrote a withering assessment of the information literacy
movement in The Chronicle of Higher Education two weeks
ago. "There isn't a serious critique of any of the
assumptions that info-literacy makes," Mr. Wilder said in
an interview. "They'll tell you that it teaches critical
thinking, but there's never been a study that measures
whether students are really lacking this, or whether
libraries can impact this."

Be that as it may, it is true that the information literacy
movement could prove a windfall for companies like the
Educational Testing Service.

Developing metrics for measuring how much students know -
or how much they have yet to learn - has become a lucrative
market. Eduventures, a research firm in Boston, estimated
the assessment market for prekindergarten to Grade 12 -
excluding the college years and beyond - at $1.8 billion
for 2003. Given President Bush's announcement last
Wednesday that he plans to expand the standardized testing
mandated under the No Child Left Behind Act - which
includes a commitment to "ensuring that every student is
technologically literate by the time the student finishes
the eighth grade" - the market for assessments is certain
to grow.

Beyond the SAT, the Educational Testing Service controls a
separate boutique market of higher-level tests like the
Graduate Record Examination and the Graduate Management
Admission Test. Despite its nonprofit status, it is the
world's largest private educational testing and measurement
organization. The company administers and scores nearly 25
million tests annually in more than 180 countries, and
posted $825 million in revenues for fiscal year 2004.

In an extensive report, "Tech Tonic: Towards a New Literacy
of Technology," published in September, the Alliance for
Childhood, a nonprofit group that is often skeptical of
technology in schools, was critical of the new test. "For
E.T.S., this is part of a broader global plan to develop
and promote international technology literacy standards,
and then offer countries around the world a chance to buy a
full array of assessment products and services that can be
used to implement their standards," the report said.

But if critics see this as an unjustified entry into an
already littered field of standardized tests, the company
argues that the information age - and a new culture of
accountability - demand it.

"I think there's always that tension," said Teresa Egan,
the project manager who is steering the test's release at
the end of this month. "People feel there's too much
testing across the board now. Or they ask whether we are
focusing so much time on testing that students don't have
time for other educational experiences.

"But the public wants accountability. People want to ensure
that colleges are actually preparing students for the
future - the future being an information society." The
technology test will cost colleges around $25 a student -
discounted to $20 for institutions that sign up during the
first testing period. Students will take the Web-based exam
in classrooms or instruction labs, logging on with access
codes purchased by their schools. Scores in the first round
will be aggregated for each institution; the company aims
to make scoring for individual students available in 2006.

In 2001, the testing company brought together an
international consortium of educators, technology
specialists and government representatives to begin
defining the core characteristics of information
consumption at the college level.

Knowing where and how to find information, they agreed, was
just the beginning. Interpreting, sorting, evaluating,
manipulating and repackaging information in dozens of forms
from thousands of sources - as well as having a fundamental
understanding of the legal and ethical uses of digital
materials - are also important components.

"Critical thinking is a central aspect of the new economy,"
said Robert B. Reich, the secretary of labor in the Clinton
administration, who is now a professor of social and
economic policy at Brandeis University. Professor Reich is
also the author of the 1991 book "Work of Nations:
Preparing Ourselves for 21st Century Capitalism," which
provided a something of a touchstone for the information
literacy movement. "Our high school curricula are locked
into an industrial age that may have only a tangential
relationship to the information age," he said in an
interview.

To the extent that efforts like the new technology test
help reshape curriculums along these lines, Mr. Reich said,
they probably will help.

According to Ms. Egan of the Educational Testing Service,
the test is also fun.

"Can you help me find a good source of products and gifts
designed for left-handers?" reads a sample question from a
fictitious office manager. "I'd like someplace that offers
a wide range of merchandise with product guarantees - also
that has an online catalog and online ordering. Discounts
would also be a plus."

Fictitious colleagues might then make suggestions via
e-mail, and the test taker might also get input by instant
message from people using screen names like SkyDiver,
JJJunior and TVJunkie. The test taker would be asked to
consider the various sources and suggestions, and to rank
them by relevance to the original request.

Other parts of the test ask students to do everything from
the seemingly mundane (like sorting e-mail messages into
appropriate folders) to head-scratching tasks like
"reordering a table to maximize efficiency in two tasks
with incompatible requirements," according to a brochure.

Asked if she had taken the test herself, Ms. Egan
responded, "What a cruel question.

"I took it earlier on, when there was no way to produce a
score from it. But I knew myself that there was a lot I
needed to learn."

http://www.nytimes.com/2005/01/17/technology/17test.html