CHEonSAT102601

The Chronicle of Higher Education

From the issue dated October 26, 2001

The SAT's Greatest Test

By BEN GOSE and JEFFREY SELINGO

   The SAT is to criticism as a halfback is to a football --
always on the receiving end.

For most of the past two decades, the College Board, which
owns the test, has done a good job holding onto the
ball,fending off critics who maintain that the test
discriminates against female and minority students.

The board has sponsored several studies that show, for
example, that the test usually gives black and Hispanic
students a helping hand in the admissions process -- it
predicts that they will perform better in college than they
actually do. And the board is scrupulously careful about
material that appears on the SAT, so that there will be no
more embarrassments about questions that would seem to favor
students familiar with yachts.

It's hardly surprising, then, that faced with an unprecedented
assault from a wholly different quarter, the test's keepers
resort to the standard defense.

"This is not a biased test," says Gaston Caperton, the College
Board's president, from his office here near Lincoln Center.
"What we have is an unequal educational system. It's not the
kids. It's not the test."

What Mr. Caperton seems to have missed is that today the
battle has shifted drastically, from accusations of bias to
questions that undermine the very basis for the test and may,
in the end, lead to its demise. Today's critics have opened an
assault on the use of what is essentially an IQ test to
measure students' ability to learn. The outcome of the debate
will affect how colleges with competitive admissions pick
students, how racially diverse those students will be, and how
high-school students prepare for college.

The College Board has for years tried to distance the SAT from
its roots in IQ tests, but the perception remains that the
most widely administered college-entrance examination measures
intelligence, not a mastery of learning. Many education
leaders -- most notably, Richard C. Atkinson, president of the
University of California system -- say that legacy creates
perverse incentives, as students waste time and money
"prepping" for the SAT's idiosyncratic questions, such as the
analogies section of the verbal exam.

If college-entrance exams were tied more closely to the
curriculum, the critics say, students would have a clear idea
of what standards they must meet, and high schools could more
easily be held accountable when students fail.

Last February, Mr. Atkinson stunned college leaders by calling
on his nine-campus, 170,000-student system to become the first
public university with competitive admissions to drop the
requirement that applicants take the SAT. The system's size
and prominence immediately led to speculation about whether
others would dump the SAT; the University of Texas at Austin
and North Carolina's public colleges are studying the issue.

Mr. Atkinson's announcement "was by far the most important
single anti-SAT effort ever in the history of the test," says
Nicholas Lemann, author of The Big Test: The Secret History of
the American Meritocracy (Farrar, Straus & Giroux), a recent
book about the SAT.

In the 1940s, the University of California was the first major
public system to require the test. "That was the key to making
the SAT the dominant test," says Robert A. Schaeffer,
public-education director at the National Center for Fair and
Open Testing, also known as FairTest. "If you follow that
historical analogy, you'll find that the key to ending the
dominance of the SAT also lies in California."

Other threats also loom. Thirteen top colleges, including
Harvard University, the Massachusetts Institute of Technology,
and the University of Michigan at Ann Arbor, are participating
in a study to determine if state exams already given to
high-school students may one day replace the SAT in college
admissions. And recent court decisions and referendums may
lead many colleges seeking a diverse student body to lower the
weight they place on the SAT -- or to ignore it altogether.

To be sure, the College Board and the Educational Testing
Service, which engineers the test, aren't exactly on the
ropes. The obsession with identifying -- and getting into --
the best colleges has been as good for the SAT as it has been
for college-guide publishers like U.S. News & World Report.
Roughly 1.3 million high-school seniors per year take the
test, and more than half take it at least twice, yielding an
annual revenue stream of more than $200-million. For now, many
admissions officers continue to rely on SAT scores to compare
students who come from high schools of widely varying quality.
Many colleges also feel they must report high average SAT
scores to the guidebooks, in order to earn top rankings and
keep applications flowing their way.

The SAT also may continue to thrive because the alternatives
to the test are embryonic, too expensive, or lacking in
political support. Mr. Lemann champions a national curriculum,
with a national exam that matches it, but when the topic comes
up on Capitol Hill, "everyone runs and hides," he says. The
state-based exams, which for now merely test whether a student
has mastered the basic skills needed to graduate from high
school, aren't of much use to a college with competitive
admissions. Some large state institutions want to de-emphasize
the SAT by considering a variety of subjective factors in
admissions, such as overcoming adversity, but first they must
find money to hire additional people to handle those reviews.

And while much has been made of the correlation between SAT
scores and family income -- the wealthier you are, the better
your score, on average -- that connection is even stronger on
the standardized subject tests now known as SAT II, formerly
called the Achievement Tests, one of Mr. Atkinson's proposed
alternatives to the SAT.

"I hear a lot of people criticize the SAT," says Kurt M.
Landgraf, president of the Educational Testing Service. "I've
yet to hear what should be put in its place."

First administered in 1926, the SAT was designed to measure
aptitude, or innate mental ability. It became widely used in
the 1940s and 1950s, thanks in large part to James B. Conant,
president of Harvard, who believed that subject-based
achievement tests favored privileged students, whose families
could afford to send them to boarding schools. The SAT, with
its multiple-choice questions and systematic scoring, was seen
as the great equalizer, a test that would allow the country's
future leaders to be tapped based on intelligence rather than
family connections. "The new elite's essential quality, the
factor that would make its power deserved where the old
elite's had been merely inherited, would be brains," Mr.
Lemann writes in The Big Test.

Today, the enthusiasm for intelligence tests has plummeted,
and most colleges claim to put more emphasis on high-school
grades than on either the SAT or its primary competitor, the
ACT.

Still, the SAT, whose two parts, verbal and mathematics, are
each scored on a 200-to-800-point scale, has no shortage of
fans. "The test is the one unchanging benchmark that can
differentiate between those students who get B's at a tough
school and those who coast with A's from an easy school," says
John Maguire, an admissions consultant and former admissions
director at Boston College.

Enrollment managers also know that recruiting students with
high SAT scores is an easy way to improve an institution's
reputation. Dan Lundquist, vice president and dean of
admissions at New York's Union College, says he uses the SAT
as a "scale tipper" with students who are similarly qualified.

But he acknowledges that there's another good reason to do the
tipping. "If you get enough [students] with 50 points higher
than the rest, you can bring your mean SAT score up," he says.

Mr. Maguire concedes that there's a certain hypocrisy at work.
"In public, people will say the SAT's aren't worth a bucket of
warm spit," Mr. Maguire says. "Nonetheless, the boards of
trustees and the college leadership are looking at [them] as
badges similar to the rankings, that indicate that they are
improving."

However, critics of the test -- and many students who take it
-- view the SAT as a black box. "When you say someone has
1100, what does that communicate to a high-school student who
wants to do better?" asks David T. Conley, an associate
professor of education policy at the University of Oregon, who
is directing a project on the use of state tests in college
admissions. "It doesn't communicate anything but to tell the
high-school student to get smarter."

Nor is it clear how to prepare for the exam, which leads many
students to Kaplan and Princeton Review, companies seen as
possessing "tricks" that help students raise their scores.
Even some high-school teachers are taking time away from basic
reading and writing instruction to prepare students for the
SAT -- a practice that Mr. Atkinson criticized.

Bill Wetzel, a freshman at New York University, says the SAT
made some courses during his junior and senior years at New
Jersey's Red Bank Regional High School downright boring. "I
noticed the difference between some classes, where the
teachers and the students were trying to get the highest
scores possible, and classes that emphasized curiosity and
real critical thinking," he says.

Mr. Wetzel, founder of a group called Students Against
Testing, notes somewhat sheepishly that his 1420 score helped
land him at NYU, although he says that if he had to do it all
over again he would attend an SAT-optional college. Now, he
hopes to organize creative protests against the SAT and other
standardized tests. He cites a recent "testfest" in Colorado
in which students read books and played music while their
peers took the SAT in a nearby building.

Using state examinations or subject tests would be an
improvement, Mr. Wetzel says, but he isn't sure they would
solve the core problem. "Both ideas have the potential for
high schools to become factories for a different kind of test
prep," he says.

Mr. Atkinson believes that his proposal, which would require
students to take three SAT II tests (in writing, math, and a
subject of each student's choice), would help students better
understand the relevance of their high-school courses. There
are more than a dozen SAT II exams, each covering a different
subject area.

Some educators want to go even further, by using state
examinations rather than the SAT. Every state requires its
high-school students to undergo some form of statewide
assessment, either at the end of their course work in core
subjects or as a requirement for graduation. Advocates say the
tests could provide a good snapshot of a student's readiness
for college.

Last summer, a North Carolina legislator proposed a bill that
would make public colleges there drop the SAT requirement in
favor of the state's assessment exam. When college officials
balked, saying the tests don't do a good-enough job of
measuring high-level skills, the sides compromised by ordering
a study of the issue by college officials. "If we're going to
require students to take state tests, then we should use them
for college entrance," says State Rep. Gene G. Arnold, a
Republican, who sponsored the bill. "It's unfair to use the
SAT when our standard course of study is not geared toward
preparing students for the test."

College officials elsewhere expect similar pressure from
lawmakers who seek a better return on the hundreds of millions
of dollars spent annually on the state exams. While those
tests were intended to increase the accountability of public
schools, some lawmakers say students have little at stake,
once they realize how easy it is to pass, and thus have failed
to take the tests seriously.

Because admissions officers would need the tests to provide
meaningful information, college officials want a role in the
development of future state tests. Thirteen research
universities are sponsoring a study that aims to help states
design better tests by agreeing on a set of skills needed by
freshmen at their institutions. In addition, the project will
create a database of current state tests, so that colleges can
compare scores on tests in different states if they choose to
use them in admissions decisions. The Association of American
Universities, a group of 61 research institutions, is
coordinating the project.

Without such a database, admissions directors say they will
face a logistical nightmare in trying to make sense of scores
from different states. John Katzman, founder and chief
executive officer of Princeton Review and a supporter of state
tests, says the College Board should take on the role of
comparing state tests, by developing a chart similar to one
that provides equivalencies for SAT and ACT scores.

"If the College Board won't do it, because they're too
committed to saving the SAT, then other people will spring up
that will, among them me," says Mr. Katzman, who favors the
elimination of the SAT. Already, 100 Princeton Review
employees are working on products to prepare students for
state tests.

Wayne Camara, the College Board's vice president for research
and development, says the group has no intention of assuming
that role. He cites a 1999 study by the National Academy of
Sciences that said such efforts were destined to fail.
"Comparing the full array of currently administered commercial
and state achievement tests to one another, through the
development of a single equivalency or linking scale, is not
feasible," the study said.

Mr. Camara notes that any effort to correlate the scores would
be incredibly complex, because some groups -- such as Hispanic
students or women -- may score lower on certain state tests
than they do on others.

College officials say it may come down to how well the state
tests predict freshman grades, which is what the SAT claims to
do. In April, the University of Minnesota-Twin Cities released
a study, financed by the College Board, that found that the
SAT reliably predicts students' academic performance, not only
as freshmen but throughout college.

"We may find that the end-of-course tests are great
predictors, or we may find that they are consistent with
everything else we're doing, in which case why would we have
to add yet another measure that schools have to collect and
send on to colleges?" asks Gretchen Bataille, senior vice
president for academic affairs at the University of North
Carolina system.

Mr. Caperton, the College Board president, says that when
states learn how much it costs to develop fair questions and
ensure security for an admissions test, they will use that
money instead for other priorities, like paying teachers more.
"Could you turn a statewide test into an admissions test?" he
asks rhetorically. "Absolutely. But when you compare what it
costs to what you get, nobody is going to spend their money
that way."

Mr. Conley, the Oregon researcher who is heading up the
Association of American Universities' project on state tests,
says the money is worth spending to give students and high
schools better measures of where they need to improve.

"The SAT is limited in its ability to provide diagnostic
information to schools," Mr. Conley says. "It was never
conceived as a means to bring about systemic improvement or to
close an achievement gap between groups."

University of California officials agree. They acknowledge
that students of different races will probably show the same
performance gaps on the SAT II and even state tests. But those
exams are better suited to closing the gaps and identifying
poor schools, they say. "By having curriculum-based tests, we
can relate [students' scores to] the quality of instruction in
schools," says Patrick Hayashi, associate president of the
University of California system.

The point differential between white and minority students is
also leading to a diminished role for the SAT, simply because
it lays bare the use of racial preferences in college
admissions. In the 2000-1 academic year, white students scored
an average of 1060 on the test, compared with 859 for black
students and 925 for Hispanic students.

Asian-American students were on top, with an average score of
1095.

Many large public colleges, which generally make admissions
decisions based on a formula rather than a subjective
"reading" of each application, have sought to preserve
enrollments of black and Hispanic students by giving them an
explicit bonus in the process. But in several high-profile
legal decisions -- including a ruling by a federal appeals
court in August that struck down the admissions process at the
University of Georgia -- such mechanical awarding of racial
preferences has been successfully challenged as
unconstitutional by white students who were denied admission.

As a result, admissions directors say that colleges may be
forced to gravitate toward a more holistic set of criteria
that recognize a wider range of achievement, such as
leadership and overcoming adversity.

"Frankly, even those schools that are not highly selective are
going to have to put in place procedures that allow for the
reading of all applicants," says Jerome A. Lucido, associate
provost and director of admissions at the University of North
Carolina at Chapel Hill. "The days of looking solely at
students' grades and SAT scores and saying 'they're in' are
coming to an end."

Some very selective public institutions, like North Carolina
and the University of Virginia, have been reading every
admissions folder for decades. Combined, the two universities
receive 32,000 applications annually, and both hire temporary
staff members to read them all. Officials at both universities
admit that taking essays and other written material into
consideration is more subjective than using only grades and
test scores, but they say that each application is reviewed by
two or more readers to ensure consistency.

The Board of Regents at the University of California will vote
on whether to establish a similar process, called
"comprehensive review," next month. (The board won't decide
whether to drop the SAT requirement until next spring.)

The proposed review process would require admissions officers
at the university system's nine campuses to evaluate every
student on a broad array of criteria, including initiative and
hardship.

The super-competitive Berkeley campus has already curtailed
the influence of the SAT. In 1998, two years after the state's
voters approved a referendum banning affirmative action in
public-college admissions, Berkeley officials started reading
every application, some 36,000 last year. Before then, the
first half of available slots for freshmen, about 4,400 seats,
were allocated solely on the basis of SAT scores and
high-school grade-point averages.

Now, those coveted first seats are given out based on a much
broader and less rigid formula. It still includes an
applicant's SAT score and GPA, but adds other academic
factors, such as the strength of the curriculum and pattern of
grades throughout high school. In the first year that Berkeley
relied less on SAT scores, officials found that 25 percent of
the admissions decisions were different than if the old
procedure had been in place. (The remaining half of the
university's freshmen are accepted using the expanded academic
criteria and other personal factors.)

Although the changes in the admissions process at Berkeley
were aimed in part to capture underrepresented minority
students in the wake of the state ban on affirmative action,
the number of black and Hispanic students accepted at Berkeley
has actually fallen since the new procedures were put in
place. University officials say they expected the minority
numbers to drop. But "we didn't have the degree of loss that
we would have had if we didn't put this extra care and effort
into reading every application," says Pamela Burnett, the
university's director of undergraduate admissions.

The College Board, which has long urged colleges not to rely
too heavily on the SAT, supports the UC system's proposed move
to a fuller review -- for reasons that are not entirely
altruistic. "If UC goes for comprehensive review, there's less
pressure to do away with the SAT," says the board's Mr.
Camara. The SAT would become just one of many tools in
evaluating applicants, he says.

But Robert Laird, a former undergraduate-admissions director
at Berkeley, wonders whether the practice of reading every
application will become widespread at large public
institutions, given the cost involved.

"As state budgets across the country shrink sharply, it's
going to be difficult to generate the legislative and
institutional financial support to put significantly more
money into the undergraduate admissions process," he says.

In the late 1990s, some states facing bans on affirmative
action discovered a cheaper way to ensure that their public
campuses remained racially diverse. California, Florida, and
Texas now automatically admit large numbers of students based
solely on class rank, essentially making SAT scores
irrelevant. Texas' public colleges began guaranteeing spots
for the top 10 percent of the state's high-school classes in
1997, and California and Florida followed in 1999, holding
slots for the top 4 percent and 20 percent, respectively. The
policies take advantage of the many high schools that are
predominantly black or Hispanic.

Now, the University of Texas at Austin is considering whether
the SAT is even worth requiring for the half of the freshman
class not automatically admitted under the state's 10-percent
policy. A faculty committee convened by the university's
president, Larry R. Faulkner, started meeting last month to
study the fairness of standardized tests in Austin's
admissions process.

The trend toward "x-percent plans" and more-subjective
admissions policies hasn't escaped the gaze of those who
uphold high standards or oppose racial preferences. Michael
McIntyre, a professor of organizational and industrial
psychology at the University of Tennessee at Knoxville who has
studied the SAT, says that if the goal at the University of
California is to maintain diversity, officials should be
honest about their motives. "A change in policy is very likely
to change the demographic mix," Mr. McIntyre says. "If that's
the goal of the the change, then let's be upfront about it.
Let's admit it if we are trying to achieve some social end,
rather than an academic end."

Mr. Lemann, the author, supports affirmative action, but
cautions that college officials may be fooling themselves if
they think a subjective admissions process will   shield them
from lawsuits. He says that legal groups leading the fight
against racial preferences, like the Center for Individual
Rights, would have a more difficult time proving that such
policies discriminate against white students, but in some
cases would still be able to do so. "The Center for Individual
Rights has gone after the low-hanging fruit," Mr. Lemann says,
"and there's a lot of it."

The College Board and ETS, meanwhile, occupy a strange
position in the affirmative-action debate. Both are firmly
within the academic establishment, which strongly defends the
use of racial preferences in admissions. The College Board is
a membership organization that represents colleges and
schools. ETS, meanwhile, lures top researchers, in part, by
providing college-like surroundings on its verdant 360-acre
campus outside Princeton, N.J. Partly because of the scrutiny
of affirmative action, both groups have counseled colleges to
avoid relying too heavily on test scores.

But now that some colleges are not even considering the SAT in
admitting large portions of their students, the traditional
alliances may be breaking down. Mr. Landgraf, the ETS
president, for example, views the "x-percent plans" as a blow
to high standards, even though they are a handy tool for
maintaining black and Hispanic enrollments. "Are the interests
of the states best served by silencing the debate," Mr.
Landgraf asks, "or by doing what's right?"

Mr. Landgraf and nearly everyone else agree that the best way
to end the debate would be to eliminate the performance gap
between students of different races. Most of those who use the
x-percent plans admit privately that they would go back to the
SAT in a minute, were it not for those racial gaps. But
despite considerable hand-wringing, and efforts at outreach,
the gaps between white, black, and Hispanic students have
barely budged in more than a decade.

The College Board sponsors a good deal of research aimed at
broadening the scope of skills measured by the SAT, and much
of that work has the secondary goal of finding measurements
upon which black and Hispanic students will score well.

Robert Sternberg, a Yale University psychology professor who
believes the SAT should be expanded to measure creative and
practical skills, is among the most prominent scholars
receiving funds from the board. To measure creativity, Mr.
Sternberg suggests showing students a single-frame cartoon and
asking them to write a caption for it. To measure practical
ability or common sense, he would include a reading about a
real-life dilemma that teenagers might face, and ask students
to identify the best way to handle the situation.

Mr. Sternberg acknowledges that grading the answers would be
more expensive than it is now; particularly on the creative
questions, a machine would not be able to do the job. But the
College Board could hire readers, as it does to grade essays
on Advanced Placement tests. The readers would use examples
that would illustrate what types of captions might earn top
scores for creativity.

"By measuring a broader range of abilities, you would no
longer need affirmative action," Mr. Sternberg says. "Some of
the kids who grow up in culturally different environments have
to develop creative and practical skills to survive. If you
grow up in a white, upper-middle-class environment, you don't
need those skills as much."

He and a team of researchers are conducting a study that
involves roughly 1,000 college students at 16 sites. "So far,
our results are in line with our hypothesis -- that we can
find better ways of predicting success," he says.

Those sorts of measures generally draw howls of protest from
critics of affirmative action, and many of the
psychometricians involved in creating the SAT are equally
dubious.

"I'd be a little concerned that you could coach someone to
pretend to be creative," says Thomas Van Essen, who develops
verbal tests at ETS. "Think of those kids you knew in high
school who were 'creative' -- they all acted the same."

The testing giant's own efforts to narrow the gap in test
scores have elicited similar skepticism. In 1999, ETS
officials revealed that they were working on a project that
would help admissions officers measure disadvantage, by
identifying as a "striver" any student who scored more than
200 points above the average score of students from a similar
background. The scale took into account 14 variables, such as
family income and parents' education, but its developers noted
that the only way to achieve "a student body that mirrors the
racial composition of the U.S. population" was to use race in
the process.

The effort was excoriated by opponents of affirmative action,
and quickly condemned by Mr. Caperton. Last month in
Princeton, a table of six ETS officials briefly fell silent
when the topic was broached.

"We didn't feel there was sufficient technical quality behind
it," says Drew H. Gitomer, senior vice president for
statistics and research."While the intent of the study was
noble, we didn't feel there was a whole lot there."

Meanwhile, the College Board's efforts to deal with its other
problem -- the perception that the SAT has no link to the
curriculum -- can best be described as modest. For years, it
reacted mainly with semantics. In 1994, it changed the name
"Scholastic Aptitude Test" to "Scholastic Assessment Test," to
suggest a measurement of educational accomplishment rather
than innate ability. A few years later, it shortened the name
to just "SAT."

College Board officials insist that the test has responded to
changes in the curriculum over time. They note that they began
permitting calculators on the math exam in 1994, to reflect
the practice of many schools. The same year, they killed a
section of questions on antonyms, to answer critics who
suggested that it was encouraging schools to spend too much
time on vocabulary devoid of context.

Similar criticisms are now being leveled at the analogies
section, and the College Board is studying whether it would be
feasible to replace that part with questions on a short,
high-level reading passage.

"The notion that this test hasn't changed in 50 years is
completely untrue," says Amy Schmidt, the College Board's
director of higher-education research and educational
evaluation.

Somewhat belatedly, the board is also beginning to help test
takers figure out what skills they need to improve to raise
their scores. This fall, for the first time, the board is
sending score reports to students who have taken the PSAT,
which high-school sophomores and juniors take as a warm-up to
the SAT.

The reports list specific skills that each student needs to
work on -- such as understanding the main ideas of a reading
passage, or applying the rules of algebra and geometry. There
are no immediate plans to provide such reports to students who
take the SAT.

"Maybe we could have done a little better job over the years
of communicating what these questions do measure," concedes
Mr. Gitomer.

Whether those efforts will be enough to save the SAT remains
to be seen. The best hope for the test may be that it's so
entrenched in the admissions process.

Mr. Conley, the advocate for state tests, doesn't expect them
to be required on college applications anytime soon. "The
college-admissions process is a conservative system, and we're
talking about a brand-new infrastructure that doesn't exist,"
he says. "It's like setting a massive ship off in a different
direction, and that's never easy or quick."

When a committee studying whether to eliminate the SAT
requirement at the University of Texas first met last month,
several professors wondered what would replace the test if its
use were abolished. They feared that a process without
standardized tests would lead to inconsistent admissions
decisions.

The SAT is still required at the highly selective public
colleges that give every application a thorough read. John A.
Blackburn, Virginia's dean of admissions, says the university
does not plan to drop the requirement.

"In a major system, where you have to make decisions about a
lot of people and where you're responsible to the public, you
must have some norm that cuts across high schools," Mr.
Blackburn says. "Until we have something better, the SAT is
really the only instrument that achieves that."

Andrew Brownstein and Eric Hoover contributed to this article.
HOW AN SAT QUESTION IS CREATED

Psychometricians at the Educational Testing Service, which
creates and administers the SAT, are skeptical that state
examinations could be used in place of the SAT.

Admissions tests, ETS officials argue, must be unambiguous,
error-free, fair, and secure -- qualities that are more easily
stated than achieved. Here's how a draft question makes it on
to the SAT:

Step 1: An ETS staff member or outside contributor drafts a
question and puts it in a database.

Step 2: The question is scrutinized by at least two subject
experts and given a fairness review to ensure that is "free
from bias." Roughly 10 percent of questions are rejected at
this stage.

Step 3: The question is tested on an unscored section of an
SAT exam. The test-takers have no idea that the question will
not be factored into their scores.

Step 4: Based on that "pre-test," ETS staff members analyze
the rigor of the question, and how well it distinguishes
between students of differing abilities, based on their scores
on approved questions. They also examine whether, among
students of comparable abilities, those from a particular race
or gender scored poorly on the question. In a recent analysis,
only 76 percent of verbal questions, and 87 percent of math
questions, survived this stage.

Step 5: The question is thrown into a pool with thousands of
other approved questions. It could sit there for as long as
three or four years before being tapped for an SAT exam.

Step 6: When it earns a spot on an SAT exam, the question is
analyzed by ETS staff members to see how it fits in
statistically with other questions on the exam. Then three to
four in-house subject experts review and revise the question.
Finally, five to eight outside experts, including members of
the SAT Committee, a group that includes admissions officers
and guidance counselors, review the entire test that the
question will appear on.

Step 7: After further editing and proofreading, the test is
printed. The whole process -- from the conception of a
question to its appearance on an exam -- takes at least 18
months to two years.

SOURCES: Educational Testing Service; Chronicle reporting
SAT BY THE NUMBERS

SOURCES: College Board; ETS; Chronicle reporting

CAN THE SAT MEASURE COMMON SENSE?

Robert Sternberg, a Yale University psychology professor,
believes the SAT should be expanded to measure practical
abilities, such as knowing how to get along with others.
Here's an example of how such a question might be worded:

You are enrolled in a class you are not excited about, but you
do care about your grade and, so far, you have been doing
fine. The class has a short paper due tomorrow. You have been
putting off working on this paper because writing will bore
you. You have been planning to work on it tonight, but you
just received a phone call from your friend who wants you to
go to this really cool party. You have heard about the people
who are throwing the party, but you have never met them. You'd
love to go. Given this situation, rate the quality of the
following options:

___ a) Get the address of the party from your friend, get your
paper done quickly, and show up at the party later than
everybody else.

__ b) Go with your friend, meet the people, and leave in an
hour or so to get your paper done.

__ c) Go with your friend and see how you like it there; make
your decision regarding the paper on the spot.

__ d) Explain to your friend that you cannot go be   cause
you have a paper due.

__ e) Go, enjoy yourself, and then stay up all night to work
on the paper.

__ f) Go to the party and then come up with an excuse so that
you can get permission to turn the paper in late.

__ g) Go, stay for a couple of hours, sleep for a couple of
hours, then get up really early to write the paper.

__ h) Get the address of the party from your friend, tell
him/her that you'll be there, and then never show up (get your
paper done).

SOURCE: Robert Sternberg, Yale University
ANSWERS (based on experts):

a: 4   b: 4   c: 2 d: 6 e: 3 f: 1 g:
4 h: 5

_________________________________________________________________

This article from The Chronicle is available online at this address:

http://chronicle.com/free/v48/i09/09a01001.htm